Revolutionizing Network Management with AIOps
This summary of Enterprise Management Associates (EMA) research explores how IT organizations are planning to use AIOps technology to enhance network infrastructure management. Based on a survey of IT professionals with extensive knowledge of using AIOps for network management, this research identifies top use cases, product requirements, barriers to success, and best practices.
AIOps is an abbreviation of the phrase “artificial intelligence for IT operations.” AIOps combines machine learning and artificial intelligence algorithms with big data and other technologies to enhance IT management. The definition of AIOps varies from vendor to vendor, but the consensus view is this: AIOps technology can find patterns in IT data, infer insights and draw conclusions from those patterns, and communicate this knowledge to IT management. Ambitious IT organizations also use AIOps to drive automation.
An industry of AIOps specialists has emerged over the last few years that takes a domain-agnostic approach to IT operations. AIOps specialists consume and analyze data from all layers of the IT technology stack to address cross-domain management use cases. EMA recently published a Radar Report on this domain-agnostic AIOps industry.
EMA has also observed robust AIOps development within the networking industry. Network infrastructure vendors and network management vendors have invested in their own homegrown AIOps technologies to enrich their solutions by training them specifically for network management use cases. Moreover, EMA research has detected strong interest in using this technology.
This research explores how the network infrastructure team in particular is using or planning to use AIOps for network management use cases.
Research Methodology and Demographics
EMA surveyed 309 IT professionals about their opinions and experiences with the application of AIOps to network management. Twenty percent of survey participants indicated that they have conducted extensive research on the subject, and 80% indicated that they have hands-on experience with AIOps-driven network management. EMA supplemented this survey data with one-on-one interviews on the subject with enterprise network infrastructure professionals.
Figure 1 reveals more about the roles these people play in their IT organizations. The majority of them work in middle management positions (e.g., IT director or supervisor). Nearly 20% are IT executives (e.g., CIO or vice president of infrastructure), and the rest are technical specialists (e.g., administrators, engineers, and architects). Figure 1 also shows that 30% of these individuals work in network operations or IT operations. Nearly one-quarter work in the IT executive suite. Many others work in security, network engineering, and IT architecture.
62% Middle IT management
19% Executive IT management
18% Technical specialists
30% Network operations/IT operations
23% IT executive suite
17% Information security/cybersecurity
15% Network security/cybersecurity
13% IT architecture
3% Data center operations
Figure 1. Job titles and IT groups
Figure 2 reveals more about the companies these people work for. The majority of these people work in large enterprises, with 1,000 to 9,999 employees. Nearly one-quarter work for midmarket enterprises. This research also offers a transatlantic view of the topic, with 63% based in North America and 37% based in Europe.
Finally, this research surveyed a variety of industries. The most prominent are manufacturers, IT consulting and professional services firms, software companies, finance and insurance companies, retailers, and healthcare organizations.
Company Size (Employees)
28% Midmarket enterprise (500 to 999)
57% Large enterprise (1,000 to 9,999)
15% Very Large enterprise (10,000+)
63% North America
16% IT professional services/ consulting
7% Healthcare/Medical/ Pharmaceutical
Figure 2. Corporate profiles
The key findings of this research include:
- 91% of enterprises believe AIOps can help overcome challenges with net- work management toolsets, but the most successful users of AIOps-driven network management are less likely to feel this way.
- Security and compliance risk is the most significant challenge to using AIOps for network management. Network complexity and data quality are also major problems.
- Only 41% of enterprises believe they are fully effective at evaluating the AIOps solutions they apply to network management.
- 80% of enterprises are interested in AIOps solutions that can anonymously combine their network data in the cloud with data from other enterprises for peer analytics and benchmarking. However, many have security and compliance concerns that will prevent them from adopting this.
- Priority use cases for AIOps-driven network management are prevalent.
- Anomaly detection
- Automated security incident remediation
- Intelligent alerting and escalation
- Automated IT service problem remediation
- Enterprises are split on whether AIOps can enable true, closed-loop network automation.
Opportunities and Challenges with AIOps-Driven Network Management
Success and Failure with Early Efforts
Although AIOps is still in early days, it’s never too early for IT organizations to assess their progress with a new technology. Figure 3 reveals that only 30% of research respondents feel that their organizations have been completely successful with applying AIOps to network management so far. More than half see room for improvement, and a small number say they are unsuccessful overall.
Improving Business Outcomes
Ninety percent of the participants in this research agree that using AIOps for network management can lead to better business outcomes for an enterprise, and 56% strongly agree with this sentiment. Figure 4 shows sharp divisions based on one’s role in an IT organization. Executives and middle managers strongly agree with the idea, but technical staff, such as engineers and architects, are less likely to strongly believe in the business benefits of AIOps. It’s possible that technical staff believe in their own hard-won skills as network infrastructure professionals than they do in the ability of a set of algorithms to intelligently enhance network management.
Individuals with hands-on experience with AIOps solutions are more likely to believe in the potential business benefits of AIOps, as are organizations that are effective at evaluating AIOps technology.
Figure 5 reveals specific benefits enterprises are hoping to achieve with AIOps-driven network management. Research participants identified five top opportunities: network optimization, operational efficiency, improved security and compliance, network resiliency, and cost reduction.
A network architect with a $10 billion retail enterprise told EMA that he sees an opportunity to improve operational efficiency and network resiliency.
We have lots of [diverse infrastructure] that pumps out metrics that go to various monitoring tools of varying levels of value, scalability, and usability. They all produce trends, alerts, and reports, and it’s a massive mess. All that stuff is swirling. The way I understand AIOps, it has the ability to look at all these toolsets, dump it all into machine learning, alert on it, act on it, and fix things.
Enterprises that are successful with AIOps are more likely to be targeting improved security and compliance. Individuals with hands-on experience with AIOps are less likely to target an edge use case with improved responsiveness to change. Middle IT management is more likely than technical staff to target operational efficiency. Europeans are more likely than North Americans to use AIOps to free up skills staff for strategic projects.
Challenges with AIOps
Business Challenges with Using AIOps for Network Management
As Figure 6 reveals, 90% of enterprises have encountered some kind of business issue while trying to apply AIOps solutions to network management. The most common issue is security and compliance risk. AIOps solutions can challenge an organization’s overall risk tolerance in a variety of ways. If the solution is cloud-delivered, many regulated industries will be reluctant to embrace it. In fact, financial companies are more likely to cite security and compliance risk (57%).
Budget limitations and skills gaps are the chief secondary business challenges. Later in this research, EMA will explore AIOps-related budget plans and training priorities. Respondents from network operations and IT operations groups were less likely (20%) to be concerned about budget, but IT executives (41%) and security professionals (37%) were more likely to worry. Very large enterprises (10,000 or more employees) also cited budget constraints more often (42%).
More than one in five participants complained that AIOps implementations for network management take too long.
A network architect with a $10 billion retailer told EMA:
We’ve tried to get it implemented, but it hasn’t been done. It’s not easy to implement and build out, and I think that’s where it got bogged down in my company.
A network operations manager with a $10 billion high-technology manufacturer told EMA:
The total cost of ownership is a big concern. Maybe I spend $15,000 to install a solution, but it costs me $100,000 a year to keep it running and delivering value. How much storage will the data consume? Also, complexity of deployment is a problem and complexity of maintenance. I worry that I will have a team of two or three people keeping it running.
The least common problem is the lack of clarity around a business case for adoption. This finding suggests that IT organizations generally recognize the potential of AIOps to enhance and possibly transform IT management. Individuals from network operations and IT operations teams were especially unlikely to see a lack of business value (9%).
Technical Challenges with Using AIOps for Network Management
Figure 7 reveals the technical issues that can obstruct AIOps use. Overall, 86% have encountered at least one technical roadblock. Two issues are most prominent. First, many IT organizations believe networks are simply too complex for AIOps solutions to fully understand them. Second, enterprises are struggling with data quality. The data collected by their monitoring and management tools needs to be improved.
All other technical issues are secondary, only significant to fewer than 20% of enterprises. Enterprises that have experienced success with applying AIOps to network management were less likely (10%) to struggle with auditing and verifying AIOps outcomes. Very large companies are more likely to struggle with these audits (29%). IT executives were more likely to complain of product quality (25%) and deployment options that don’t meet requirements (25%). Europeans were also more likely to complain of product quality (23%).
Overall AIOps strategy has some impact on usability. For instance, enterprises that take a cross-domain approach to AIOps, adopting tools that support IT management in general, are less likely (12%) to struggle with usability of the solutions. Enterprises that allow network teams to adopt their own AIOps solutions specifically aimed at network management are more likely to complain of usability (25%).
AIOps Initiatives: Planning and Implementation
Evaluating AIOps Technology for Network Management
Figure 8 reveals that only 36% of enterprises feel they are entirely effective at evaluating AIOps solutions for network management. Forty-one percent are only slightly effective, and 12% feel they are ineffective. There is a strong correlation between one’s ability to evaluate AIOps and success with the technology.
Seventy-four percent of successful AIOps projects are very effective at evaluating AIOps, versus 25% of somewhat successful AIOps projects. Sixty percent of unsuccessful AIOps projects reported that they were ineffective at evaluating AIOps.
A network operations manager with a $10 billion high–technology manufacturer told EMA:
I would say we are average at best, ill-equipped at worst, [for evaluating AIOps]. We have to be able to understand whether it’s reliable and its conclusions are correct. What we run into with every evaluation [of AIOps] is that every vendor has their own twist on how the data is ingested or presented. Because of the abstraction and the secret sauce, it’s hard to say what is superior to what. I need a reliable measurement of accuracy.
Enterprises that are very effective at evaluating AIOps solutions are also more likely to:
- Trust AIOps to automate networks
- Prefer AIOps solutions that are designed specifically for network management, rather than cross-domain management
- Prefer AIOps solutions that are specific to a single network infrastructure vendor’s products
Deployment and Administration of AIOps Solutions
Many AIOps solutions have significant compute requirements, and some vendors deploy some or all of their solutions via the cloud, where they can scale the solution gracefully. In conversations with network managers, particularly those in heavily regulated industries, EMA analysts have heard anecdotally that cloud-based AIOps solutions present an unacceptable security risk. Thus, EMA asked research participants to describe their deployment preferences for AIOps. Figure 9 reveals that only 17% of enterprises require an on-premises deployment for AIOps. Instead, the majority of companies prefer a hybrid solution that is deployed across the cloud and on-premises resources. More than one-quarter of enterprises prefer a solution that lives exclusively in the cloud.
Although hybrid deployments of AIOps are preferred, EMA’s data suggests that this isn’t a best practice. Hybrid deployments are more popular among somewhat successful AIOps implementers (64%) and less popular among more successful ones (46%). A pure on-premises solution is favored more often by successful organizations (21%) and less by somewhat successful ones (10%). On the other hand, individuals with hands-on experience applying AIOps to network management are more likely (55%) to favor hybrid deployments than those without (39%).
Figure 10 reveals preferences that enterprises have for the administration of AIOps solutions. More than half want to administer and maintain these solutions themselves, while more than one-third want a managed solution.
Fully managed solutions are the preference of successful AIOps users (51%), versus just 31% of somewhat successful organizations. Technical staff (55%) and middle managers (56%) are more likely to require an internally administered solution, while IT executives are less likely (37%) to want one.
Figure 11 explores the factors that influence deployment and administrative requirements for AIOps solutions. As alluded to earlier, security and compliance risk is the major driver of these preferences, particularly IT executives (53%) and information security professionals (71%). Network architecture and engineering professionals are less driven by risk (39%).
Cost, implementation complexity, and time to value are the chief secondary drivers of these requirements. Implementation complexity is more influential among research participants who lack hands-on experience with the technology (40%). Time to value is a bigger factor for network architecture and engineering professionals (36%), but less pressing for IT executives (18%) and information security (18%).
Network-Specific Versus Domain-Agnostic AIOps
While there is a growing market of vendors who offer domain-agnostic AIOps solutions that can optimize IT management across an organization, EMA has also observed a growing number of net- working vendors enriching their own products with network-specific AIOps technology. Figure 12 reveals which flavor of AIOps enterprises prefer to apply to network management. There is slightly more interest in domain-agnostic AIOps than net- work-specific solutions, but the largest number of enterprises wants to use a combination of both types of solutions. This indicates that many enterprises expect to use multiple AIOps products to enhance network management, from both net- working vendors and general AIOps vendors.
The research found that individuals who possess hands-on experience with AIOps are more likely AIOps solutions designed specifically to address network management use cases to prefer network-specific solutions. These people may have had a bad experience with domain- agnostic products, but it’s more likely that their incumbent networking vendors enhanced compulsory products with AIOps capabilities, giving many people early experience with and a bias toward these network-specific solutions.
IT executives are more likely to want to use a combination of both types of solutions. Large enterprises also have this preference and so do North Americans.
General AIOps Product Requirements
Figure 13 details the top general requirements that enterprises have for AIOps-driven network management solutions. The top two are integrations with network management and ease of use. North Americans were more likely than Europeans to rate both of these requirements as important. Ease of use is more important to middle managers (31%) than IT executives (15%).
Many enterprises also require a large volume and variety of AI training data, a breadth of analytics and machine learning techniques, and proven solutions that work in “my environment.” Successful enterprises are more likely (30%) to set requirements around the volume and variety of AI training data, a breadth of analytics and machine learning techniques, and proven solutions that work in “my environment.” Successful enterprises are more likely (30%) to set requirements around the volume and variety of data used to train AI.
Fluency with diverse ranges of data types, multi-vendor support, affordability, and low administrative overhead are the least important.
Verifying AIOps Insights
Sixty-four percent of enterprises will require a workflow from their AIOps vendors that allows them to verify the insights and conclusions that these solutions provide about network management, as Figure 14 details. Another 35% think a workflow would be helpful. Only 1% perceive no value from this capability.
Successful users of AIOps-driven network management are more likely to require these workflows (77%), as are enterprises that are very effective at evaluating AIOps solutions (81%). IT executives (80%) also tend to set this as a requirement over those further down the organization. This requirement is more common among enterprises that have cross-domain AIOps strategies (70%). AIOps implementations that are driven solely by the network team are less likely to set such a requirement (51%).
EMA also asked survey respondents if these work- flows help the network operations teams trust AIOps technology. Eighty-six percent affirmed this idea, including 42% who strongly agreed. Respondents with hands-on AIOps experience were more likely to strongly agree with this notion. Notably, only 34% of individuals from the network operations team strongly agreed. Thus, there may be some disconnect on this topic.
Spotlight: AIOps and the Data Fed to it
Data is essential to AIOps. The technology uses data to learn, and it uses ongoing analysis of data to apply what it has learned to IT operations. Earlier in this report, EMA noted that data quality is the number-two technical challenge that enterprises encounter when applying AIOps to network management. This section explores the issue of data from multiple perspectives to help enterprises understand the data requirements they will encounter when adopting AIOps-driven networking solutions.
Essential Network Data
Figure 15 reveals the types of network data that enterprises believe are most important to AIOps. This is the data that they want their solutions to collect and analyze. There are three major priorities:
- Network flows
- Network configuration files and data
- Core network services data (e.g., DNS queries and records, IP address space, DHCP handshakes)
Configuration data is especially important to enterprises that are successful with AIOps (47%) versus just 33% of somewhat successful organizations. Core network services data is a priority for network engineering and architecture professionals (41%). Wi-Fi metrics, data extracted from proprietary controllers (e.g., an SD-WAN controller), and device metrics collected via SNMP and other protocols are secondary data priorities. Device metrics are more important to successful organizations (33%).
Streaming telemetry is a lower priority, but it is also a new source of data that is not widely deployed across the industry. As it matures and makes its way into more enterprises, its value to AIOps may increase. Interest in logs is also low, but Europeans (16%) have a bit more interest in using it.
Essential Non-Network Data
Figure 16 identifies the sources of non-network data that enterprises consider essential for network-focused AIOps. Cloud provider performance data is the biggest priority, which suggests that enterprises are relying on AIOps to correlate network and cloud performance. IT executives and middle management are more likely to prioritize this data than technical staff.
User data, service desk tickets, and cloud provider cost data are also significant priorities. Data extracted from social media, ChatOps solutions, and CMDB solutions are the lowest priorities. Of course, CMDB data overlaps with the more important configuration data pulled from network devices or network management systems.
Successful AIOps initiatives are more likely (40%) to prioritize topology and application dependency data, as are information security professionals (41%). IT executives are more likely (35%) to value data from DevOps tools. Technical staff, such as engineers and architects, are more likely to value social media data (25%).
Peer Analysis with AIOps
Providers of AIOps solutions can enhance their technology by feeding it larger and more diverse sets of data. Analysis of data across different customers’ networks can make such solutions more sophisticated. They extract more sophisticated insights by comparing events in one network to events observed over time in other networks. They can also present valuable business intelligence to network managers by showing them benchmarks. For instance, how does one investment bank’s net- work performance compare to other investment banks in the city?
Figure 17 reveals that 80% of enterprises are interested in this peer analysis. Successful AIOps users are even more interested (95%). Enterprises that consider themselves very effective with AIOps solution evaluation are also highly interested (90%). IT executives (90%) are more interested than technical staff (73%).
Midsized enterprises are more interested (87%) than very large enterprises (67%). Software companies (90%) and IT services and consulting firms (98%) are very likely to want peer analysis, but government agents (56%) and healthcare organizations (55%) are less open to this capability, which brings up the issue of security and compliance.
Figure 18 reveals that 33% of enterprises cannot allow their data to be uploaded to the cloud for peer analysis due to security or compliance concerns. Nearly half are willing to do this, but they will require strong assurances from vendors about the security of their platforms. Only 15% have no concerns, since the data is typically anonymized.
IT executives are more likely to have no concerns (28%), versus 13% of technical staff and 12% of middle management.
This section explores how enterprises plan to use AIOps solutions for network management. It reveals which uses enterprises are prioritizing and which they are not. It looks at what parts of the network enterprises want to manage with AIOps solutions. It also reveals training priorities that enterprises will have as they prepare their network teams to use the technology.
A network operations manager with a $10 billion high-technology manufacturer told EMA:
[Data mining and anomaly detection] are a priority. We had an issue where we were trying to understand a network flow that looked like a DDoS attack. We had a look at it through our data lake, and it took three hours to get an answer. In parallel – as an experiment – I had someone tackle the same problem with our AIOps tool, which was able to do it in 22 minutes. It reduced the amount of time that we spent simply structuring queries and mining data.
Priority Use Cases
Figure 19 reveals which generic AIOps uses enterprises are prioritizing or use the most when applying the technology to network management. Anomaly detection, automated security incident remediation, intelligent alerting and escalation, and automated IT service problem remediation are the most popular.
Automated escalation is a significant priority for very large enterprises (64%) and for information security professionals (63%).
Enterprises that are the most successful with AIOps were more likely to prioritize more use cases. The biggest gaps, where successful organizations were more likely to prioritize a use case than less successful organizations, were capacity optimization, automated vendor management, and intelligent alerting.
On the subject of intelligent vendor management, a network architect with a $10 billion retail enterprises told EMA:
I care about routers and switches as a network architect. I would like the ability to match up inventory management with product life cycles so that an AIOps tool could reach out to my vendor when hardware is end of life or when PSIRTs and bug fixes come out. That’s better than getting a spreadsheet from my vendor. It would improve maintenance.
Figure 20 reveals the use cases that are least likely to be a priority at all. Change management, event correlation, and automated vendor management are the biggest non-priorities. However, as noted, successful enterprises are more likely to value automated vendor management.
Closing AIOps Skills Gaps in the Network Team
Figure 21 reveals the training priorities that enterprises have set for the network teams to prepare them for implementing and using AIOps. Data management is the top priority. Enterprises want to make sure that network teams are capable of feeding high-quality data into AIOps solutions. Earlier in the research, EMA found that poor data quality is a major technical challenge when applying AIOps to network management. Data management skills will be critical to resolving this problem.
Knowledge of AI and machine learning algorithm development is the second priority. This suggests that some enterprises are developing AIOps capabilities internally, or at least want to modify commercial solutions in some way, perhaps via vendor APIs. API skills are right behind algorithm development on this list of priorities. Algorithm development (51%) and API skills (47%) are both higher priorities for successful organizations. However, technical staff, such as engineers and architects, consider algorithm development a low priority (18%).
The ability to evaluate AIOps technology is right behind API skills. This priority, combined with the development expertise mentioned before, tracks with EMA’s earlier finding that many enterprises plan to audit AI and machine learning algorithms when evaluating AIOps solutions.
Finally, tool-specific training is the last significant training priority.
A network architect with a $10 billion retainer commented on tool-specific training.
I think some of our people can handle working with AIOps, but not a lot of them. In networking and security, there aren’t enough people who could do the provisioning and configuration and understanding this stuff.
AIOps and Automation
This section explores the idea of driving network automation via AIOps. Many enterprises are open to the concept of closed-loop automation, but their comfort with fully embracing it is low. Network managers have always had reservations with turn- ing over control of a network to a machine.
Trusting AIOps-Driven Automation
EMA asked respondents whether they trust AIOps to automate three general network operations tasks:
- Automated remediation of suspicious or malicious activity
- Automated remediation of network availability and performance problems
- Automated capacity management
EMA asked respondents to say whether they trust AIOps to automate the above three tasks at two levels:
- Network changes that have minimal impact on the network
- Network changes that have significant impact on the network.
Figure 22 reveals that enterprises are more likely to trust all three of these tasks, as long as the potential impacts on the network are minimal. Trust drops slightly when those automated changes have a potentially larger impact on the network. However, overall, trust in AIOps-driven network automation is high.
Figure 23 moves the conversation up a level, from the automation of individual tasks to closed loop automation. EMA asked respondents whether they agree with the following statement:
Networking is too complex to fully automate with AI technologies.
The industry is very split on this idea. Forty-two per- cent generally believe networking is too complex for closed-loop automation via AIOps, and 39% generally disagree. Nearly 20% are unsure. Organizations that are successful with AIOps are more likely to have extreme points of view on both sides of this issue. They are more likely to strongly disagree that the network is complex (28%), but they are also more likely to strongly agree that the network is too complex (24%). This points to a fundamental disagreement on the future of network automation.
A network architect with a $40 billion pharmaceutical company told EMA:
We’ve been talking about automation for the last five years, and managers are just so afraid to automate five lines of configuration on 300 routers at same time—even if those lines would have no impact on production. Our environment is highly regulated, so it just isn’t possible. If you want to change one device, you have put in a ticket. And the earliest you can do the change is two weeks out.
This research found that enterprises see great potential for the application of AIOps to network management. Most of the subject matter experts in this research believe that it can lead to better business outcomes for their enterprises. Nine out of ten believe that AIOps can address many of the shortcomings of their existing network management solutions. They are also enthusiastic about their ability to automate much of their networks and to streamline operations.
However, AIOps-driven network management has plenty of room for improvement. Only 30% of the enterprises in this survey have been fully successful with this technology so far. Most of them see room for improvement. They want to see vendors advance and mature their capabilities, particularly around predictive analysis, root-cause analysis, network baselining, and anomaly detection.
Enterprises themselves need to retool. Most of them expect that they’ll have to upgrade at least some parts of their network infrastructure to maximize the value of AIOps. They also have to train personnel to close skills gaps within their organizations and improve solution evaluation and implementation. Finally, this research found that data quality is a major issue for the successful use of AIOps for network management. If network management tools are collecting and storing bad data, this will lead to poor outcomes in AIOps solutions. Network management organizations have a lot of work ahead of them if they want to realize the full potential of AIOps.
EMA will continue to track AIOps closely across all of its research practice areas, including networking. EMA analysts see great potential in the concept of AIOps, but also acknowledge the risk for disappointment. Enterprise decision-makers should review this report carefully, particularly the best practices uncovered among the most successful users of AIOps.
About Enterprise Management Associates, Inc.
Founded in 1996, Enterprise Management Associates (EMA) is a leading industry analyst firm that provides deep insight across the full spectrum of IT and data management technologies. EMA analysts leverage a unique combination of practical experience, insight into industry best practices, and in-depth knowledge of current and planned vendor solutions to help EMA’s clients achieve their goals. Learn more about EMA research, analysis, and consulting services for enterprise line of business users, IT professionals, and IT vendors at www.enterprisemanagement.com. You can also follow EMA on Twitter or LinkedIn.