Network monitoring in the enterprise has never been easy. Even before organizations began moving software and infrastructure to the cloud, a typical enterprise used four to 10 tools just to monitor and troubleshoot their own networks, according to analyst and consulting firm Enterprise Management Associates.
The public cloud adds another complex wrinkle to network visibility. Traditional monitoring tools center around the health and performance of individual network elements. Today’s digital business era requires a more holistic view of networks with the ability to glean and correlate data from diverse cloud environments using big data analytics and machine learning.
Today 40% of organizations consider themselves multi-cloud users, having two or more cloud service providers for their organization, according to a survey by Kentik. One-third of organizations have a hybrid-cloud environment, with at least one cloud service provider and some type of traditional infrastructure that is company-owned, co-located, or a third-party data center.
“There’s a lot of different types of data that people collect and analyze on the network – everything from device metrics to NetFlow to packets to logs to active synthetic monitoring, and no one vendor does it all very well. Most of them don’t even try to do all of it,” says Shamus McGillicuddy, research director at EMA.
As a result, 35% of multi-cloud users have three to five monitoring tools, including log-management tools (48%), application-performance-management tools (40%), open-source tools (34%), and network performance management tools (25%).
“Network people tell me they just can’t find end-to-end tools. They have a really good view of the data center, a good view of AWS, a good view of Azure, but they can’t pull it all together,” McGillicuddy says.
“The environment is getting far more complex,” says Bob Laliberte, senior analyst at Enterprise Strategy Group. “So it will be critical for them to find very sophisticated tools that will allow that complex environment to become simple to manage.”
Easier said than done. Network professionals often complain that existing device-centric network monitoring does not scale or provide the needed visibility for cloud and digital-business-era applications. Cloud-native monitoring tools, such as Amazon CloudWatch, Azure Monitor or GCP Stackdriver, are less piecemeal and can observe all infrastructure and application layers, but some users find cloud tools often lacking in features and visibility, not to mention that they don’t integrate well with on-premises tools.
No vendor has come up with a “big picture” monitoring solution, and one shouldn’t be expected anytime soon because of the vast differences between the networks you own and those you rent, analysts say. But there are ways to close the gap just a little and achieve better visibility across the networks.
Connecting islands of network insight
In a hybrid cloud environment, “you’re always going to have islands of visibility. The important thing is to look for opportunities to integrate those islands,” McGillicuddy says.
One of the most valuable sources of data for a network monitoring tool is a management system API used to pull data from other platforms, whether it’s from AWS, or an IT service management platform like ServiceNow that’s sending ticket data, or a security monitoring tool.
“If you’re going to try to pull these things together, you need a network monitoring vendor that has a very modern API on the tool that gives you
