Cloud Monitoring is Critical to Realizing AI-Powered Benefits

Public cloud Infrastructure-as-a-Service (IaaS) offers many benefits, but it is important to note that it is still infrastructure. Just like physical infrastructure, there are many ways that applications and workloads can be implemented, optimized, secured, and managed.

Binary scrolling across a cloud
Figure 1: Visibility into intra-cloud East-West Traffic is Critical to Realize the benefits of AI-Powered Results

The infrastructure and other shared resources comprise a Virtual Private Cloud (VPC). Your organization will have and operate its own VPC in one or more clouds (i.e., your organization may only use single-cloud or use multiple clouds from different providers). The entire I&O team that includes NetOps, SecOps, AppOps, DevOps, CloudOps, and SRE has the shared responsibility to collaboratively configure, optimize, secure, and operate your organization’s VPC environments.

These objectives typically include regulatory and end-user experience requirements that are deemed necessary for the applications, services, and workloads to meet. It is common for these requirements to be formally defined and documented as a Service Level Experience and/or Service Level Agreement.

Performance and security are always paramount to organizational success. Different cloud operators may offer features such as Virtual TAPs (aka vTaps) and virtual Network Packet Brokers (NPB), network management, and overall performance management. In the absence of such features, it’s up to the I&O organization to implement workarounds and use third-party solutions (covered later in this blog). This information aides I&O personnel with the visibility and data needed to assure security, performance, and experiences. These I&O responsibilities for cloud shine the spotlight on the criticality of network performance monitoring using tools that are explicitly designed for cloud and virtualized environments. The types of tools broadly fall into these categories – Application Performance Monitoring (APM), Network Detection and Response (NDR), IT Operations Management (ITOM), and AI for IT Operations (AIOps).

AI Use is Growing, Pervasive, and Ubiquitous

Artificial Intelligence (AI) and Machine Learning (ML) are new types of workloads that are increasingly being used. While not applications per se, the performance of these workloads must be monitored and optimized.

The use of AI is a growing trend as is reinforced by results from numerous sources and market analysts; see the links in the related resources section at the end of this blog that reveal recent and projected growth rates. AI workloads will increase as the underlying technology and benefits of AI-powered solutions progress.

AI Transforms Data into Actionable Intelligence Using Analytics and ML

AI transforms data into actionable intelligence using analytics and ML to drive desired outcomes that are unattainable by humans alone. AI therefore augments human skills and intelligently assists them to efficiently complete work and life tasks. AI also offloads more tasks because intelligent automation that adapts to changing situations without constant human oversight is now possible.

To leverage the many benefits of AI effectively, organization’s data processing infrastructure must maintain a high-level of performance to handle AI workloads to assure the attainment of desired outcomes. So, if your organization uses AI or plans to, then the success of the AI results rely on attaining performance metrics specific to each use case. Assuring the metrics are met requires a consistent and thorough understanding of the AI workloads and corresponding east-west traffic in your cloud environment.

AI and ML are Sources of East-West Traffic

Artificial Intelligence uses Machine Learning and Advanced Analytics, all of which are processing-intensive workloads that operate on big data and generate a lot of I/O and intra-cloud east-west traffic.

AI makes it possible to provide the right information, recommendation, offer, or assistance to the right person at the right time. Timeliness is important when someone is shopping for an item, a relevant discount offer must be generated and delivered within a couple seconds. Similarly, when it comes to preventing cyber-attacks – the shortest possible time to detect and respond to a threat is extremely critical.

AI-Powered Use Cases

Time-to-results from AI workloads is critical to the success of AI-powered use cases and applications, especially for end-user experiences and when real-time results are required for intelligent automation, personalized recommendations, and cybersecurity. Conversely, if the time-to-results is too slow, then the benefits will be lessened or not realized at all.

East-west traffic is also driven by modern cloud-native software implementations that are typically architected using microservices and containers, especially for cloud and virtualized environments. Communications between microservices and containers, including those running AI and ML workloads are typically via APIs that also contribute to intra-cloud east-west traffic.

Network Performance Monitoring of East-West Traffic

Production AI workloads are increasingly run in cloud and other virtualized environments that create a lot of I/O and east-west traffic. In some cases, east-west traffic in virtualized environments can be mostly or fully within a single physical server or rack of server blades that are not exposed for monitoring using physical appliances and other traditional methods.

Cloud monitoring focused on network traffic in these types of virtualized environments requires software-based methods that tap into traffic passing between virtual machines for delivery to APM, NDR, ITOM, and AIOps solutions, and storage too. You can also view network analytics, KPIs, and other information using cPacket Networks’ cClear-V® Analytics Engine.

Virtual TAPs and Virtual Network Packet Brokers  

These methods include native packet/traffic mirroring that is available from some public cloud providers such as AWS with its Traffic Mirroring service and Google Cloud with its Packet Mirroring service. Virtual TAPs (aka vTaps) and virtual Network Packet Brokers (NPB), covered in more detail in the next section, are other ways of using software to effectively monitor traffic and mirror network packets in cloud and virtualized environments.

Network flow information that summarizes the connectivity and load between two endpoints is another method of network traffic analysis. This type of information, commonly referred to as NetFlow data, can be created or obtained in several ways, including using the  cPacket Networks cProbe-V flow generator and exporter as shown in Figure 2. NetFlow data is an industry standard that is commonly consumed by performance management and security solutions, and other tools that analyze and visualize NetFlow data.

Combining packet capture and NetFlow data provides comprehensive visibility and affords interoperability with many tools and solutions so you are best enabled to assure that your AI workloads are meeting time-to-results requirements.

Virtual Tapping Using a Virtual Network Packet Broker

Virtual Tapping Using a Virtual Network Packet Broker - cPacket
Figure 2: Using a Virtual Network Packet Broker to Gain Visibility in Cloud and Virtualized Environments

If a mirrored packets/traffic feature or service is not available for your cloud or virtualized environment or at the locations that you need to monitor, then you can use a virtual NPB such as the cPacket Networks cVu-V.  as a vTap as shown in Figure 2. Configuring the virtual NPB for inline operation makes it possible to monitor subnet ingress and egress traffic.

Simply create virtual NPB instances and insert them at strategic points to gain visibility by mirroring network packets to one or more destinations. As traffic transparently passes through the virtual NPB, it can also perform advanced brokering functions such as filtering, suppressing duplicate packets, load balancing, etc.

The inline configuration and corresponding tap-and-broker use case gives you

  • visibility to overcome the lack of natively mirrored packets/traffic
  • greater visibility in cloud environments by supplementing packet/traffic mirroring services with your own custom packet/traffic mirroring
  • inter-VM visibility in on-premises virtualized environments by supplementing physical network TAPs and NPBs

An example of a custom mirroring point would be between an application server and a database server running on separate virtual machines. Virtual NPB instances can be instantiated and inserted as needed, which makes the custom packet/traffic mirroring option very helpful for monitoring specific traffic to isolate and solve specific problems, strengthen security posture, and deliver specific tools to your AIOps and other solutions.

Additional guidance on using the cVu-V virtual NPB to gain visibility in cloud and virtualized environments is available by viewing the workshop video listed in the related resources section at the end of this blog.

Cloud Monitoring for Single-Cloud, Multi-Cloud, and Hybrid Environments

Thus far the discussion and diagrams have implied one or more VPCs in a single cloud, that is using only one provider’s cloud (e.g., AWS, Microsoft Azure, Google Cloud). An important consideration for the types of tools and solutions mentioned in this blog that you might choose, especially if from a single vendor, is that they seamlessly provide the same monitoring, data, dashboards, administration, functionality, and benefits in all clouds and across multi-cloud and hybrid environments in a unified manner through a single-pane-of-glass.


AI workloads are being used more and more, and like other workloads, they are being deployed and run in cloud and virtualized environments. These workloads generate east-west traffic that must be monitored so the I&O organization can effectively manage and assure adequate delivery of AI-powered results that are integral to successful applications and services. In cloud and virtualized environments, monitoring is accomplished using either or both cloud-provided packet/traffic mirroring and custom packet/traffic mirroring. These options provide the visibility, monitoring, and data necessary for the I&O organization to meet their assurance objectives efficiently and effectively in all cloud environment variations: single-cloud, multi-cloud, and hybrid.

To find out how cPacket Networks can help you with your AI-powered initiatives, request a demo.   

About the Authors

Ron Stein is the director of product marketing at cPacket Networks. Ron possesses technical expertise in the areas of networking, experience assurance, cloud, Big Data, AI, ML, Advanced Analytics, and IoT. His market and industry experience spans technology, healthcare, financial services, utilities, telecommunications, public safety, smart cities, and IT Operations.

Reese Jones, contributing author – Reese is a database administrator and freelance writer whose interests lie in AI, analytics, IoT, and how these fields continue to develop and impact our daily lives.

Resources to Learn More