Google Cloud adds Kubernetes metadata to flow logs

Google Cloud has added Google Kubernetes Engine metadata to VPC Flow Logs for Kubernetes network traffic, giving users workload-level attribution for traffic captured in Google Cloud's virtual network.

The update addresses a long-standing visibility problem in Kubernetes environments, where pods are temporary and IP addresses can change as workloads scale, fail or move between nodes. In those conditions, raw flow logs that show only IP addresses, ports and protocols can make it hard to identify which application or service generated a connection.

Under the new approach, traffic records collected through VPC Flow Logs can be annotated with Kubernetes context linked to the IP address at the time the traffic was sampled. This includes the cluster name and location, pod name, namespace, owning controller, receiving service, and node and zone details at both ends of a flow.

The metadata is available when flow logs are enabled for a subnet hosting a GKE cluster. Because the information is attached when traffic is observed in the virtual network, the context remains available even if the pod no longer exists by the time an engineer investigates an incident.

Network visibility

Google is positioning the feature as a way to improve day-to-day troubleshooting and architecture analysis without adding extra agents or sidecars to clusters. The logs can also complement observability tools already running inside clusters, including Container Network Interface and dataplane monitoring systems.

For organisations that register GKE workloads in App Hub, records can also carry application-level labels identifying each endpoint's business application, environment and criticality. That can help when services inside a cluster communicate with external resources also registered in App Hub, such as Cloud Run services.

The release also sets out several operational requirements for customers who want to use the data in Flow Analyser, a tool in Network Intelligence Centre that presents log records as charts and tables. Users need to upgrade their log bucket to Observability Analytics, enable VPC Flow Logs at the organisation level if they want broad coverage across subnets, and keep full metadata enabled so namespace and pod information appears in records.

Standard GKE clusters also need intranode visibility enabled, while Autopilot clusters already include that setting. Google noted that enabling intranode visibility on Standard clusters restarts control plane and node components and adds some overhead per node, and recommended testing on non-production systems first.

Traffic analysis

Flow Analyser can then group and filter traffic by cluster, namespace, workload, pod or service. Google outlined several uses for the enriched data, including mapping dependencies between microservices, examining which workloads call a particular service, and identifying traffic that crosses zones inside regional clusters.

In one example, grouping traffic by source and destination GKE workload turns a table of network flows into an application dependency map. This lets engineers see service-to-service communication directly in workload terms rather than looking up transient IP addresses after the fact.

Another use case focuses on a single service. By filtering on a server workload and aggregating traffic bidirectionally between client and server, engineers can see which callers account for the largest share of traffic and track changes over time. The same view can also be switched to latency, allowing teams to rank callers by round-trip time rather than volume.

The metadata can also expose zone-related inefficiencies. Because VPC Flow Logs include the zone at both ends of each flow, engineers can filter for traffic where source and destination zones differ and identify pod pairs generating cross-zone traffic. In regional clusters, that can point to avoidable inter-zone data transfer charges as well as added latency.

Google said a built-in GKE setting can help address that issue by preferring healthy endpoints in the caller's zone, provided each zone has enough replicas to handle its own traffic.

Limits and caveats

Google also outlined several limits. Pod-to-internet traffic does not include pod annotations because the pod IP is replaced by the node IP before the flow is sampled. Traffic blocked by network policy never reaches the virtual network, so those events must be investigated through NetworkPolicy logging instead.

There is also a trade-off in the nature of flow logs themselves. Because the logs are sampled rather than captured exhaustively, byte and packet counts are better suited to ranking and trend analysis than to precise accounting for every connection.

The release reflects a broader effort by cloud providers to make Kubernetes networking easier to inspect at the workload level as operators manage increasingly distributed applications made up of short-lived components. In practice, the feature's value will depend on how widely customers enable the logging, analytics and cluster settings needed to surface the data.

For customers already using VPC Flow Logs, Google said the simplest path is to upgrade the default logging bucket and begin analysis in Flow Analyser.