Google Open-Sources k8s-aibom to Tame Shadow AI on GKE

Your Kubernetes cluster might be hosting AI workloads you don’t even know about. Developers deploy models without formal registration, and traditional security scanners often miss them because nobody wants to slow down development with privileged DaemonSets or kernel-level access. Google Cloud just open-sourced a tool that breaks this deadlock: k8s-aibom.

This lightweight, unprivileged Kubernetes controller continuously monitors your cluster API and container environments to automatically detect running AI runtimes—like vLLM and Triton—and generate standard CycloneDX Machine Learning Bill of Materials (ML-BOMs). No sidecars, no eBPF, no pod-spec edits. Just audit-grade visibility straight from runtime.

Zero-Friction Architecture

k8s-aibom deploys as a single, unprivileged Deployment in its own namespace. It respects both the CISO’s need for total visibility and the SRE’s mandate for cluster stability. The discovery pipeline works in four stages:

1. Scrape cluster workloads: Monitors KServe resources, Deployments, StatefulSets, DaemonSets, and Jobs across the cluster.
2. Identify AI stacks: Pattern-matches container images, env vars, and command-line args to detect serving runtimes (vLLM, Triton, TGI, Ollama), agent frameworks (LangChain, AutoGen, CrewAI), vector databases (Milvus, Qdrant, pgvector), and training jobs.
3. Generate standard manifests: Compiles discovered artifacts into OWASP CycloneDX 1.6 ML-BOMs.
4. Export to sinks: Attaches the BOM to a custom resource status and optionally routes it to Google Cloud Storage or webhooks.

Application teams don’t need to change their pods or CI/CD pipelines. And because k8s-aibom treats cluster state as a pure functional input, identical clusters produce byte-identical BOMs—perfect for GitOps diffs and change-detection alerts.

Confidence Model: Intent vs. Inference

Raw telemetry is often noise. k8s-aibom solves this with a deterministic Confidence Model that categorizes assets into three tiers:

• Declared: Explicitly defined by the developer (e.g., --model meta-llama/Llama-2-7b). Clear human intent.
• Inferred: Derived by pattern-matching (e.g., ^vllm/.* container signatures).
• Unresolved: AI presence detected but exact model parameters unknown. Flags for security review.

This taxonomy lets compliance reviewers instantly separate engineering intent from machine inference, building an unassailable chain of trust.

Audit-Grade Security

k8s-aibom establishes an immutable evidence trail. The controller uses a dedicated service account with minimal IAM permissions (only roles/storage.objectCreator). The GCS sink enforces DoesNotExist preconditions, so once an ML-BOM is written, it can’t be overwritten or tampered with—even by compromised cluster actors.

Governance Ready

By automating CycloneDX 1.6 ML-BOMs, k8s-aibom bridges the gap between runtime state and global regulatory frameworks like the EU AI Act, NIST AI RMF, and ISO/IEC 42001. It helps unblock GKE AI deployments by providing the empirical data needed for compliance audits.

Check out the k8s-aibom GitHub repo to inspect the controller, review CRD definitions, and contribute. For more on managing shadow AI, see Google Cloud’s governance tips.

Original announcement published on Google Cloud.