conndeck blog

Image Scanning + Admission Control: Stop Vulnerable Images

The ticket that started all this was a single line from our security scanner: "172 critical CVEs currently deployed in production." It was a Friday, 4:30 PM, and the number was real.

The worst part wasn't the number. It was that every one of those images had been scanned in CI -- we had Trivy in the pipeline, green checks everywhere. The findings came from three places the pipeline never saw: images deployed by hand during incidents, third-party Helm charts with pinned versions from eight months ago, and base images whose CVEs were published *after* our last build. Scanning in CI had given us a false sense of completeness, which is worse than no scanning, because at least with no scanning you know you're blind.

The fix that actually worked was treating admission control as the enforcement point and CI scanning as the feedback mechanism -- and, just as importantly, building an exception workflow before anyone needed one.

Quick answer: Scan images in CI with Trivy or Grype so developers get fast feedback, but enforce at the cluster edge with an admission policy -- a Kyverno verifyImages rule or a Gatekeeper constraint -- that only allows images from approved registries with a passing scan attestation. Add an in-cluster scanner like trivy-operator to catch hand-deployed images and newly published CVEs against old workloads. Handle exceptions with scoped, expiring objects like Kyverno's PolicyException, never global mutes. Roll every policy out in Audit mode first.

Scan in CI for feedback, not enforcement

CI scanning is non-negotiable, but understand what it is: fast feedback next to the code. A Trivy step that fails the build on criticals looks like this:

$ trivy image --severity CRITICAL,HIGH --exit-code 1 --ignore-unfixed registry.example.com/api:v2.14.3
registry.example.com/api:v2.14.3 (debian 12.6)
========================================
Total: 4 (HIGH: 4, CRITICAL: 0)

┌────────────┬───────────────┬──────────┬────────┬───────────────────┬───────────────┐
│  Library   │ Vulnerability │ Severity │ Status │ Installed Version │ Fixed Version │
├────────────┼───────────────┼──────────┼────────┼───────────────────┼───────────────┤
│ libssl3    │ CVE-2026-31210│ HIGH     │ fixed  │ 3.0.13-1~deb12u1  │ 3.0.15-1      │
└────────────┴───────────────┴──────────┴────────┴───────────────────┴───────────────┘

Two flags here carry the whole philosophy. --exit-code 1 makes the scan a gate instead of a report nobody reads. --ignore-unfixed filters out the noise of CVEs with no patch available, which is where scanning initiatives go to die -- a developer who gets paged for an unfixable finding will learn to ignore fixable ones. Grype is an equally good choice and does the same job with grype --fail-on critical <image>; pick whichever your team will actually maintain, and don't run both unless you enjoy reconciling two databases' opinions of the same CVE.

Enforce at admission, because CI has holes

Manual deploys, GitOps repos someone applied from their laptop, Helm charts vendored last year -- all of it bypasses CI. Admission control is the backstop. With Kyverno, the policy that requires an attested scan report looks roughly like this:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-scan-attestation
spec:
  validationFailureAction: Audit   # flip to Enforce after two quiet weeks
  background: true
  rules:
    - name: check-vulnerability-attestation
      match:
        any:
          - resources:
              kinds: ["Pod"]
              namespaces: ["payments", "ledger", "frontend"]
      verifyImages:
        - imageReferences:
            - "registry.example.com/*"
          attestations:
            - type: "https://trivy.dev/attestation/vuln/v1"
              conditions:
                - all:
                    - key: "{{ scanner.result.criticalCount }}"
                      operator: Equals
                      value: 0
              attestors:
                - entries:
                    - keyless:
                        subject: "https://github.com/example-org/api/.github/workflows/scan.yml@refs/heads/main"
                        issuer: "https://token.actions.githubusercontent.com"

The pattern: your CI job scans the image and, on a pass, creates a cosign attestation recording the result. The admission policy verifies that attestation -- who signed it, what it says -- before the pod is allowed to exist. No attestation, no deploy. Pair it with a mutate rule that resolves tags to digests, because v2.14.3 is a name, not an identity, and attestations attach to digests.

The validationFailureAction: Audit line is doing the same job audit mode does for Pod Security Standards: run it quietly for two weeks, read the PolicyReports, fix what surfaces, then enforce. Every admission policy I've seen enforced on day one has been disabled by day three.

Scan the cluster itself

Even with CI and admission both clean, there's a third gap: time. An image that was clean in March is a liability in August if a critical CVE dropped in June. An in-cluster scanner -- trivy-operator is the common choice -- continuously scans what's actually running and exposes findings as CRDs:

$ kubectl get vulnerabilityreports -A -o wide
NAMESPACE   NAME                                        REPOSITORY                    TAG       CRITICAL   HIGH
payments    deployment-ledger-api-api                   registry.example.com/api      v2.14.3   1          4
monitoring  daemonset-node-exporter-node-exporter       quay.io/prometheus/node-ex... v1.8.2    0          2

This is also where you'll find the workloads that never went through your pipeline at all. During a rollout of this exact setup I mostly watched the reports accumulate cluster-wide in conndeck rather than scripting kubectl loops, because the shape of the problem -- which namespaces, which teams -- is what decides your rollout order. However you view them, alert on CRITICAL counts, not totals, or you'll train everyone to ignore the alerts.

Exceptions: scoped, expiring, embarrassing to request

You will need exceptions. The vendor image with an unfixable medium CVE that their support contract says you may not rebuild. The frozen legacy service migrating off next quarter. The mistake is handling these as config-file mutes that never die.

Kyverno's PolicyException is the right shape -- a real object, reviewed in git, scoped to a policy, a rule, and specific images, with a namespace boundary:

apiVersion: kyverno.io/v2beta1
kind: PolicyException
metadata:
  name: vendor-reporting-cve-exception
  namespace: reporting
spec:
  exceptions:
    - policyName: require-scan-attestation
      ruleNames: ["check-vulnerability-attestation"]
  match:
    any:
      - resources:
          kinds: ["Pod"]
          selector:
            matchLabels:
              app: vendor-reporting

Put the CVE ID and an expiry date in the object name or a comment, and set a calendar reminder. For CI-level ignores, .trivyignore entries go through the same code review as everything else -- which is the point. An exception that requires a pull request, an owner, and a date is a controlled risk. An exception that's a line in a YAML file someone copied from a blog post is a backdoor with good documentation.

Frequently asked questions

How do I block vulnerable container images from deploying to Kubernetes?

Scan every image in CI with Trivy or Grype and fail the build on critical findings, then enforce at the cluster edge with an admission policy in Kyverno or Gatekeeper that rejects images which aren't in an approved registry or lack a passing scan attestation. CI scanning is where developers get fast feedback, and admission control is the backstop that catches anything that bypassed the pipeline, like manual deploys and side-loaded manifests.

Should I use Trivy or Grype for container image scanning?

Both are solid and free, and the reality is to pick the one that fits your pipeline shape. Trivy scans images, filesystems, git repos, and Kubernetes clusters in one tool and has the bigger ecosystem of integrations, while Grype is narrowly focused on image and filesystem scanning with a clean library for embedding. Most platform teams land on Trivy for its all-in-one surface and its first-class trivy-operator for in-cluster scanning.

How do I handle false positives in image vulnerability scanning?

Never mute findings globally; scope every exception to a specific image and CVE with an expiry date. Trivy supports a .trivyignore file and Grype supports ignore rules in its config, both of which live next to the code and go through code review like any other change. For admission-level blocks, use a proper exception resource like Kyverno's PolicyException so the bypass is visible, auditable, and self-expiring.

What is the difference between scanning in CI versus scanning in the cluster?

CI scanning gates what gets built and pushed, and gives developers feedback in minutes next to the code that caused it. In-cluster scanning with something like trivy-operator catches reality: images deployed by hand, drifted tags, and newly published CVEs against images that have been running untouched for months. You need both, because a clean scan at build time says nothing about a critical CVE published last night.

How do I write a Kyverno policy to require image scan attestations?

Use a Kyverno ClusterPolicy with a verifyImages rule that checks a vulnerability attestation on the image, typically a cosign attestation produced by your scanner in CI. The policy names the attestation type, the keyless identity or key allowed to sign it, and can be paired with a mutate rule that rewrites image tags to digests so the attested artifact is exactly what runs. Start the policy in Audit mode, watch the policy reports, then flip to Enforce.

The field-notes version

Image security that works has three layers, and each one exists to cover the others' blind spots: CI for fast feedback, admission for enforcement, in-cluster scanning for time and drift. Skip any one and you have a hole you can't see.

And build the exception workflow first. The security boundary you can't route around politely is the one people route around impolitely.