ArgoCD ApplicationSets: Stop Copy-Pasting Your Environments

It was 4:47 on a Friday when I found out that our api-service app in the Frankfurt staging cluster had been running a three-week-old container image. Not because of some exotic drift. Because someone (me, if I'm honest) had bumped the target revision in six of our seven cluster-specific Application manifests, and simply missed the seventh. Every file looked identical at a glance. That's the problem.

We were on ArgoCD v2.9.3 at the time, managing a handful of apps across dev, staging, and prod on three clusters. That's maybe 20 Application manifests. Each one a carbon copy of the others except for the destination server, a couple of Helm values, and the sync wave annotation. Nobody ever got them all right in one pass, and the failure mode was silent. ArgoCD doesn't page you when your manifest is missing; it just happily keeps syncing whatever you told it last time.

Quick answer: ArgoCD ApplicationSets replace hand-maintained, per-cluster Application manifests with a single template plus a generator that stamps out the Applications for you. Start with the list generator on your most-duplicated app, then graduate to the git or cluster generator when you're tired of enumerating clusters by hand. The win isn't speed; it's that the "updated six of seven manifests" class of silent failure becomes impossible. Watch for the cascade-on-delete default, and if you use the PR generator, GitHub's API rate limit.

The actual cost of copy-pasted Applications

The thing about near-identical YAML is that diff-based review stops working. When five files differ by two lines each, reviewers stop reading. Changes like "add the retry backoff limit to all staging apps" become a sed command run nervously against every file, followed by a prayer.

And the drift compounds. Over a few months, our Application manifests picked up real divergences that nobody intended. One cluster still had automated: {} with prune: false from an old experiment. Another had a sync wave annotation someone added during an incident and never propagated. Prod's syncPolicy had selfHeal: true in two manifests and was missing in a third, which meant prod silently tolerated manual edits to a deployment for weeks. We only noticed when a hotfix got "mysteriously reverted" overnight. GitOps doing its job, against a human who didn't know it.

What an ApplicationSet actually is

An ApplicationSet is a controller that watches a template plus one or more generators, and stamps out Application resources for you. Instead of you maintaining N Applications by hand, you maintain one ApplicationSet and a source of truth for "what are my clusters/environments." The controller does the copying, and it does it correctly every time.

That's the entire pitch. It's not glamorous. It just deletes an entire class of human error.

The list generator: your first win

Before, the Frankfurt staging manifest. Multiply by seven:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-service-eu-staging
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  project: platform
  source:
    repoURL: https://github.com/acme/platform.git
    targetRevision: 1.14.2
    path: charts/api-service
    helm:
      values: |
        replicas: 4
        region: eu-central-1
  destination:
    server: https://frankfurt.example.internal:6443
    namespace: api-service
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

After. One ApplicationSet covers the whole matrix:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: api-service
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: frankfurt-staging
            url: https://frankfurt.example.internal:6443
            region: eu-central-1
            revision: 1.14.2
            wave: "1"
          - cluster: virginia-staging
            url: https://virginia.example.internal:6443
            region: us-east-1
            revision: 1.14.2
            wave: "2"
  template:
    metadata:
      name: api-service-{{.cluster}}
      annotations:
        argocd.argoproj.io/sync-wave: "{{.wave}}"
    spec:
      project: platform
      source:
        repoURL: https://github.com/acme/platform.git
        targetRevision: "{{.revision}}"
        path: charts/api-service
        helm:
          values: |
            replicas: 4
            region: {{.region}}
      destination:
        server: "{{.url}}"
        namespace: api-service
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Now "roll out 1.14.3" is one edit, or two edits if you're staggering the rollout by wave. It literally cannot be applied to six of seven clusters anymore.

The git and cluster generators: stop listing things by hand

The list generator is honest and boring, which is why I like it. But it still makes you enumerate clusters in a file. The next two generators remove even that.

The git generator treats directories in a repo as the source of truth. Point it at clusters/*/config.json (or just directories with the directories mode) and every new folder becomes a rendered Application. Add a cluster? Commit a directory. Delete one? Remove the directory and the app goes away. Onboarding a new environment becomes a git commit instead of a YAML archaeology session.

The cluster generator goes further: it reads the cluster secrets registered in ArgoCD itself. Every cluster the ArgoCD control plane knows about gets templated parameters like {{.name}}, {{.server}}, and whatever labels you put on the cluster secret. That last part is the useful bit. Label a cluster secret env: prod or region: eu and you can target subsets with a selector, no per-cluster enumeration at all. For cluster-addons (ingress-nginx, cert-manager, your CNI config) this is the generator you want. The app genuinely should exist everywhere ArgoCD reaches, and the cluster generator expresses exactly that.

One practical note: ApplicationSets create Applications, which can be heavy on big multi-cluster setups. Set spec.syncPolicy.preserveResourcesOnDeletion: true if you ever plan to delete an ApplicationSet without torching what it built, because the default (false) cascades and deletes everything it generated. I've watched a teammate discover this in a test environment. He was pale for about ten minutes.

The gotchas nobody puts in the README

ApplicationSet templates are Go templates under the hood, and historically they were a limited dialect. Until ArgoCD 2.8, you couldn't use conditionals or Sprig functions in the classic template style, which meant "set replicas to 4 in prod but 1 in dev" needed awkward workarounds like passing everything through generator values. Setting goTemplate: true on the ApplicationSet fixes this and gives you Sprig, but it changes the placeholder syntax from {{.cluster}} to {{ .cluster }} (note the spaces) and it's a one-way door per resource. Also, anything you can't express in the template still has to come from the generator. There's no escape hatch for arbitrary per-app logic, and occasionally that's genuinely annoying.

The PR generator deserves its own warning. It sounds wonderful: open a pull request against your app repo, get a preview Application deployed automatically. And it works, right up until you run it against a busy repo on GitHub with a shared token. The default config re-lists open PRs every 30 minutes per ApplicationSet, and with a few sets running you will eat GitHub's rate limit (5,000 requests/hour authenticated) surprisingly fast. The symptom is weird: preview apps just stop appearing, and the controller logs fill with 403 rate limit errors that are easy to miss if you aren't watching argocd-applicationset-controller specifically.

Two more footguns with previews. First, requeueAfterSeconds and refresh behavior can make preview apps churn, so set explicit labels and watch your ArgoCD repo-server load. Second, cleanup: when a PR closes, the preview Application is deleted, but if your app's finalizers or namespaces were created outside the Application's own sync, you can end up with orphaned namespaces named pr-1847 littering the dev cluster. We found eleven of them during an audit. One was three months old and still holding a LoadBalancer.

Finally, server-side apply. Generated Applications are managed objects, and if you later try to kubectl apply a hand-edited version on top of one, you'll fight field ownership. Get used to kubectl apply --server-side and to the fact that the ApplicationSet is the writer now. Editing the child Application directly gets reverted on the next reconciliation, which is correct, but it'll confuse anyone who didn't get the memo.

Frequently asked questions

What is an ArgoCD ApplicationSet?

An ApplicationSet is a controller that watches a template plus one or more generators and stamps out ArgoCD Application resources for you. Instead of maintaining N near-identical Application manifests by hand, you maintain one ApplicationSet and a source of truth for your clusters and environments. It deletes an entire class of copy-paste errors, like the missed seventh manifest that started this post.

How do I manage ArgoCD Applications across multiple clusters without duplicating YAML?

Start with the list generator: one ApplicationSet whose elements list each cluster's parameters, rendered through a shared template. When you outgrow enumerating clusters by hand, move to the git generator, where directories in a repo become Applications, or the cluster generator, where every cluster registered in ArgoCD gets templated automatically. Rolling out a new version becomes one edit instead of N.

What's the difference between the list, git, and cluster generators in ApplicationSets?

The list generator takes a hardcoded list of cluster parameters inside the ApplicationSet itself. The git generator treats directories or config files in a repo as the source of truth, so adding a cluster is a git commit. The cluster generator reads the cluster secrets registered in ArgoCD, and it's the right fit for cluster addons like ingress-nginx or cert-manager that should exist everywhere ArgoCD reaches.

Why do ArgoCD PR generator preview apps suddenly stop appearing?

Almost always GitHub API rate limiting. The PR generator re-lists open pull requests every 30 minutes per ApplicationSet by default, and a few sets pointed at a busy repo with a shared token will burn through the 5,000 requests per hour authenticated limit surprisingly fast. The only symptom is preview apps silently not appearing, with 403 errors buried in the argocd-applicationset-controller logs.

What happens when you delete an ApplicationSet?

By default it cascades: every Application the ApplicationSet generated gets deleted too, along with whatever those Applications were managing. If that's not what you want, look at preserveResourcesOnDeletion before you delete anything. A teammate of mine learned this in a test environment and was pale for about ten minutes.

The takeaway

Hand-maintained Application manifests fail quietly, and quiet failures are the expensive kind. The Friday incident cost us nothing in the end, a stale staging build that QA caught on Monday. But it was luck, not process. Moving to ApplicationSets didn't make us faster in any way a dashboard would show. It made a category of mistake impossible, which is worth more.

If you're still copying Application YAML per cluster, start with the list generator on your most-cloned app. It takes an afternoon. And once your environments exist as data instead of duplicated files, tools that show you cluster state at a glance, which is the kind of thing we built conndeck for, start making a lot more sense, because the state is finally uniform enough to read.