QoS Classes: Guaranteed, Burstable, and Who Gets Evicted First

On a Saturday morning in March, our cluster lost a node to a bad firmware update, and the remaining nodes started sweating. Memory pressure crept up for about twenty minutes, and then the evictions started.

The pods that died first were not the ones I would have chosen. Our fraud-scoring service -- Burstable, moderately important -- survived untouched. A batch of dev namespace experiments -- BestEffort, unimportant -- also survived, because they were tiny. What got evicted was a logging sidecar-heavy deployment that was using 3x its memory request on a node that had run out of patience. I remember staring at the eviction events thinking the kubelet had made a bad decision. It hadn't. It had made exactly the decision the QoS rules dictate, and I just didn't know the rules as well as I thought I did.

So here are the rules.

Quick answer: Kubernetes assigns every pod a QoS class computed from its resource fields: Guaranteed when all containers have CPU and memory requests equal to limits, Burstable when at least one container has requests or limits but the equality doesn't hold, and BestEffort when nothing is set. Under node memory pressure, the kubelet evicts pods ranked by how far their usage exceeds their requests -- BestEffort first, over-consuming Burstable pods next, Guaranteed last. Set honest requests and memory limits on everything that matters, and you'll land in Burstable or Guaranteed and survive the next tight node.

How the class is computed

You never set a QoS class. Kubernetes derives it after admission, from the resource fields, and stamps it into the pod's status:

• Guaranteed: every container in the pod (init containers included, via the effective request math) has CPU *and* memory limits set, *and* requests equal to those limits for both resources. One missing field or one inequality anywhere in the pod and you're not Guaranteed.
• BestEffort: no container has any requests or limits at all. The class of cowboys and tutorials.
• Burstable: everything in between. At least one container has at least one request or limit, but the pod fails the Guaranteed test.

Check any pod:

$ kubectl get pod fraud-api-6c8b9d7f5-kl2mn -n risk -o jsonpath='{.status.qosClass}'
Burstable

Note it's in status, not spec -- the class reflects what the pod actually got, including defaults a LimitRange injected at admission. A pod you *thought* had no resources can turn out Burstable because the namespace defaulted a request into it.

What the class actually controls

The QoS class is not a vanity label. It drives three concrete behaviors.

Eviction order under node pressure. When a node's memory (or ephemeral storage) crosses an eviction threshold, the kubelet ranks pods by usage relative to requests: pods using the most beyond what they reserved die first. BestEffort pods have zero requests, so any usage at all puts them at the front of the line. Guaranteed pods can't exceed their requests by definition, so they're last. Burstable pods are ranked by the size of their overage -- which is how my logging deployment, 3x over its request, outranked the tiny BestEffort experiments on that Saturday.

The kernel OOM killer's aim. The kubelet sets each container's oom_score_adj based on QoS: roughly +1000 for BestEffort, a middle value for Burstable scaled by request size, and -998 for Guaranteed. When the node-level OOM killer fires -- which happens when memory is gone before the kubelet's eviction loop reacts -- it picks the highest score. Same hierarchy, blunter instrument.

CPU behavior on the node. Guaranteed pods with integral CPU requests are eligible for the static CPU manager policy, getting pinned exclusive cores. Everyone else shares the CFS pool, weighted by requests.

Eviction in practice: thresholds and signals

The kubelet watches eviction signals and acts at thresholds, with defaults like:

memory.available < 100Mi
nodefs.available < 10%
imagefs.available < 15%

You'll see it in the node conditions and events:

$ kubectl describe node worker-14
...
Conditions:
  Type             Status   Reason
  MemoryPressure   True     KubeletHasInsufficientMemory
Events:
  Warning  EvictionThresholdMet  Attempting to reclaim memory
  Normal   NodeHasInsufficientMemory
$ kubectl get pods -A --field-selector=status.phase=Failed | grep Evicted | head
logging    fluent-bit-shipper-7d4f9   0/1   Evicted   ...

When a pod is evicted, it's terminated and rescheduled by its controller -- assuming there *is* somewhere to put it, which, on a cluster already tight on memory, is its own adventure. And a detail worth knowing: the kubelet only evicts pods whose usage exceeds requests when the pressure is severe; in soft-eviction scenarios with grace periods configured, it can also respect PDBs, but hard evictions don't wait for anyone.

One more asymmetry people trip over: a container exceeding its own memory *limit* gets OOM killed by its cgroup no matter its class, instantly, even on a node with free memory. QoS governs who dies when the *node* is short. Limits govern who dies when the *container* is greedy. Different mechanisms, different corpses.

Choosing a class deliberately

My rules, after enough Saturdays:

• Latency-critical services (the checkout path, the auth service): Guaranteed. Requests = limits, sized from measured P99 usage. Yes, you pay for the headroom you don't use. That's what reliability costs.
• Everything else production: Burstable, with honest requests and memory limits always set. You keep burst headroom, you keep most of the eviction protection, you don't inflate the scheduler's arithmetic.
• BestEffort: fine for scratch jobs and dev experiments. Unacceptable for anything with users. No requests also means no CPU-based HPA and no claim when the node gets tight -- you're a stowaway, and stowaways go overboard first.

When I need to answer "who's actually at risk on this node," seeing live usage against requests across a namespace in one view -- the exact thing we built conndeck to surface -- beats kubectl top plus a spreadsheet. But even with plain kubectl, the two commands above tell you who's next in line.

Frequently asked questions

What are the three QoS classes in Kubernetes?

Guaranteed means every container has CPU and memory requests exactly equal to limits. Burstable means at least one container has a request or limit set but the pod doesn't qualify as Guaranteed. BestEffort means no container has any requests or limits at all. The class is computed from the resource fields alone; you never set it directly.

How do I check a pod's QoS class?

Run kubectl get pod <name> -o jsonpath='{.status.qosClass}' or look at the QoS Class line in kubectl describe pod. It's in status, not spec, because Kubernetes computes it from the resource requests and limits after admission, including anything a LimitRange injected.

Which pods get evicted first under node memory pressure?

The kubelet ranks pods by how far their usage exceeds their requests, not strictly by class. BestEffort pods, which exceed their zero requests by definition, go first, then Burstable pods using more than requested. Guaranteed pods are last because their usage can't exceed their requests. A Burstable pod far over its request can be evicted before a BestEffort pod using almost nothing, but in practice BestEffort dies first.

Should I make all my pods Guaranteed?

Not blindly. Guaranteed pods get the strongest eviction protection and stable CPU via static allocation possibilities, but requests equal to limits means zero headroom for bursting and larger reservations that inflate cluster cost. Make latency-critical pods Guaranteed, leave normal services Burstable with honest requests, and never ship BestEffort for anything you care about.

Does QoS class affect the OOM killer inside a container?

Indirectly. The kubelet adjusts each container's oom_score_adj based on QoS class, so when the node's kernel OOM killer fires, BestEffort containers are the most attractive targets and Guaranteed containers the least. But a container exceeding its own memory limit gets killed regardless of class, because that limit enforcement is per-cgroup and absolute.

If you remember one thing

QoS classes are a three-line computation with outsized consequences: they decide eviction order, aim the kernel OOM killer, and gate CPU pinning. Guaranteed costs you headroom, BestEffort costs you survival, and Burstable with honest numbers is where most workloads belong.

The next time a node gets tight, the eviction order won't be a mystery. It'll be arithmetic you can predict -- and, better, arithmetic you engineered on purpose.