Kubernetes 1.37 moves Memory QoS to betaKubernetes 1.37 moves Memory QoS to beta

When critical services share a node with everything else

In a homelab, it is easy to place services that must stay up next to workloads that can wait. Home Assistant, a database, or cluster control components may share a node with downloads, indexers, and experiments. When memory becomes scarce, limits still matter, but there has been little room to tell the kernel which memory should be preserved before the situation gets worse.

Kubernetes 1.37 promotes Memory QoS to beta and enables the MemoryQoS feature gate by default. That does not mean an upgrade automatically applies reservation or throttling to every Pod. The default kubelet configuration does not write memory.min, memory.low, or memory.high; each behavior must be enabled explicitly. (kubernetes.io)

QoS is no longer only about eviction order

Kubernetes classifies Pods as Guaranteed, Burstable, or BestEffort from their requests and limits. That classification already affected eviction decisions when a node was under resource pressure. With Memory QoS and cgroup v2, it also determines which memory protection or control the kubelet can apply. (kubernetes.io)

A Guaranteed Pod needs every container to define CPU and memory requests and limits, with matching values for each resource. A Burstable Pod has at least one request or limit but does not meet the Guaranteed requirements. A BestEffort Pod has no CPU or memory requests or limits, so it gets no memory protection from the tiered reservation policy. (kubernetes.io)

Protection is built on top of requests

The memoryReservationPolicy: TieredReservation setting enables memory reservation based on the QoS class. For Guaranteed Pods, the kubelet sets memory.min to the memory request. For Burstable Pods, it sets memory.low to the request, which the kernel will try to retain preferentially when memory is under pressure. (kubernetes.io)

The distinction matters. memory.min is strict protection against memory reclaim, while memory.low is a preference that the kernel can override under severe pressure. BestEffort Pods receive neither protection, which is usually sensible for disposable homelab workloads. (kubernetes.io)

To make Kubernetes classify a service as Guaranteed, setting memory alone is not enough. CPU requests and limits must also match for every container in the Pod. This is a basic example of a workload I would treat as critical:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: critical-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: critical-service
  template:
    metadata:
      labels:
        app: critical-service
    spec:
      containers:
        - name: app
          image: nginx:1.29
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "250m"
              memory: "256Mi"

With TieredReservation, this Pod can receive memory.min based on its memory request. There is no need to write to cgroups directly from the workload manifest: the kubelet uses declared resources and the resulting QoS class. That makes existing requests more operationally important, so they deserve a review before enabling the policy. (kubernetes.io)

memory.high can slow growth before the hard limit

The other part of Memory QoS is throttling through memory.high. The kubelet controls it with memoryThrottlingFactor, a value greater than zero and less than or equal to one. For Burstable Pods, the threshold is calculated from the request, the limit, and that factor, allowing memory allocation to be throttled before the workload reaches memory.max. (kubernetes.io)

The documented formula is memory.high = requests + memoryThrottlingFactor * (limits - requests). Guaranteed containers do not receive memory.high because their requests already equal their limits. For BestEffort containers, Kubernetes uses node allocatable memory as part of the calculation. (kubernetes.io)

If I want both tiered reservation and throttling, this is a valid kubelet configuration:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
memoryThrottlingFactor: 0.9
memoryReservationPolicy: TieredReservation

Both settings can be used independently. I can omit memoryThrottlingFactor when I only want memory.min and memory.low protection. I can also leave memoryReservationPolicy at None if my goal is only to slow memory growth before a hard limit is reached. (kubernetes.io)

The upgrade change worth checking first

In Kubernetes 1.37, the default value for memoryThrottlingFactor becomes null. So even though MemoryQoS is enabled by default, the kubelet does not configure memory.high unless the field is set explicitly. The goal is to avoid introducing unexpected memory throttling for workloads that were previously running without it. (kubernetes.io)

There is an important check here for anyone who used this feature before. If the kubelet configuration file already has an explicit memoryThrottlingFactor, that value is retained and throttling continues to work. If you relied on the old default without declaring it, add the field to the kubelet configuration before or during the upgrade. (kubernetes.io)

The fine print matters

Memory QoS requires Linux nodes running cgroup v2. The documentation recommends kernel 5.9 or newer because memory.high throttling on older kernels can trigger a known livelock issue. If a homelab is based on an older distribution, checking the kernel version matters just as much as updating Kubernetes manifests. (kubernetes.io)

It also does not turn requests into a magic estimate of what an application really needs. With TieredReservation, the policy applies to every Pod on the node according to its QoS class; there is no per-Pod opt-in or opt-out for reservation. Before enabling it on a mixed-use node, decide which workloads truly need protection and which ones should remain reclaimable under pressure. (kubernetes.io)

There is a particularly tricky case with Guaranteed Pods that use a large page cache. Because request and limit are equal, memory.min can equal memory.max; if the limit leaves no headroom for cache, the kernel may be unable to reclaim enough memory and an OOM kill can still happen. Protection does not replace sensible memory sizing. (kubernetes.io)

It is also worth remembering that Memory QoS does not remove limits or change a Pod’s QoS classification. A container that exceeds its memory limit can still be killed and restarted. The feature gives the kernel more precise guidance before that point, but it cannot fix a memory leak or compensate for badly chosen resource settings. (kubernetes.io)

How I would introduce it in a homelab

I would start by identifying services that genuinely need to survive a memory fight: stateful storage, home automation, DNS, monitoring, and cluster components. Then I would verify that their Pods are actually Guaranteed, including sidecars and init containers where applicable. I would enable TieredReservation on a test node first and observe the result before rolling it out across the cluster. (kubernetes.io)

For elastic or less important services, I would use requests and limits that keep them Burstable. In that category, memory.low provides softer protection and memory.high can start slowing growth before the hard limit is hit. But memoryThrottlingFactor should not be copied blindly: it is a capacity and latency decision for each node. (kubernetes.io)

The useful part of Kubernetes 1.37 is not simply that it enables a switch. It is that it gives us a better way to distinguish services that should retain memory from workloads that can give it back, using resource declarations we already maintain. On small homelab nodes with many services sharing the same machine, that distinction can matter more than adding another server.

Source: Kubernetes v1.37: Memory QoS Graduates to Beta

Official documentation: Pod Quality of Service Classes and Memory QoS