Cluster Autoscaling

Nodes are added and removed for you as your workloads grow and shrink. This
article explains how that works on AWS environments, what the node pools are,
how to pin a workload that must never be interrupted, and what to check when
scaling does not behave as you expect.

How autoscaling works

AWS environments use Karpenter for node autoscaling. No action is required
from your team for routine scaling.

When a pod cannot be scheduled because no node has enough available resources,
Karpenter:

1. Selects an appropriate node type based on the pod's resource requests.
2. Launches a new instance and registers it with the cluster.
3. The pod is scheduled onto the new node.

When nodes are underutilised, Karpenter consolidates workloads onto fewer nodes
and terminates the excess.

Autoscaling decisions are driven entirely by your resource requests, so
accurate requests matter more here than anywhere else. Setting them is covered
in Workload Resources and Health Checks.

Node pools

Each cluster has two general node pools and two pools reserved for jobs.

The job pools only start nodes while jobs are running on them, and only
workloads that opt in are placed there — see
CronJobs for how a CronJob opts in with two settings.

Your Components are scheduled onto the appropriate pool based on their
configuration. Node pool settings for your Environment are adjustable in SVIEW.

SPOT instances can be reclaimed by AWS at two minutes' notice. Stateless
workloads should be written to tolerate a restart at any time — handle SIGTERM
and shut down cleanly. Stateful workloads belong on the data pool.

Configuration — pinning workloads off SPOT nodes

Some workloads must not tolerate a SPOT interruption at all: a long-running
batch job that cannot resume from a partial state, or a migration that must run
to completion.

For a CronJob, set dedicatedNodes: true as described in CronJobs.
For any other workload, such as a long-running worker, pin the deployment or
pod to the uninterruptible-jobs pool with a matching node affinity and
toleration:

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: pool-type
              operator: In
              values:
                - uninterruptible-jobs
tolerations:
  - key: "nodepool-type"
    operator: "Equal"
    value: "uninterruptible-jobs"
    effect: "NoSchedule"

Use interruptible-jobs in both places instead for work that can safely be
re-run. Both blocks are needed: the affinity sends the pod to that pool, and
the toleration lets it past the taint that keeps other workloads off it. If
your Environment has no such pool, raise a request.

Troubleshooting scaling issues

If the problem persists after these checks, raise a request with the affected
namespace, the Component name, and the pod status you are seeing.

Related articles

• Event-Driven Autoscaling with KEDA — scaling workers on queued work rather than CPU
• Workload Resources and Health Checks
• Pod Stuck in Pending (FailedScheduling) — reading the scheduling message when a pod never starts
• Components and Deployments
• Getting Help