Back to articles
Internals June 2026

How the Kubernetes scheduler actually picks a node

For platform engineers who want to understand the scheduler's decision-making process well enough to predict and explain its behavior.

The Kubernetes scheduler is one of the most consequential components in your cluster, yet most engineers treat it as a black box. A pod goes in; it lands somewhere. When things work, nobody asks why. When pods end up on the wrong node or refuse to schedule at all, the debugging starts blind.

This article is for platform engineers who want to understand the scheduler's decision-making process well enough to predict and explain its behavior. By the end, you should be able to look at a cluster and know why a pod landed where it did, why certain configurations lead to hotspots, and how to debug scheduling decisions without guessing.

What problem does understanding this solve?

Scheduler opacity creates several recurring problems:

The scheduler is deterministic. Given the same inputs, it makes the same decision. Once you understand the inputs and the algorithm, scheduling behavior becomes predictable.

The scheduling cycle: two phases

Every unscheduled pod goes through a scheduling cycle that consists of two phases:

  1. Filtering — Eliminate nodes that cannot run the pod.
  2. Scoring — Rank the remaining nodes to find the best fit.

After scoring, the scheduler picks the highest-scoring node and binds the pod to it. If multiple nodes tie for the highest score, the scheduler picks one at random. If no nodes pass filtering, the pod stays Pending.

This two-phase model is the core mental model. Everything else — plugins, extension points, scheduler profiles — is implementation detail around these two phases.

Filtering: what disqualifies a node

Filtering applies a series of predicates (called Filter plugins in the current scheduler framework) that each return a binary answer: can this node run this pod, yes or no?

A node must pass every filter to remain a candidate. One failure eliminates it.

Default Filter plugins in Kubernetes 1.30+

The default scheduler profile includes roughly 15 filter plugins. Here are the ones that cause the most scheduling failures:

Plugin What it checks
NodeResourcesFit Does the node have enough allocatable CPU, memory, and extended resources (GPUs, etc.) to satisfy the pod's requests?
NodePorts Are the host ports requested by the pod's containers available on this node?
NodeAffinity Does the node satisfy the pod's nodeAffinity rules?
PodTopologySpread Does scheduling here violate the pod's topology spread constraints?
InterPodAffinity Does scheduling here satisfy or violate inter-pod affinity/anti-affinity rules?
TaintToleration Does the pod tolerate all taints on the node?
NodeUnschedulable Is the node marked unschedulable (cordoned)?
VolumeBinding Can the required persistent volumes be bound or provisioned on this node?
VolumeRestrictions Does the pod violate volume access mode restrictions (e.g., ReadWriteOnce already mounted elsewhere)?
VolumeZone Is the node in a zone where the required volumes exist?

How filtering actually fails

When you see a Pending pod with the event message 0/5 nodes are available, the scheduler is telling you that every node failed at least one filter. The message usually includes a breakdown:

0/5 nodes are available: 2 Insufficient cpu, 2 node(s) had taint {node-role.kubernetes.io/control-plane: }, 1 node(s) didn't match Pod's node affinity/selector.

This tells you exactly which filters rejected which nodes. If you're debugging, start here.

The resource fit filter in detail

NodeResourcesFit is the most common culprit for pending pods. It compares the pod's resource requests (not limits) against the node's allocatable capacity minus what's already requested by scheduled pods.

Key points:

Scoring: how nodes are ranked

After filtering, the scheduler runs Score plugins against every surviving node. Each plugin returns a score from 0 to 100 for each node. The scheduler normalizes and weights these scores, then sums them to produce a final score per node.

Default Score plugins in Kubernetes 1.30+

Plugin What it rewards Weight
NodeResourcesBalancedAllocation Nodes where CPU and memory usage are balanced (neither starved nor oversupplied) 1
NodeResourcesFit (scoring mode) Depends on strategy: LeastAllocated favors emptier nodes; MostAllocated favors fuller nodes 1
InterPodAffinity Nodes that satisfy preferred pod affinity rules 1
NodeAffinity Nodes that match preferred node affinity rules 1
PodTopologySpread Nodes that improve topology spread evenness 2
TaintToleration Nodes with fewer untolerated taints (prefer nodes with fewer taints overall) 1
ImageLocality Nodes that already have the pod's container images cached 1

Resource scoring strategies

The NodeResourcesFit plugin supports three scoring strategies configured via scheduler profiles:

  1. LeastAllocated (default): Prefer nodes with the most remaining capacity. This spreads load across nodes.
  2. MostAllocated: Prefer nodes with the least remaining capacity. This packs pods tightly, useful for cost optimization with cluster autoscaler.
  3. RequestedToCapacityRatio: Score based on a custom curve that you define.

The default LeastAllocated strategy is why, all else being equal, pods spread across nodes rather than packing onto a few.

Why image locality matters less than you'd think

ImageLocality gives a small boost to nodes that have already pulled the pod's container images. In theory, this reduces pull time. In practice, the weight is low (1) and images on modern registries pull fast. You'll rarely see this plugin change a scheduling decision unless nodes are otherwise tied.

Inter-pod affinity and anti-affinity: the expensive rules

InterPodAffinity is the most computationally expensive part of scheduling. It requires the scheduler to examine pods already running on each node to determine if they match the affinity/anti-affinity selectors.

Required vs. preferred

Affinity rules come in two strengths:

The cost of anti-affinity at scale

Anti-affinity rules like "don't schedule this pod on a node that already runs a pod with label app=web" require the scheduler to iterate through pods on every candidate node. For clusters with thousands of pods, this adds measurable latency to scheduling cycles.

If you're running a Deployment with podAntiAffinity set to requiredDuringSchedulingIgnoredDuringExecution and topologyKey: kubernetes.io/hostname, you're telling the scheduler: "Never put two replicas on the same node." This works, but it also means each replica consumes an entire node from the scheduler's perspective. If you have 50 replicas and 10 nodes, you cannot schedule.

A worked example

Consider a Deployment with these constraints:

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          app: cache
      topologyKey: kubernetes.io/hostname

This says: "Do not schedule a cache pod on any node that already has a cache pod." The scheduler must, for each candidate node:

  1. List all pods on that node.
  2. Check if any pod matches the label selector app=cache.
  3. If yes, fail the filter for this node.

For 100 candidate nodes with 50 pods each, that's 5,000 label comparisons per scheduling cycle for one pod. The scheduler caches aggressively to mitigate this, but the cost is real.

The scheduler framework and extension points

Since Kubernetes 1.19, the scheduler is built on a plugin architecture called the Scheduler Framework. The two-phase model (filter, score) is implemented as extension points where plugins hook in.

Extension points

Extension point When it runs Purpose
PreFilter Before filtering Compute state that filters will need (e.g., precompute affinity matches)
Filter During filtering Eliminate nodes that cannot run the pod
PostFilter After filtering (only if all nodes were filtered out) Attempt preemption or other recovery
PreScore Before scoring Compute state that scorers will need
Score During scoring Assign a score to each surviving node
NormalizeScore After scoring Normalize scores to 0-100 range
Reserve After node selection Temporarily reserve resources on the chosen node
Permit Before binding Allow, deny, or delay binding (used by gang scheduling)
PreBind Before binding Run actions that must complete before binding
Bind Binding Write the pod-to-node binding to etcd
PostBind After binding Run cleanup or notification actions

Why this matters to operators

If you're running third-party scheduler plugins (e.g., from a service mesh, GPU scheduler, or gang scheduler), they hook into these extension points. When scheduling behaves unexpectedly, knowing which plugins are registered at which extension points helps you identify the culprit.

You can inspect the scheduler's configuration:

kubectl get configmap -n kube-system kube-scheduler -o yaml

Or, if your distribution uses a different configuration method, check the scheduler's --config flag and the associated KubeSchedulerConfiguration object.

Scheduler profiles: running multiple schedulers in one

Kubernetes supports scheduler profiles — distinct configurations that run within the same scheduler binary. Each profile has its own name, plugins, and plugin configurations.

A common use case: running a default profile with LeastAllocated for general workloads and a second profile with MostAllocated for batch jobs where bin packing saves cost.

To use a specific profile, set spec.schedulerName on the pod:

spec:
  schedulerName: batch-scheduler

If the pod doesn't specify a scheduler name, it uses default-scheduler.

Real examples of scheduler decisions

Example 1: Pod pending due to resource fragmentation

Cluster state:

New pod requests 1 CPU.

Filtering result: Node A has 0.5 CPU available — fails NodeResourcesFit. Node B has 0.8 CPU available — fails. Node C has 0.2 CPU available — fails. Pod stays Pending even though the cluster has 1.5 CPUs "free" in aggregate.

Lesson: Resource fragmentation is a real problem. The scheduler cannot defragment by moving pods. Consider the descheduler project or overprovisioning.

Example 2: Unintended node concentration

Cluster state:

What happens: The scheduler uses LeastAllocated scoring. After scheduling the first pod, one node has slightly more allocation. The second pod goes to a different node. This continues until pods are spread across all 5 nodes.

However, if 4 nodes have DaemonSets consuming 1.5 CPUs each and the fifth node doesn't (e.g., it's new and DaemonSets haven't rolled out yet), the fifth node has significantly more remaining capacity. The scheduler will favor that node heavily until the DaemonSets arrive.

Lesson: DaemonSet resource requests materially affect scheduling distribution. New nodes can temporarily absorb disproportionate load.

Example 3: Topology spread with skew

Pod spec:

topologySpreadConstraints:
- maxSkew: 1
  topologyKey: topology.kubernetes.io/zone
  whenUnsatisfiable: DoNotSchedule
  labelSelector:
    matchLabels:
      app: api

Cluster state:

New pod attempts to schedule.

Filtering result: The constraint says skew cannot exceed 1. Current skew is 1 (Zone A has 3, others have 2). Scheduling another pod in Zone A would make skew 2 — fails. The pod can schedule in Zone B or C.

Scoring result: Zone B and C both satisfy the constraint. If resources are equal, the scheduler may pick either or use other scoring factors to break the tie.

Lesson: maxSkew is evaluated after the hypothetical scheduling. The scheduler checks: "If I put this pod here, would skew exceed the limit?"

Debugging why a pod landed on a specific node

When a pod schedules and you want to know why, you have several tools:

1. Check scheduler events

kubectl describe pod <pod-name>

Look at the Events section. You'll see a Scheduled event with the message:

Successfully assigned default/my-pod to node-3

This confirms which scheduler made the decision and which node was chosen. If scheduling failed, you'll see FailedScheduling with details.

2. Enable verbose scheduler logging

If you control the scheduler, increase the verbosity:

--v=4  # Shows scoring details
--v=5  # Shows filtering details

At verbosity 4, you'll see log lines like:

"Score plugin result" plugin="NodeResourcesFit" pod="default/my-pod" node="node-3" score=75

This tells you exactly how each plugin scored each node.

3. Use scheduler simulator (for what-if analysis)

The kube-scheduler-simulator project lets you load cluster state and simulate scheduling decisions in a UI. This is useful for capacity planning and debugging complex interactions.

4. Examine the pod's requests and node allocatable

kubectl describe node <node-name>

Look at Allocatable and Allocated resources. Compare against your pod's requests. If the numbers don't add up, check for DaemonSets, static pods, or pods in terminating state that still consume reservations.

Scheduler performance and limits

The default scheduler is single-threaded for the scheduling cycle but parallelizes binding. For most clusters, this is fine. The scheduler can process hundreds of pods per second.

At scale (thousands of pods pending simultaneously), scheduling throughput becomes a concern. Common mitigations:

Summary

The Kubernetes scheduler is a two-phase machine: filter nodes that can't run the pod, score nodes that can, pick the highest scorer. The machinery under this — plugins, extension points, profiles — is flexible, but the core algorithm is predictable once you understand it.

Understanding this matters when:

The scheduler is deterministic. Given the same cluster state and pod spec, it makes the same decision. If the decision seems wrong, the cluster state or pod spec isn't what you think it is. That's where debugging starts.