High Load Average

Overview

Load average is one of the most misunderstood Linux metrics. It’s not a CPU percentage — it’s roughly the average number of processes that are either running or waiting for a resource (CPU or uninterruptible I/O) over the last 1, 5, and 15 minutes.

The Problem

uptime shows a load average that looks alarmingly high, but it’s not immediately clear whether the server is actually in trouble, or what’s causing it.

Investigation

uptime
# load average: 12.50, 8.30, 5.10
nproc               # how many cores does this box have? Load must be read RELATIVE to this.

A load of 12 means very different things on a 4-core box (severely overloaded) versus a 32-core box (barely utilized).

# Is it CPU-bound or I/O-bound? This is the critical fork in the investigation.
top                    # check overall %CPU usage
ps aux | awk '$8 ~ /R/' | wc -l    # count processes in Running state
ps aux | awk '$8 ~ /D/' | wc -l      # count processes in UNINTERRUPTIBLE SLEEP (D) state

vmstat 1 5                              # 'r' column = runnable processes, 'b' column = blocked processes
iostat -x 1 5                              # if many processes are in D state, check disk %util/await too

flowchart TD A[High load average] --> B{CPU usage also high?} B -->|Yes| C[Genuinely CPU-bound — see High CPU Usage runbook] B -->|No, CPU is low| D{Processes in D state?} D -->|Yes| E[I/O-bound — see Slow Disk I/O runbook] D -->|No| F[Check for excessive process creation/context switching]

Root Cause Possibilities

Genuine CPU saturation — many processes actively competing for CPU time (confirm with high %CPU in top).
I/O wait dominating — processes stuck in D state waiting on disk or network storage, inflating load without high CPU usage (see Slow Disk I/O).
Fork bomb or runaway process spawning — a bug or misconfiguration rapidly creating new processes.
Misreading load relative to core count — the load itself may be entirely normal for the box’s size; the “problem” is a missing baseline, not an actual issue.
NFS or network-mount hangs — processes accessing an unresponsive network filesystem can pile up in D state, driving load up with no local disk or CPU explanation at all.

Resolution

# If genuinely CPU-bound -> follow the High CPU Usage runbook
# If I/O-bound (D-state processes) -> follow the Slow Disk I/O runbook

# If it's excessive process creation (fork bomb / bug)
ps aux --sort=-pid | head -20    # newest PIDs — a rapidly growing PID count is a strong signal
ulimit -u                           # per-user process limit — worth capping if this is a recurring risk

# If it's an unresponsive NFS mount
mount | grep nfs
umount -f /mount/point                # force unmount if truly hung (data loss risk for in-flight writes — use carefully)

Prevention

Always alert on load average / nproc, not the raw load number — build this ratio into monitoring dashboards so the same threshold makes sense across differently-sized hosts.
Set nofile/nproc ulimits appropriately to contain the blast radius of a runaway process-spawning bug.
Monitor D-state process counts as a distinct metric from CPU-bound load — they point to entirely different remediation paths.
For NFS-dependent workloads, configure sensible mount timeouts (soft vs hard mount options) so an unresponsive server doesn’t hang client processes indefinitely.

Summary

Load average must always be read relative to core count (nproc) — the same number means something completely different on different hardware. The critical diagnostic fork is CPU-bound (high %CPU in top) versus I/O-bound (processes in D state, confirmed with iostat) — these point to entirely different runbooks (High CPU Usage vs. Slow Disk I/O) and different fixes.

Pro Tip

A load average that’s high but STABLE (not climbing) with correspondingly normal application response times is often just… the server’s actual, correctly-sized utilization. Don’t chase a “problem” that’s really just a missing baseline for what normal looks like on that specific host.