High CPU Usage

Overview

CPU pegged at 100% (or a load average far above core count) usually has one of two shapes: a single runaway process, or many processes collectively saturating the box. The investigation path differs for each.

The Problem

A server is sluggish, alerts show CPU at 90-100%, and users report slow responses or timeouts.

Investigation

uptime                       # load average vs. nproc — is this actually CPU-bound?
nproc                          # number of cores, for context

top -o %CPU                      # live view, sorted by CPU — usually shows the culprit immediately
ps aux --sort=-%cpu | head -10     # top 10 CPU consumers, snapshot
mpstat -P ALL 1 5                    # per-core breakdown — one core pegged vs. all cores busy tells different stories

Once you’ve found the offending PID:

ps -p <PID> -o %cpu,%mem,etime,cmd    # how long has it been running? memory usage too?
cat /proc/<PID>/status                  # detailed process state
top -H -p <PID>                           # per-THREAD CPU usage within that one process

For a deeper look at what the process is actually doing:

strace -c -p <PID>          # summarize which syscalls it's making, and how often
perf top -p <PID>              # (if perf is installed) sampling profiler — shows hot functions

Root Cause Possibilities

• A single runaway process — infinite loop, retry storm, or a stuck request handler that never returns.
• Too many processes for the box — legitimate load has outgrown the instance size.
• A cron job overlap — the same scheduled task running concurrently because a previous run never finished (see Cron & Scheduling).
• Garbage collection thrashing (JVM/Node apps) — CPU spent almost entirely on GC instead of real work, often paired with high memory pressure.
• A crypto-mining or compromise scenario — an unfamiliar process consuming CPU that nobody on the team recognizes.

Resolution

# If it's a genuine bug/runaway process — stop it gracefully first
kill -15 <PID>          # SIGTERM: let it clean up
kill -9 <PID>              # SIGKILL: only if it doesn't respond to SIGTERM

# If it's a resource-sizing issue (legitimate load, undersized instance)
# — scale vertically (bigger instance) or horizontally (more instances/replicas)

# If it's an unfamiliar/suspicious process
ps -p <PID> -o cmd,user,lstart    # who started it, and when
lsof -p <PID>                        # what files/network connections it has open
# isolate and investigate before killing — this may be a security incident, not a bug

Prevention

• Set resource limits (cgroups, Kubernetes resources.limits.cpu) so one runaway process can’t starve the whole host.
• Add flock to cron jobs that could overlap (see Cron & Scheduling tutorial) to prevent overlapping runs from compounding CPU load.
• Alert on load average relative to core count (load / nproc), not an absolute number — the same load average means very different things on a 2-core vs. 32-core box.
• Baseline normal CPU usage per service so an unfamiliar spike is caught quickly, not normalized away.

Summary

High CPU is diagnosed top-down: top/ps --sort=-%cpu to find the process, then top -H -p or strace -c -p to see what it’s actually doing inside. The fix depends entirely on the root cause — kill a runaway process, scale a genuinely undersized host, fix an overlapping cron job, or escalate as a possible security incident for an unrecognized process.

Pro Tip

Always check mpstat -P ALL before assuming it’s “one runaway process” — a single core pegged at 100% while others sit idle often points to a single-threaded bottleneck in the application itself, not something kill will meaningfully fix without a code or configuration change.