Services & systemd

What Is systemd?

systemd is the init system and service manager used by most modern Linux distributions (RHEL, Ubuntu, Debian, Amazon Linux 2+). As PID 1, it starts services in parallel based on a dependency graph rather than a fixed sequential order, and centralizes logging into the journal.

flowchart TD SYSTEMD["systemd (PID 1)"] --> SVC1[nginx.service] SYSTEMD --> SVC2[postgresql.service] SYSTEMD --> SVC3[docker.service] SYSTEMD --> TARGET[multi-user.target] TIMER[backup.timer] -->|triggers| SVC4[backup.service]

Unit Types

Unit	Purpose
`.service`	A long-running daemon or one-shot task
`.timer`	Schedules a `.service` to run (systemd’s cron replacement)
`.socket`	Socket-activated service startup (start on first connection)
`.mount`	Filesystem mount point
`.target`	A named synchronization point / group of units (like `multi-user.target`)

Managing Services with systemctl

sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx
sudo systemctl reload nginx        # reload config without dropping connections (if supported)
sudo systemctl enable nginx        # start automatically on boot
sudo systemctl disable nginx
sudo systemctl status nginx        # current state + recent log lines

systemctl list-units --type=service --state=running
systemctl list-units --failed

Anatomy of a Unit File

# /etc/systemd/system/myapp.service
[Unit]
Description=My Application
After=network.target postgresql.service
Requires=postgresql.service

[Service]
Type=simple
User=deploy
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/start.sh
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target

After= / Before= — ordering only, doesn’t imply a dependency.
Requires= — a hard dependency; if it fails, this unit fails too.
Wants= — a soft dependency; failure doesn’t block this unit.
Restart=on-failure — systemd itself supervises and restarts crashed services.

sudo systemctl daemon-reload      # required after editing any unit file

Logs: journalctl

journalctl -u nginx                 # all logs for one unit
journalctl -u nginx -f               # follow, like tail -f
journalctl -u nginx --since "1 hour ago"
journalctl -p err -b                 # only errors, current boot
journalctl --disk-usage              # how much space the journal is using
sudo journalctl --vacuum-time=7d     # trim journal to last 7 days

Timers — systemd’s Cron Replacement

# backup.timer
[Timer]
OnCalendar=daily
Persistent=true

[Install]
WantedBy=timers.target

systemctl list-timers               # see all scheduled timers and next run time
sudo systemctl enable --now backup.timer

Timers are generally preferred over cron on systemd systems because they integrate with the journal, support Persistent=true (catch up on a missed run after downtime), and can depend on other units.

Production Considerations

Always run systemctl daemon-reload after editing a unit file — otherwise systemd keeps using the old in-memory definition.
Set Restart=on-failure with a sane RestartSec for production services — without it, a crashed service just stays down.
journalctl is bound by size/time retention (/etc/systemd/journald.conf) — for long-term log retention, ship logs to an external system (ELK, CloudWatch, Loki) rather than relying on the local journal.

Quick Interview Answer

“systemd is the modern init system and service manager — it’s PID 1, starts units in parallel based on dependencies declared with After/Requires/Wants, and centralizes logs into the journal. systemctl manages service lifecycle and enablement, journalctl reads logs, and .timer units replace cron with journal-integrated, dependency-aware scheduling.”

Common Mistakes

Editing a .service file and forgetting systemctl daemon-reload, then wondering why changes don’t take effect.
Confusing After= (ordering only) with Requires=/Wants= (actual dependency semantics).
Relying solely on the local journal for logs that need to survive longer than the configured retention window.