Redirection & Pipes

The Three Standard Streams

Every command has three open file descriptors by default: 0 (stdin, input), 1 (stdout, normal output), 2 (stderr, error output). Understanding these numbers is the key to understanding every redirection operator.

Basic Redirection

command > output.txt         # redirect stdout, OVERWRITE
command >> output.txt          # redirect stdout, APPEND
command < input.txt              # redirect stdin FROM a file
command 2> errors.txt              # redirect stderr only
command 2>> errors.txt               # append stderr

command > output.txt 2>&1        # stdout AND stderr both to the same file (order matters! see below)
command &> output.txt              # shorthand for the line above, bash-only
command > /dev/null 2>&1             # discard ALL output — common in cron jobs/background daemons

Why 2>&1 Order Matters

command > output.txt 2>&1     # CORRECT: stdout -> file, THEN stderr -> "wherever stdout now points" (the file)
command 2>&1 > output.txt       # WRONG (usually): stderr -> "wherever stdout currently points" (the terminal), THEN stdout -> file

Redirections are processed left to right — 2>&1 must come after stdout has already been redirected to the file for both streams to end up there.

Pipes: Connecting Commands

cat access.log | grep "404" | wc -l
ps aux | grep nginx | grep -v grep
cat file.txt | sort | uniq -c | sort -rn

A pipe connects one command’s stdout to the next command’s stdin. Note: stderr is NOT piped by default — only stdout is.

command 2>&1 | tee log.txt      # pipe BOTH stdout and stderr into the next command

tee: Split Output to a File AND the Screen

long_running_task.sh | tee output.log            # see it live AND save it
long_running_task.sh | tee -a output.log            # append instead of overwrite
command | tee /dev/tty | wc -l                        # see raw output while also counting lines

Here Documents & Here Strings

cat <<EOF
Multi-line
text block
EOF

mysql -u root -p mydb <<SQL
SELECT * FROM users LIMIT 10;
SQL

grep "pattern" <<< "$VARIABLE"    # here-string: feed a single variable's content as stdin

Process Substitution

Process substitution treats a command’s output as if it were a file — useful when a tool expects a filename but you have a command’s output instead.

diff <(sort file1.txt) <(sort file2.txt)      # compare two commands' output as if they were files
while read -r line; do echo "$line"; done < <(curl -s https://api.example.com/list)

Custom File Descriptors

exec 3> custom_log.txt        # open FD 3 for writing
echo "logged via FD 3" >&3
exec 3>&-                       # close FD 3

# Swap stdout and stderr temporarily
command 3>&1 1>&2 2>&3 3>&-

A Practical Example: Logging Script Output

#!/usr/bin/env bash
LOGFILE="/var/log/myapp/deploy.log"

exec > >(tee -a "$LOGFILE") 2>&1    # send ALL script output (stdout+stderr) to both terminal and log file

echo "Starting deployment..."
./run_deploy_steps.sh
echo "Deployment complete"

Production Considerations

• Always separate stdout and stderr in production scripts run unattended (cron, systemd) — mixing them makes it harder to distinguish real errors from normal progress output when reviewing logs.
• command > file 2>&1 vs command 2>&1 > file produce genuinely different results — this ordering mistake is one of the most common redirection bugs.
• Use set -o pipefail (see Exit Codes & Error Handling) so a failure in an EARLY stage of a pipeline isn’t silently masked by a later stage’s success.

Quick Interview Answer

“stdin (0), stdout (1), and stderr (2) are the three default file descriptors. > overwrites, >> appends, and 2>&1 redirects stderr to wherever stdout currently points — order matters, since redirections apply left to right. Pipes connect stdout of one command to stdin of the next, but NOT stderr by default. tee splits output to both a file and the screen, and process substitution (<(command)) lets a command’s output be used where a filename is expected.”

Common Mistakes

• Writing command 2>&1 > file when command > file 2>&1 was intended, losing stderr to the terminal instead of the file.
• Forgetting pipes don’t carry stderr by default, missing error output when debugging a pipeline.
• Redirecting output to /dev/null too aggressively during debugging, hiding the exact error message needed to diagnose a failure.