Redirection

Core idea

Every Unix program automatically gets three open channels at startup: standard input (stdin, file descriptor 0), standard output (stdout, file descriptor 1), and standard error (stderr, file descriptor 2). By default stdin is the keyboard, stdout is the terminal, and stderr is also the terminal. The shell lets you redirect any of these to a file, or pipe them between programs — and that single mechanism is the source of most of the command line's power.

The Unix philosophy crystallises here: write small programs that do one thing well, have them read from stdin and write to stdout, and use redirection and pipes to compose them into bigger workflows. ls lists files. sort sorts lines. uniq removes duplicates. wc -l counts lines. None of them know about each other. Yet ls /usr/bin /bin | sort | uniq | wc -l counts the unique program names across both directories — a useful piece of work that nobody had to write.

Why it matters

Mental model

The three streams

A running command is a black box with three pipes attached. Two emerge from it (stdout and stderr), one feeds into it (stdin). By default they are all wired to the terminal; redirection re-wires them.

Copy sourceZoom

Redirection operators

| Operator | Meaning | |---|---| | > file | Send stdout to file, overwriting it. | | >> file | Append stdout to file. | | < file | Read stdin from file. | | 2> file | Send stderr to file. | | 2>> file | Append stderr to file. | | 2>&1 | Send stderr to wherever stdout is currently going. | | &> file (bash) | Send both stdout and stderr to file. | | &>> file (bash) | Append both stdout and stderr to file. | | > /dev/null 2>&1 | Discard everything. |

A subtle but important rule: 2>&1 must come after the stdout redirection. cmd > out.log 2>&1 correctly sends both streams to out.log. cmd 2>&1 > out.log ties stderr to the terminal first, then redirects stdout to the file — different result.

Pipes are the killer feature

A pipe connects the stdout of one command directly to the stdin of the next, in memory, without an intermediate file. The notation is |.

Copy sourceZoom

Pipes vs redirection — the most common confusion

cmd > file      writes cmd's stdout to a FILE (the destination is a filename)
cmd | other     pipes cmd's stdout to OTHER's stdin (the destination is another command)

A typo here is dangerous: ls > less does not page ls's output — it overwrites a file named less in the current directory with ls's output. There is a true story in the source book of a sysadmin who, as root in /usr/bin, typed ls > less and obliterated the less program file on his server. Always think about whether you want a file or a pipeline before you press the operator.

Practical application

Saving and combining streams

Building a useful pipeline

The standard text-toolkit verbs are cat, sort, uniq, grep, wc, head, tail, and tee. A few classic compositions:

# How many unique program names are installed?
ls /usr/bin /bin | sort | uniq | wc -l

# Show only duplicates between /bin and /usr/bin
ls /usr/bin /bin | sort | uniq -d

# Find every file under /etc that mentions "ssh"
grep -rli ssh /etc/ 2>/dev/null

# Watch a logfile grow in real time
tail -f /var/log/syslog

# Capture a pipeline's intermediate state while still passing it on
ls /usr/bin | tee bin-listing.txt | grep zip

The last example uses tee — a "tee fitting" in the pipe metaphor. It reads stdin, writes it to one or more files, and also forwards it down the pipeline. Use tee when you want to inspect an intermediate stage without breaking the chain, or when you want both a logfile and a live view.

cat and stdin

cat with no filename reads from stdin until end-of-file (Ctrl+D). This makes it the world's simplest text editor:

$ cat > notes.txt
The quick brown fox
jumps over the lazy dog
[Ctrl+D]

$ cat notes.txt
The quick brown fox
jumps over the lazy dog

That same behaviour explains how cat participates in pipelines — when its stdin is connected to another command's stdout, it copies it through unchanged.

Example

You want to find every Python file in a project that imports requests, count them, and save the list — but without filling your screen with permission errors from directories you cannot read.

$ grep -rln "import requests" ~/code/ 2>/dev/null | tee py-with-requests.txt | wc -l
17

Reading the pipeline left to right:

• grep -rln "import requests" ~/code/ — recursively (-r) search for the pattern; -l makes grep print only filenames (not matching lines); -n is harmless here but a habit worth keeping for line numbers.
• 2>/dev/null — silence permission-denied errors by routing stderr to the bit bucket.
• | tee py-with-requests.txt — write the filename list to a file and keep passing the list onward.
• | wc -l — count the lines (one per filename), printing 17.

Now py-with-requests.txt contains the full list and your terminal shows the count. One line, four commands, three streams handled correctly. No script, no temporary files, no library — just the shell composing existing tools the way Unix has been doing it for half a century.

Related lessons