The pipe (|) is one of the foundational things in UNIX. The Pipe allows you to redirect STDOUT to STDIN of the next commend.
Understanding STDIN, STDOUT, and STDERR is important to better understand the pipe. I have an article which explains STDIN, STDOUT, and STDERR here. Make sure you read it before proceeding.
Now that you understand STDIN, STDOUT, and STDERR, we can proceed with explaining the pipe (|) functionality.
Let’s start with a simple command:
ls -l /usr/bin
You will probably see too much information for the eye to catch, now let’s assume that you are searching for the all the files containing the file word. Let’s use the pipe redirection to accomplish that, try the following command:
ls -l /usr/bin | grep file
Now you will notice that the output is greatly reduced to only what you are looking for, great, but what really happened behind the scenes with the pipe STDOUT/SUDIN redirection? The command ls -l /usr/bin outputs all the information to STDOUT and without the pipe, it simply goes uncaptured and gets displayed to the user terminal session. When we add the pipe | the STDOUT gets captures and redirected to STDIN of the command grep file. The command grep file simply parses the STDIN line-by-line and if the word file is found, the line is displayed.
The thing that is not really clear to many users is that the pipe only redirects STDOUT to STDIN, leaving STDERR. So if the command gives an error, the error is simply displayed to the user terminal without being captured by the pipe. For example, let’s run the following command:
ls -l /usr/binn | grep file
You should see the following output:
ls: cannot access ‘/usr/binn’: No such file or directory
If the command would have also redirected STDERR, which was the stream of the error message, we should have seen an empty string. In order to also redirect STDERR to the pipe command, we need to redirect STDERR to STDIN, at that point, the pipe will pick it up and pass it to the next command. The following example shows the STDERR to STDIN redirection:
ls -l /usr/binn 2>&1 | grep abc
If you execute the above command, you will get nothing, an empty string, which is the expected behavior.
Redirecting STDERR is very important when all the output has to be caught. UNIX gives you this flexibility to use at your discretion.
Sometimes in script you need to mask the error and only forwards STDOUT to the next command. The following example will write STDERR to /dev/null device, which as the name says, the output is nullified. Let’s try the following command:
ls -l /usr/bin 2>/dev/null | grep file
Now in this case, STDERR is not displayed nor passed to the next command.
Let’s summarize few important things about the pipe:
- The pipe redirects STDOUT from the previous command as STDIN of the next command
- The pipe only redirects STDOUT not STDERR
- In order to also redirect STDERR, you have to redirect STDERR to STDOUT with 2>&1
I hope that the pipe is clearer now and that I have helped you in your UNIX journey. Have fun and type away…