UNIX Tutorial

10. Fun with pipes

Commands covered in this section:   cut, sort, pr

Redirection is useful for making a process use a file for input or output, but what if we want to send the output of one process directly to the input of another? To do that, we need a pipe. A pipe is a method of inter-process communication (IPC) which allows a one-way flow of information between two processes on the same machine. We use the "vertical bar" character ("|") to create a pipe between two processes.

Figure 10.1   A UNIX pipe

Exercise 10.1

Let's combine the actions we carried out in the exercises of the previous section into a single command using a pipe. To do this, we will issue the who command, followed by the pipe symbol, followed by the mail command.

$ who | mail -s "let's try a pipe" username@domain.com

The result of this operation is that the output of the who command will be sent directly to the input of the mail command, without the intermediate step of storing the data in a file.

The pipe construct is very powerful, because it allows the user to join several small programs to create a custom tool to solve the problem at hand. This often leads to uses that the developers never imaged while creating the individual programs. We'll use another example to illustrate this idea further.

Let's say you want a sorted list of the usernames for all the users on the system. Since there might be a lot of usernames, you'll also want to display the usernames in several columns. The password database (usually stored in the file /etc/passwd) includes the username along with other information about each user. Each line in the file contains the record for a given user. The individual fields are delimited with the colon character (':') as shown in the example below:

joe:x:60:60:Joe Quigley:/home/joe:/usr/bin/tcsh

On many UNIX systems, the password database is not stored locally on each machine. Instead, it is distibuted via the Network Information Service (NIS). (This was formerly called Yellow Pages, or YP.) If your system uses NIS, you can list the password database by entering ypcat passwd. If your system uses a local password file, you would enter cat /etc/passwd instead.

What we need to solve our problem is a tool that will echo the contents of the password database, cut out the first field (the username) for each entry, sort the resulting list, and then print the list in multiple columns. By combining a number of smaller tools, we can create the tool we need right on the spot. Table 10.1 summarizes the comands we'll use to build our pipeline.

Exercise 10.2

Now let's try out our pipeline:

$ ypcat passwd | cut -f1 -d: | sort | pr -5 | less
aaron         cpresley      jam           lefty         pdank
adriano       cringely      jand          lucinda       pearl
agorman       ddale         joebob        ludmilla      pinky
.             .             .             .             .
.             .             .             .             .
.             .             .             .             .

If it's still not clear how you got the result you did, try removing each piece of the pipeline one at a time, going from right to left. In other words, remove "| less" from the command above, then remove "| pr -5", and so on.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.