Mumbai University > Information Technology > Sem 5 > Open Source Technology

Marks: 10M

Year: Dec 2015

The ability to use pipes and redirects to combine Linux commands in an effi cient way can save administrators lots of time. The piping and redirection options are among the most powerful features of the Linux command line. Piping is used to send the result of a command to another command, and redirection sends the output of a command to a file.

Piping:

A pipe is a form of redirection that is used in Linux and other Unix-like operating systems to send the output of one program to another program for further processing.

Examples

A pipe is designated in commands by the vertical bar character, which is located on the same key as the backslash on U.S. keyboards. The general syntax for pipes is: command_1 | command_2 [| command_3 . . . ] This chain can continue for any number of commands or programs. A very simple example of the benefits of piping is provided by the dmesg command, which repeats the startup messages that scroll through the console(i.e., the all-text, full-screen display) while Linux is booting (i.e., starting up). dmesg by itself produces far too many lines of output to fit into a single screen; thus, its output scrolls down the screen at high speed and only the final screenful of messages is easily readable. However, by piping the output of dmesg to the filter less, the startup messages can conveniently be viewed one screenful at a time, i.e.,

dmesg | less

less allows the output of dmesg to be moved forward one screenful at a time by pressing the SPACE bar and back one screenful at a time by pressing the b key. The command can be terminated by pressing the q key. (The more command could have been used here instead of less; however, less is newer than more and has additional functions, including the ability to return to previous pages of the output.) The same result could be achieved by first redirecting the output of dmesg to a temporary file and then displaying the contents of that file on the monitor. For example, the following set of two commands uses the output redirection operator (designated by a rightward facing angle bracket) to first send the output of dmesg to a text file called tempfile1 (which will be created by the output redirection operator if it does not already exist), and then it uses another output redirection operator to transfer the output of tempfile1 to the display screen:

dmesg > tempfile1

tempfile1 > less

However, redirection to a file as an intermediate step is clearly less efficient, both because two separate commands are required and because the second command must await the completion of the first command before it can begin. The use of two pipes to chain three commands together could make the above example even more convenient for some situations. For example, the output of dmesg could first be piped to the sort filter to arrange it into alphabetic order before piping it to less:

dmesg | sort -f | less

The -f option tells sort to disregard case (i.e., whether letters are lower case or upper case) while sorting. Likewise, the output of the ls command (which is used to list the contents of a directory) is commonly piped to the the less (or more) command to make the output easier to read, i.e.,

ls -al | less

or

ls -al | more

ls reports the contents of the current directory (i.e., the directory in which the user is currently working) in the absence of any arguments (i.e., input data in the form of the names of files or directories). The -l option tells ls to provide detailed information about each item, and the -aoption tells ls to include all files, including hidden files (i.e., files that are normally not visible to users). Because ls returns its output in alphabetic order by default, it is not necessary to pipe its output to the sort command (unless it is desired to perform a different type of sorting, such as reverse sorting, in which case sort's -r option would be used).

This could just as easily be done for any other directory. For example, the following would list the contents of the /bin directory (which contains user commands) in a convenient paged format:

ls -al /bin | less

The following example employs a pipe to combine the ls and the wc (i.e., word count) commands in order to show how many filesystem objects (i.e., files, directories and links) are in the current directory:

ls | wc -l

ls lists each object, one per line, and this list is then piped to wc, which, when used with its -l option, counts the number of lines and writes the result to standard output (which, as usual, is by default the display screen). The output from a pipeline of commands can be just as easily redirected to a file (where it is written to that file) or a printer (where it is printed on paper). In the case of the above example, the output could be redirected to a file named, for instance, count.txt:

ls | wc -l > count.txt

The output redirection operator will create count.txt if it does not exist or overwrite it if it already exists. (The file does not, of course, require the .txt extension, and it could have just as easily been named count, lines or anything else.) The following is a slightly more complex example of combining a pipe with redirection to a file: echo -e "orange \npeach \ncherry" | sort > fruit

The echo command tells the computer to send the text that follows it to standard output, and its -e option tells the computer to interpret each \nas the newline symbol (which is used to start a new line in the output). The pipe redirects the output from echo -e to the sort command, which arranges it alphabetically, after which it is redirected by the output redirection operator to the file fruit. As a final example, and to further illustrate the great power and flexibility that pipes can provide, the following uses three pipes to search the contents of all of the files in current directory and display the total number of lines in them that contain the string Linux but not the string UNIX:

cat * | grep "Linux" | grep -v "UNIX" | wc -l

In the first of the four segments of this pipeline, the cat command, which is used to read and concatenate (i.e., string together) the contents of files, concatenates the contents of all of the files in the current directory. The asterisk is a wildcard that represents all items in a specified directory, and in this case it serves as an argument to cat to represent all objects in the current directory.

The first pipe sends the output of cat to the grep command, which is used to search text. The Linux argument tells grep to return only those lines that contain the string Linux. The second pipe sends these lines to another instance of grep, which, in turn, with its -v option, eliminates those lines that contain the string UNIX. Finally, the third pipe sends this output to wc -l, which counts the number of lines and writes the result to the display screen.

Redirects:

In order to understand about Redirecting operators in Linux we should know how we communicate with a computer. When we are communicating with a computer there should be a way to do it and this is achieved by STDIN (0), STDOUT (1), STDERR (2) file descriptors. These file descriptors are assigned with a number as shown below.

STDIN –> 0 STDOUT –> 1 STDERR –> 2

STDIN: stands for STandarD Input

STDOUT: The abbreviation is STandarD OUTput

STDERR: This is abbreviated as STandarD ERRor

Example1: Redirect output of a command to a file for future reference. Example: Redirect output of fdisk -l to a file file1.txt fdisk -l > file1.txt The above command will save fdisk -l command output to file1.txt and you will not see any output when you execute above command. Example: Redirect the content of /etc/password file to xyz.txt file. cat

/etc/passwd > xyz.txt

Example2: Use output redirect append operators to redirect fdisk command output once again so that I can see both the previous output and present one in single file.

fdisk -l >> file1.txt

Example3: Redirect error of a command to a file and output to the screen.

ls -hrtfd 2> file3.txt

Exampl4: Redirect and append error to a file and output to the screen.

ls -wertdf 2>> file4.txt

Exampl57: Redirect both output and error to a file

ls -ewrdter &> file5.txt