Using Bash To Output To Screen And File At The Same Time

EchoI’ve recently written about redirecting the output of commands (standard out and standard error) to a file using bash. That is part of fundamental bash usage, but what if you want to redirect the output of a command to a file but also have that output go to the screen. Well, the bash shell has a handy little command that allows you to do just that. The command is called tee.

Rather than redirecting the output of a command to a file using the standard technique e.g.:

alan@alan-ubuntu-desktop:~/scrap$ ls -al > blah.txt
alan@alan-ubuntu-desktop:~/scrap$

You can instead pipe the output to tee and use that command to save the output to a file and at the same time echo everything to the screen e.g.:

alan@alan-ubuntu-desktop:~/scrap$ ls -al | tee blah.txt
total 8
drwxr-xr-x  2 alan alan 4096 2009-09-29 21:16 .
drwxr-xr-x 37 alan alan 4096 2009-09-29 21:14 ..
-rw-r--r--  1 alan alan    0 2009-09-29 21:16 blah.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:09 file1.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:10 file2.txt
alan@alan-ubuntu-desktop:~/scrap$

Simple but very useful in certain situations.

Echoing and Saving Both Stdout And Stderr

Normally you’re only able to do the above with stdout. After all, you can’t pipe both stdout and stderr to tee. But we can combine redirection with the tee command to both echo and save the output and error streams at the same time. We first redirect standard error to point to the same place as stdout and then we pipe the whole mess to tee in order to both save and echo at the same time e.g.:

alan@alan-ubuntu-desktop:~/scrap$ ls -al 2>&1 | tee blah.txt
total 8
drwxr-xr-x  2 alan alan 4096 2009-09-29 21:16 .
drwxr-xr-x 37 alan alan 4096 2009-09-29 21:19 ..
-rw-r--r--  1 alan alan    0 2009-09-29 21:22 blah.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:09 file1.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:10 file2.txt
alan@alan-ubuntu-desktop:~/scrap$

Echoing and Saving Just Stderr

We run into a bit of trouble however, if we’re only interested in saving and echoing stderr and don’t care about stdout. The trouble is, you can’t pipe stderr, pipes only work with stdout. As you may have guessed we can use some fancier redirection to get around this obstacle :).

File descriptors such as 1 for stdout, 2 for stderr (and 0 for stdin) are not the only file descriptors we can use with bash. Other numbers can be treated as file descriptors as well and most of the time they are just sitting there unused. What we want to do is use one of these extra file descriptors as a temporary variable to allow us to ‘swap’ stdout and stderr e.g.:

  • first we point 3 to where stdout is pointing (the screen), i.e. we essentially make a copy of the stdout file descriptor and assign it to 3
  • we then redirect stdout somewhere else so that it doesn’t interefere with what we are really interested in
  • we then redirect stderr to 3 (where stdout used to point), i.e. in essence we assign the file descriptor that used to be referenced as 1 and is currently referenced as 3 to be referenced as 2 :)
  • the pipe is attached to the file descriptor itself, so doing this swap means we will now pipe error output
alan@alan-ubuntu-desktop:~/scrap$ ls -al 3>&1 1>/dev/null 2>&3- | tee blah.txt
alan@alan-ubuntu-desktop:~/scrap$ ls -al
total 12
drwxr-xr-x  2 alan alan 4096 2009-09-29 21:51 .
drwxr-xr-x 37 alan alan 4096 2009-09-29 21:40 ..
-rw-r--r--  1 alan alan    0 2009-09-29 21:58 blah.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:09 file1.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:10 file2.txt
-rw-r--r--  1 alan alan  385 2009-09-29 21:46 yadda.txt
alan@alan-ubuntu-desktop:~/scrap$

Hmm, this looks like nothing has happened and the blah.txt file is empty, so we got no output. This is because we didn’t get any errors on the previous command, lets simulate an error by breaking our ls command on purpose e.g.:

alan@alan-ubuntu-desktop:~/scrap$ ls - garbage 3>&1 1>/dev/null 2>&3- | tee blah.txt
ls: cannot access -: No such file or directory
ls: cannot access garbage: No such file or directory
alan@alan-ubuntu-desktop:~/scrap$ ls -al
total 16
drwxr-xr-x  2 alan alan 4096 2009-09-29 21:51 .
drwxr-xr-x 37 alan alan 4096 2009-09-29 21:40 ..
-rw-r--r--  1 alan alan  100 2009-09-29 22:00 blah.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:09 file1.txt
-rw-r--r--  1 alan alan    0 2009-09-29 21:10 file2.txt
-rw-r--r--  1 alan alan  385 2009-09-29 21:46 yadda.txt
alan@alan-ubuntu-desktop:~/scrap$ cat blah.txt
ls: cannot access -: No such file or directory
ls: cannot access garbage: No such file or directory
alan@alan-ubuntu-desktop:~/scrap$

Now that we’re getting errors, everything is working as expected, our blah.txt file has the same output as the tee command printed to the screen, which in turn is the standard error output of the ls command. By the way, notice the little minus after the 3 when we do the last redirect, this is not just cosmetic. It allows us to close the file descriptor 3 now that we no longer need it.

It is all reasonably simple, but surprisingly slippery if you try to fully wrap your head around it.

There are lots of interesting and useful Bash tips like the ones above in the Bash Cookbook which is one of the books that I am going through at the moment. Check it out if you’re interested. Although I am sure I’ll be blogging about many more interesting bash tricks as I myself learn about them (or am reminded of their existence), so you can just wait for that :).

Image by temporalata

  • http://kingsly.net/kingsly/ Kingsly

    It has nothing to do with “bash”.
    tee is a standard *nix command that’ll work on any shell

    $ which tee
    /usr/bin/tee

    • http://www.skorks.com Alan Skorkin

      Well, you’re right tee will work on any *nix, however, there was more to the post than just noting the existence of tee. The rest was all about redirection and file descriptors, and these will not necessarily work the same with every shell on any *nix.

      I am working with bash, and I try to write the post in such a way that I know everything I say will work for sure, therefore I specify bash. Of course if the stuff in a post happens to work in a different shell or on a different OS that’s just a bonus :).

  • http://www.zulius.com Tim White

    Nice technique for separating stdout and stderr. That may come in handy with a bash function library I wrote for displaying nicely formatted output messages:

    http://www.zulius.com/freebies/bash-beauty-output-for-bash-scripts/

  • http://passingcuriosity.com Thomas Sutton

    Nice article! I’m subscribing to your RSS feed.

    Do note though that redirecting file descriptors as you’ve described is specified in section 2.7 of volume 3 of the POSIX:2008/IEEE Std 1003.1-2008 standard. Anything that calls itself /bin/sh — often bash on GNU/Linux systems — should definitely do this!

    Thanks.

    • http://www.skorks.com Alan Skorkin

      Thanks mate, another good reason for me to get my act together and start posting again :).

  • Myles

    Devil/details alert. tee is a GNU-coreutils program, not a feature of bash (also part of the GNU Project, but not specifically coreutils). So the upshot is that tee will work equivalently with all shells.

    • http://www.skorks.com Alan Skorkin

      Nothing wrong with details :)

  • http://www.jeremyjohnstone.com Jeremy Johnstone

    How come you don’t just use:

    2>&1 >/dev/null

    instead of pocketing stdout into a new descriptor (aka 3 above)?

    • http://www.skorks.com Alan Skorkin

      You’re right, good point and a great tip, doing what you suggest works just as well, thanks for sharing.

  • ten

    If you still want to see stdout on screen as well, use:

    command ls -l file notfile 3>&2 2>&1 1>&3 3>&- | tee blah.txt

    See also:

    http://codesnippets.joyent.com/posts/show/8769

  • Gopinath

    How do i transfer the temporary redirecting variable 3 to a another variable,and how does the “&-” symbol helps me .

    Regds
    Gopinath

  • Kekkonen

    Thanks, helped me alot! :)

  • Felipe Alvarez

    How can I make all output from a script print to the screen, and save to a file, at the same time, without having to type ” | tee filename” one hundred times throughout the script?