Shell Programming and Scripting

I have to redirect STDERR messages both to screen and also capture the same in a file but STDOUT only to the same file.
I have searched in this formum for a solution, but something like

Code:

srcipt 3>&1 >&2 2>&3 3>&- | tee errs

doesn't work for me...

Has anyone an idea???

I never use this kind of redirection...

You might missing 1. not sure..
try this

Don't know if I understand your question correct, but this works I think:

Code:

script 2>&1 1> errs | tee errs

If no error, result is only in file "errs", if error result is both on screen and in file "errs".

Lets start with some general information to make the problem understandable:

A Unix process is like a (y-shaped) water hose: you fill something in (via <stdin>) and something comes out (via <stdout>) the one way and something else the other way (<stderr>). You can - to continue the analogy - put a bucket under each of the outlets, even the same bucket under several outlets, but they will still remain different outlets of data.

By default, when a process is born, its 3 default I/O-channels are directed to:

stdin: keyboard
stdout: display
stderr: display

As <stdout> and <stderr> are both pointing to display, why is it that

Code:

process_1 | process_2

picks up the output from <stdout> but not from <stderr>? The answer is that "|" is a special form of connector, not just another bucket like ">". "|" means: redirect <stdout> of process_1 to <stdin> of process_2.

Now there is another redirection device, which is:

Code:

process_1 2>&1

This means: redirect output channel 2 (=<stderr>) to where output channel 1 (=<stdout>) points to right now. All redirections are read and carried out from left to right.

Having understood this let us try to solve your problem:

Code:

script 2>&1

will redirect <stderr> to <stdout>, so the next "|" will catch <stderr> output now too. Closer!

Code:

script 2>&1 | tee -a /some/file

This will pick up everything coming out of <stderr> and <stdout> of script and display it as well as appending it to "/some/file". Closer again, but <stdout> should not be displayed, so we will have to direct it away before the pipe picks up its input:

Code:

script 2>&1 1>/some/file | tee -a /some/file

This finally does what we want: output to <stdout> is put into "/some/file", output to <stderr> is being displayed before being appended to /some/file" too.

The only uncertainty left is that i am not sure if the exact sequence of the messages will be preserved, especially if there is high load and many messages. You will have to try that. I'll be tankful if you could post a follow-up telling us this.

I hope this helps.

bakunin

The y-shaped hose and bucket analogy paints a memorable image. Nicely done. The only thing I'd add, explicitly (it's implied in your explanation), is that pipe redirection occurs before other, left-to-right redirections.

Even without the pipe and redirections, you can't depend on your average script/executable to emit messages in the exact order that they're generated, since typically a mix of unbuffered (stderr) and buffered (stdout) streams are used.

Of much greater importance is that script 2>&1 1>/some/file | tee -a /some/file involves multiple processes writing to the same file without any form of communication. Writes from script will clobber writes from tee, or vice versa. If they at least shared a file descriptor, while one message could still split another in two, there would never be any overwriting.

Regards,
Alister

Thank you for this, as well as the additional info. I was not aware that <stderr> is unbuffered while <stdout> is not, so i have learned more here than i have explained. Nice gain. ;-))


It is probably a good idea to do like i have always done (out of luck - you have given me a reason after all) in my scripts: prepend standard and error output with respective prefixes:

Code:

#! /bin/ksh
...
print -u1 "MSG: $(timestamp) , starting the action"
action
if [ $? -gt 0 ] ; then
     print -u2 "ERR: $(timestamp) , action did not work"
else
     print -u1 "MSG: $(timestamp) , action worked out well"
fi
...

bakunin

Bakunin, it appears that I was editing my previous post just as you were responding. I apologize for that incovenience.

Your suggestion won't work at all. As I mentioned in my post (probably after you read it but before you posted), script and tee will clobber each other since they're using file descriptors backed by independent file descriptions each with their own offset.



That will not work for the same reason as bakunin's suggestion.



That approach cannot be made to work because, from the tee-side of the pipe, the distinction between stdout and stderr has been lost; they've been merged into one stream.

The following should work:

Code:

{
    exec 3>&1    # Save the current stdout (the cat pipe)
    script 2>&1 1>&3 | tee -a /dev/tty
} | cat > logfile

That may look like a useless use of cat (Hi, Corona ), but writing to a pipe guarantees atomicity for writes up to PIPE_BUF bytes inclusive.

If you rather not have that second pipe, you can simply delete highlighted text, leaving only the logfile redirection. However, if you do that, you may see interleaving of messages even for small writes.

Regards,
Alister