Shell Programming and Scripting

I've seen valid uses of cat on a single file before, but this is the first valid use I've seen for no files. Interesting. This is, of course, assuming that these programs output lines atomically, but still.

I hadn't realized that files wouldn't necessarily be atomic, for that matter.

Uhm...
After a few tries:

Code:

$ { exec 3>&1; ls file none 2>&1 1>&3 | tee -a /dev/tty; } |cat >logfile
ls: impossibile accedere a none: File o directory non esistente

$ cat logfile
ls: impossibile accedere a nonefile
: File o directory non esistente

BTW: reading the thread, at first I've been a bit confused about what you all mean with "script": a script or the command /usr/bin/script? LOL.
--
Bye

Try ( ) instead of { } .

It's the same: "Each command in a pipeline is executed as a separate process (i.e., in a subshell)". So you have a subshell even with { } | ....

However:

Code:

$ ( exec 3>&1; ls file none 2>&1 1>&3 | tee -a /dev/tty; ) |cat >logfile; cat logfile
ls: impossibile accedere a none: File o directory non esistente
ls: impossibile accedere a nonefile
: File o directory non esistente

--
Bye

@alister:
So from now on we can say "alister told us so" when we are using useless use of cat
No seriously, in my suggestion I was thinking either STDOUT or STDERR, and that was wrong (was testing with a simple command not a script). Now when I'm testing with a script that has output I see that output to STDOUT is overwriting some characters in STDERR in the resulting "logfile".

But in your suggestion, I don't understand what is happening, see comments:

Code:

{
    # ok, this is saving STDOUT to &3(?)
    exec 3>&1    # Save the current stdout (the cat pipe)
    # here STDOUT is set to &3 which is STDOUT that was saved above???
    # Why is that necessary?
    script 2>&1 1>&3 | tee -a /dev/tty
} | cat > logfile

The blame for that behavior almost certainly lies with your ls implementation. Which ls are you using?

I can reproduce that behavior with GNU ls. I cannot reproduce it with busybox ls. A closer look using strace (I booted a Linux system just for this ) confirmed my suspicions: GNU ls isn't even trying to write a line at a time.

This is how GNU ls and Busybox ls attempt to write an error message for a nonexistent file named idont:

Code:

# GNU ls
write(2, "ls: ", 4)
write(2, "cannot access idont", 19)
# In Lem's example, interleaving occurred here
write(2, ": No such file or directory", 27)
write(2, "\n", 1)

# Busybox ls
write(2, "ls: idont: No such file or directory\n", 37)

There is no way to fix or workaround that, short of fixing GNU ls (or whatever code it depends on for generating its error messages). Perhaps other ls implementations (and other utilities for that matter) suffer from that problem, but there's nothing that can be done about this at the shell level.

With that many writes for a single, relatively short error message, there's a good chance that another process will be given the chance to write to the shared pipe. Each write is still atomic, it's just the message that's broken.

Regards,
Alister

---------- Post updated at 04:31 PM ---------- Previous update was at 04:26 PM ----------

You need to save stdout at the time that it's pointing to the cat-pipe because, later, when the shell builds the script|tee pipeline and redirect's script's stdout to the tee-pipe, if not for fd 3, there would be no way to refer to the cat-pipe (which script needs so that its stdout can bypass tee).

Regards,
Alister

Yep, alister, my ls is ls (GNU coreutils) 8.5.

Just for the sake of curiosity, I was wondering whether we could try an ugly workaround (I know it's at least ugly): if GNU ls splits too much its error messages, what about a little pause between ls and tee?

It seems to work. Here below the count of "nonefile"s gives us the number of
broken logs:

Code:

$ for i in {1..1000}; do { exec 3>&1; ls file none 2>&1 1>&3 | tee -a /dev/null; } |grep nonefile; done |wc -l
56

so 56 broken logs out of 1000 tries, versus:

Code:

for i in {1..100}; do { exec 3>&1; ls file none 2>&1 1>&3 | { sleep 1; tee -a /dev/null; }; } |grep nonefile; done |wc -l
0

0 broken out of 100 tries.


BTW: of course, if one doesn't care at all about having all the errors at the beginning or at the end of the log, a solution is obvious:

Code:

command 2>&1 >logfile | tee -a error; cat error >>logfile

or

Code:

command 2>&1 >output | tee -a logfile; cat output >>logfile

But... ça va sans dire.
--
Bye