Dropping the file handle discussion - I'll read and carefully consider what you had to say on the subject. I do recognize this as an important problem: as I think about these things that I would want to put in a shell, not all of them necessarily fit, and in the end not all of them are worth the trouble.

For now I want to continue the discussion with the assumption that features we talk about would be implemented without too many fundamental changes to how the shell works.

Well, yeah, any program can create a PTY - but if you want to actually display multiple TTY-using programs simultaneously, you need something like screen or xterm - a virtual terminal. I could write my own virtual terminal like "screen" but for a first pass it'd probably be more productive to bend "screen" to my needs.

True... The main use cases I'm thinking about here would be cases where the program expects to display to a TTY. For the jobs to get input from the TTY - well, screen or xterm could handle that but if your loop is full of interactive prompts then maybe you need to redesign the code that's inside that loop, as you say. But if someone wanted to do that anyway, it could nevertheless be done via the screen mechanism I described...

Another way to look at this would be to say it's part of an introduction of proper threading to the shell - meaning that multiple loop iterations running concurrently can modify the variable (and the shell would provide enough synchronization to ensure that the environment isn't corrupted, structurally speaking) - and if better synchronization is needed, then the user has to write critical regions into their shell code.

Introducing the concept of threads may complicate other areas of the shell, however: for instance at present if you pipe a bunch of different shell commands together, all but one of those is going to be run in a subshell with no ability to change the environment of the main shell process. Introducing threading would raise the question of whether all those built-in jobs (and even the parenthesis syntax in general) should now be threads. This would be an incompatible change (though behavior of which part of a pipeline is executed in the main shell process varies between shells already) but it might also make sense.

My installed version of wget always writes out to /dev/tty (and using some kind of control characters to update the progress bar in-place) when /dev/tty is available.

When there's no TTY (running inside emacs, in this case) the progress bar is redrawn every 50KB or so - on a new line. If two wgets (with no /dev/tty) are run concurrently with shared stdout, the display winds up corrupted as the two processes write to stdout simultaneously: (not necessarily a full line at a time: a dot is written for each kilobyte received)

Code:

   700K .......... ......... 10%  513K 14s
   750K ................. .......... .......... .... .......... .......... .......... .......... 11%  559K 14s
   800K ............. 10%  434K 15s
   750K .......... ............ ........ .......... ......... .......... .......... ............. .......... 11%  488K 15s
   800K .......... ........ 12%  385K 14s
   850K .......... .......... ............. .......... ... ............... ........... .......... 12%  637K 13s
   900K ............ 12%  473K 14s

If you were running those two programs concurrently while you're at the console, watching, that's not a particularly useful way to display them. You don't really need to see all those lines of history, you already know that before you reached 12% you were at 11%, and before that 10%, and so on. (For that matter, that kind of information isn't useful for a log, either... That kind of output is mostly just useful if you want to read it into another program that implements another progress display method.) Really you just want to see the filenames of the files currently being fetched and the progress on each one. wget already has a good mechanism for displaying its own current status: using "screen" or similar makes it easy to take advantage of this.

Still, regardless of whether the output is readable before you interleave it, it's not going to be readable after you interleave it unless you make the right kind of choice about how the values should be interleaved. (Or, in the case of programs that display to the TTY, some kind of TTY-sharing mechanism.)

A related problem that I didn't address is stderr. Synchronizing on newlines would probably be adequate for most cases, but it'd probably make sense to provide all the same multiplexing/redirection options that stdout gets, in case somebody needed it. (For instance, if you were running gcc in the loop, probably you'd want to synchronize per loop step rather than letting stdout and stderr get interleaved per-line or "whenever"...)