Sponsored Content
Full Discussion: Weird redirection behaviour
Top Forums Shell Programming and Scripting Weird redirection behaviour Post 302694337 by Lem on Thursday 30th of August 2012 11:47:40 AM
Old 08-30-2012
I've been told that also
Code:
$ ls imafile idontexist 2> >(tee err >outanderr) | tee -a outanderr >/dev/null

has created problems to someone, so it isn't safe.

Thinking it were safe confused me.

Of course when there is only one command, as in
Code:
command >file 2>&1

the writes are sequential, so there cannot be problems.

When there are many (sub)processes, created by pipes or command substitution or whatever else, all concurrently writing on the same file, I cannot see a clear way to be sure they won't overlap - using only the shell tools: /var/log/syslog works flawlessly, but there's a daemon in control, Which I think has its own importance. Smilie
Depending on the scheduler, on the number of CPUs and maybe on many other things, some solutions may seem to work here for a while, but don't work there.

Beside some awful workarounds with a very tiny field of application, like something that:
Code:
( ls imafile idontexist 2>&1 >&3 | { sleep 1; tee err >&3; } ) 3>outanderr

I can't think of any real solution. But since I know very little at all about these black boxes we call "computers", I wouldn't be surprised to be totally wrong.

Thanks to anyone reading all this awkward stuff.
--
Bye
 

9 More Discussions You Might Find Interesting

1. Programming

Can some 1 explain why this behaviour

#include <iostream> using namespace std; int main() { const int l_test = 999999999; int *l_ptr = (int*) &l_test; cout<<"Constant Addr:"<<&l_test<<" Value:"<<l_test<<endl; cout<<"Pointer Addr:"<<l_ptr<<" Value:"<<*l_ptr<<endl; *l_ptr = 888888888; // Manipulating... (2 Replies)
Discussion started by: helpmenow
2 Replies

2. UNIX for Advanced & Expert Users

Weird sudo behaviour

Hi gurus. I implemented sudo and have the following in my sudo config file *************** # User alias specification User_Alias VENDOR = user1 # User privilege specification VENDOR ALL = NOPASSWD: /bin/, /sbin/, /usr/local/bin/, \ !/bin/su,... (1 Reply)
Discussion started by: geomonap
1 Replies

3. Shell Programming and Scripting

Weird sed behaviour in script

I've written a small script to replace certain words in all the the files in a directory. #!/bin/sh #Get list of files to be edited file_list=`ls -p` for i in $file_list do echo "Processing $i" alteredi=`echo "$i" | sed -e 's/\//d/'` if then if then #actual altering (2 Replies)
Discussion started by: Peetrus
2 Replies

4. Shell Programming and Scripting

cp -R behaviour

i 've noticed the following difference between freebsd cp and gnu cp from the freebsd cp man page: -R ... If the source_file ends in a /, the contents of the directory are copied rather than the directory itself. ... on gnu cp from the man pagewhile on gnu cp manpage: ‘-r'... (2 Replies)
Discussion started by: aegis
2 Replies

5. Shell Programming and Scripting

Weird script behaviour !

Hello, I am getting an infinite loop from a script in Linux. Here is the last version of the script in question. As you can see I tried to define everything properly: #!/bin/ksh # Script to loop over a series of dates set -ex typeset -i start_date=20090701 typeset -i... (2 Replies)
Discussion started by: stavros
2 Replies

6. Shell Programming and Scripting

find: "weird" regex behaviour

I have these two files in current dir: oos.txt oos_(copy).txt I execute this find command:find . -regex './oos*.txt'And this outputs only the first file (oos.txt)! :confused: Only if I add another asterisk to the find find . -regex './oos*.*txt' do I also get the second file... (7 Replies)
Discussion started by: courteous
7 Replies

7. Ubuntu

Weird rm behaviour

I am little bit confused by the behaviour of rm in Ubuntu. It seems that as a regular user I can delete files owned by another user even when the permissions are set to 644. Here is an example: cjohnson@carbon:~/test$ sudo touch testfile cjohnson@carbon:~/test$ ls -al total 8 drwxr-xr-x... (2 Replies)
Discussion started by: ccj4467
2 Replies

8. Programming

different behaviour in fg and bg

fg = foreground bg = background I have a cobol program that I start with a very simple script. The script is not at fault as it has not changed and the program worked in fg and bg before. I have altered the logging in the program and moved my cursor declare to working storage. The program runs... (6 Replies)
Discussion started by: Bruble
6 Replies

9. Programming

Sort behaviour

I see strange results when sorting with -n options and I wander if somebody can explain it. Input file and two results: $ cat aa 14 -1 11 -1 0 -1 0 $ sort -u aa -1 0 (1 Reply)
Discussion started by: migurus
1 Replies
IPC::Run3(3)						User Contributed Perl Documentation					      IPC::Run3(3)

NAME
IPC::Run3 - run a subprocess with input/ouput redirection VERSION
version 0.046 SYNOPSIS
use IPC::Run3; # Exports run3() by default run3 @cmd, $in, $out, $err; DESCRIPTION
This module allows you to run a subprocess and redirect stdin, stdout, and/or stderr to files and perl data structures. It aims to satisfy 99% of the need for using "system", "qx", and "open3" with a simple, extremely Perlish API. Speed, simplicity, and portability are paramount. (That's speed of Perl code; which is often much slower than the kind of buffered I/O that this module uses to spool input to and output from the child command.) "run3($cmd, $stdin, $stdout, $stderr, \%options)" All parameters after $cmd are optional. The parameters $stdin, $stdout and $stderr indicate how the child's corresponding filehandle ("STDIN", "STDOUT" and "STDERR", resp.) will be redirected. Because the redirects come last, this allows "STDOUT" and "STDERR" to default to the parent's by just not specifying them -- a common use case. "run3" throws an exception if the wrapped "system" call returned -1 or anything went wrong with "run3"'s processing of filehandles. Otherwise it returns true. It leaves $? intact for inspection of exit and wait status. Note that a true return value from "run3" doesn't mean that the command had a successful exit code. Hence you should always check $?. See "%options" for an option to handle the case of "system" returning -1 yourself. $cmd Usually $cmd will be an ARRAY reference and the child is invoked via system @$cmd; But $cmd may also be a string in which case the child is invoked via system $cmd; (cf. "system" in perlfunc for the difference and the pitfalls of using the latter form). $stdin, $stdout, $stderr The parameters $stdin, $stdout and $stderr can take one of the following forms: "undef" (or not specified at all) The child inherits the corresponding filehandle from the parent. run3 @cmd, $stdin; # child writes to same STDOUT and STDERR as parent run3 @cmd, undef, $stdout, $stderr; # child reads from same STDIN as parent "undef" The child's filehandle is redirected from or to the local equivalent of "/dev/null" (as returned by "File::Spec->devnull()"). run3 @cmd, undef, $stdout, $stderr; # child reads from /dev/null a simple scalar The parameter is taken to be the name of a file to read from or write to. In the latter case, the file will be opened via open FH, ">", ... i.e. it is created if it doesn't exist and truncated otherwise. Note that the file is opened by the parent which will croak in case of failure. run3 @cmd, undef, "out.txt"; # child writes to file "out.txt" a filehandle (either a reference to a GLOB or an "IO::Handle") The filehandle is inherited by the child. open my $fh, ">", "out.txt"; print $fh "prologue "; ... run3 @cmd, undef, $fh; # child writes to $fh ... print $fh "epilogue "; close $fh; a SCALAR reference The referenced scalar is treated as a string to be read from or written to. In the latter case, the previous content of the string is overwritten. my $out; run3 @cmd, undef, $out; # child writes into string run3 @cmd, <<EOF; # child reads from string (can use "here" notation) Input to child EOF an ARRAY reference For $stdin, the elements of @$stdin are simply spooled to the child. For $stdout or $stderr, the child's corresponding file descriptor is read line by line (as determined by the current setting of $/) into @$stdout or @$stderr, resp. The previous content of the array is overwritten. my @lines; run3 @cmd, undef, @lines; # child writes into array a CODE reference For $stdin, &$stdin will be called repeatedly (with no arguments) and the return values are spooled to the child. &$stdin must signal the end of input by returning "undef". For $stdout or $stderr, the child's corresponding file descriptor is read line by line (as determined by the current setting of $/) and &$stdout or &$stderr, resp., is called with the contents of the line. Note that there's no end-of-file indication. my $i = 0; sub producer { return $i < 10 ? "line".$i++." " : undef; } run3 @cmd, &producer; # child reads 10 lines Note that this form of redirecting the child's I/O doesn't imply any form of concurrency between parent and child - run3()'s method of operation is the same no matter which form of redirection you specify. If the same value is passed for $stdout and $stderr, then the child will write both "STDOUT" and "STDERR" to the same filehandle. In general, this means that run3 @cmd, undef, "foo.txt", "foo.txt"; run3 @cmd, undef, $both, $both; will DWIM and pass a single file handle to the child for both "STDOUT" and "STDERR", collecting all into file "foo.txt" or $both. "\%options" The last parameter, "\%options", must be a hash reference if present. Currently the following keys are supported: "binmode_stdin", "binmode_stdout", "binmode_stderr" The value must a "layer" as described in "binmode" in perlfunc. If specified the corresponding parameter $stdin, $stdout or $stderr, resp., operates with the given layer. For backward compatibility, a true value that doesn't start with ":" (e.g. a number) is interpreted as ":raw". If the value is false or not specified, the default is ":crlf" on Windows and ":raw" otherwise. Don't expect that values other than the built-in layers ":raw", ":crlf", and (on newer Perls) ":bytes", ":utf8", ":encoding(...)" will work. "append_stdout", "append_stderr" If their value is true then the corresponding parameter $stdout or $stderr, resp., will append the child's output to the existing "contents" of the redirector. This only makes sense if the redirector is a simple scalar (the corresponding file is opened in append mode), a SCALAR reference (the output is appended to the previous contents of the string) or an ARRAY reference (the output is "push"ed onto the previous contents of the array). "return_if_system_error" If this is true "run3" does not throw an exception if "system" returns -1 (cf. "system" in perlfunc for possible failure scenarios.), but returns true instead. In this case $? has the value -1 and $! contains the errno of the failing "system" call. HOW IT WORKS
(1) For each redirector $stdin, $stdout, and $stderr, "run3()" furnishes a filehandle: o if the redirector already specifies a filehandle it just uses that o if the redirector specifies a filename, "run3()" opens the file in the appropriate mode o in all other cases, "run3()" opens a temporary file (using tempfile) (2) If "run3()" opened a temporary file for $stdin in step (1), it writes the data using the specified method (either from a string, an array or returned by a function) to the temporary file and rewinds it. (3) "run3()" saves the parent's "STDIN", "STDOUT" and "STDERR" by duplicating them to new filehandles. It duplicates the filehandles from step (1) to "STDIN", "STDOUT" and "STDERR", resp. (4) "run3()" runs the child by invoking system with $cmd as specified above. (5) "run3()" restores the parent's "STDIN", "STDOUT" and "STDERR" saved in step (3). (6) If "run3()" opened a temporary file for $stdout or $stderr in step (1), it rewinds it and reads back its contents using the specified method (either to a string, an array or by calling a function). (7) "run3()" closes all filehandles that it opened explicitly in step (1). Note that when using temporary files, "run3()" tries to amortize the overhead by reusing them (i.e. it keeps them open and rewinds and truncates them before the next operation). LIMITATIONS
Often uses intermediate files (determined by File::Temp, and thus by the File::Spec defaults and the TMPDIR env. variable) for speed, portability and simplicity. Use extreme caution when using "run3" in a threaded environment if concurrent calls of "run3" are possible. Most likely, I/O from different invocations will get mixed up. The reason is that in most thread implementations all threads in a process share the same STDIN/STDOUT/STDERR. Known failures are Perl ithreads on Linux and Win32. Note that "fork" on Win32 is emulated via Win32 threads and hence I/O mix up is possible between forked children here ("run3" is "fork safe" on Unix, though). DEBUGGING
To enable debugging use the IPCRUN3DEBUG environment variable to a non-zero integer value: $ IPCRUN3DEBUG=1 myapp PROFILING
To enable profiling, set IPCRUN3PROFILE to a number to enable emitting profile information to STDERR (1 to get timestamps, 2 to get a summary report at the END of the program, 3 to get mini reports after each run) or to a filename to emit raw data to a file for later analysis. COMPARISON
Here's how it stacks up to existing APIs: compared to "system()", "qx''", "open "...|"", "open "|..."" o better: redirects more than one file descriptor o better: returns TRUE on success, FALSE on failure o better: throws an error if problems occur in the parent process (or the pre-exec child) o better: allows a very perlish interface to Perl data structures and subroutines o better: allows 1 word invocations to avoid the shell easily: run3 ["foo"]; # does not invoke shell o worse: does not return the exit code, leaves it in $? compared to "open2()", "open3()" o better: no lengthy, error prone polling/select loop needed o better: hides OS dependencies o better: allows SCALAR, ARRAY, and CODE references to source and sink I/O o better: I/O parameter order is like "open3()" (not like "open2()"). o worse: does not allow interaction with the subprocess compared to IPC::Run::run() o better: smaller, lower overhead, simpler, more portable o better: no select() loop portability issues o better: does not fall prey to Perl closure leaks o worse: does not allow interaction with the subprocess (which IPC::Run::run() allows by redirecting subroutines) o worse: lacks many features of "IPC::Run::run()" (filters, pipes, redirects, pty support) COPYRIGHT
Copyright 2003, R. Barrie Slaymaker, Jr., All Rights Reserved LICENSE
You may use this module under the terms of the BSD, Artistic, or GPL licenses, any version. AUTHOR
Barrie Slaymaker <"barries@slaysys.com"> Ricardo SIGNES <"rjbs@cpan.org"> performed routine maintenance since 2010, thanks to help from the following ticket and/or patch submitters: Jody Belka, Roderich Schupp, David Morel, Jeff Lavallee, and anonymous others. perl v5.18.2 2013-06-11 IPC::Run3(3)
All times are GMT -4. The time now is 01:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy