Have each subshell write stderr and stdout to its own logfile Post: 302916754

Sponsored Content

Top Forums Shell Programming and Scripting Have each subshell write stderr and stdout to its own logfile Post 302916754 by LMHmedchem on Thursday 11th of September 2014 06:54:49 PM

09-11-2014

Registered User

Quote:

Originally Posted by MadeInGermany

Code:

./01_batch_script.sh ... > $FOLD_GROUP.out 2>&1 &

I ended up using this version because it created fewer log files and I took this advice to use some additional data from my script to make up the log file name.

It would be nice to have some output to the terminal to follow progress, but it is easy enough to check the log files. The text editor that I use will reload the log file if there is a change.

This is what it looks like now,

Code:

# launch one batch-train script in background for each value in fold group list
for FOLD_GROUP in "${FOLD_GROUP_LIST[@]}"
do
   # continue training on $FOLD_GROUP folds
   ./01_batch_script.sh $CORES \
                        $BATCH_PROJ \
                        $BATCH_STOP_ON_SUBSET \
                        $BATCH_STOP_ON_STAT \
                        $SET \
                        $FOLD_GROUP \
                        $RND_SEED \
                        $DATASET_STRING \
                        $ERRTOL \
                        $BATCHES \
                        $START_MODE \
                        $START_LR \
                        $MAX_EPOCH_BATCH \
                        $OA_PRINT_PRECISION \
                        $BATCH_PROC > 'logfile_'$SET'_fg'$FOLD_GROUP'_'$START_MODE'-'$START_LR'.txt' 2>&1 &
   # to prevent terminal overrun
   sleep 2
done

# wait for all subshells to return before resuming
wait

LMHmedchem

LMHmedchem

View Public Profile for LMHmedchem

Find all posts by LMHmedchem

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Redirect stdout and stderr

How can I redirect and append stdout and stderr to a file when using cron? Here is my crontab file: */5 * * * * /dir/php /dir/process_fns.php >>& /dir/dump.txt Cron gives me an 'unexpected character found in line' when trying to add my crontab file. Regards, Zach Curtis POPULUS

2. Shell Programming and Scripting

Start subshell with different STDOUT

Hello Before I get to my question let me explain the situation. I am writing a ksh script to startup several instances of an application. That is done by executing another ksh script (let's call it startApp) with some arguments. Now, startApp writes some information to stdout that I don't want...

3. Shell Programming and Scripting

precedence of stderr and stdout

#!/usr/bin/perl open(STDOUT, ">>$Textfile") open(STDERR, ">>$Textfile") print "program running\n"; $final = join("+", $initial,$final) #5 close (STDOUT); close (STDERR);Hi all, above is my perl code. Notice i have captured the stdout and stderr to the same textfile. my code is expected to...

4. Shell Programming and Scripting

How to use tee with stdout and stderr?

I have been doing this: make xyz &> xyz.log &; tail -f xyz.log The problem with this is that you never can ge sure when "make xyz" is done. How can I pipe both stderr and stdout into tee so both stderr and stdout are copied both to the display and to the log file? Thanks, Siegfried

5. Shell Programming and Scripting

redirect STDOUT to a file in a subshell

Hi, I would like to avoid re-directing line by line to a file. What is the best way to re-direct STDOUT to a file in a subshell? Thanks in advance. Cheers Vj

6. Shell Programming and Scripting

redirect stdout and stderr to file wrong order problem with subshell

Hello I read a lot of post related to this topic, but nothing helped me. :mad: I'm running a ksh script with subshell what processing some ldap command. I need to check output for possible errors. #!/bin/ksh ... readinput < $QCHAT_INPUT |& while read -p line do echo $line ...

7. Shell Programming and Scripting

stderr/stdout

Can somebody explain to me why the diff output is not going to stderr? Yet when I issue a diff from the command line the return code is -ne 1. I am guessing diff always writes to stdout??? Is there away I can force the difff to write to stderr USING THE CURRENT template. If possible, I...

8. Shell Programming and Scripting

stdout, stderr redirection

Hi all, can someone help me with the next redirection? i want to redirect the stdout+stderr of a command to the same file (this i can do by prog &> file) but in addition i want to redirect only the stderr to a different file. how can i do this please? (in BASH) thanks.

9. Solaris

SMF in Solaris wont write to stderr/stdout ?

Hi, I got a process (c written) that we usually run this way : EmsChkQu >> /EMS/log/EmsChkQu.log 2>&1 When trying to use it as a service I defined it this way : <?xml version='1.0'?> <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'> <service_bundle...

10. Programming

Interactive Python 3.5+ sys.stdout.write() AND sys.stderr.write() bug?

(Apologies for any typos.) OSX 10.12.3 AND Windows 10. This is for the serious Python experts on at least 3.5.x and above... In script format sys.stdout.write() AND sys.stderr.write() seems to work correctly. Have I found a serious bug in the interactive sys.stdout.write() AND...

LEARN ABOUT HPUX

libbash

LIBBASH(7)							  libbash Manual							LIBBASH(7)

NAME

     libbash -- A bash shared libraries package.

DESCRIPTION

     libbash  is  a package that enables bash dynamic-like shared libraries. Actually its a tool for managing bash scripts whose functions you may
     want to load and use in scripts of your own.

     It contains a 'dynamic loader' for the shared libraries ( ldbash(1)), a configuration tool (ldbashconfig(8)), and some libraries.

     Using ldbash(1) you are able to load loadable bash libraries, such as getopts(1) and hashstash(1).  A bash shared library that can be  loaded
     using
     ldbash(1) must answer 4 requirments:

     1.     It must be installed in $LIBBASH_PREFIX/lib/bash (default is /usr/lib/bash).

     2.     It must contain a line that begins with '#EXPORT='.  That line will contain (after the '=') a list of functions that the library
	    exports.  I.e. all the function that will be usable after loading that library will be listed in that line.

     3.     It must contain a line that begins with '#REQUIRE='.  That line will contain (after the '=') a list of bash libraries that are
	    required for our library.  I.e. every bash library that is in use in our bash library must be listed there.

     4.     The library must be listed (For more information, see ldbashconfig(8)).

   Basic guidelines for writing library of your own:
     1.     Be aware, that your library will be actually sourced. So, basically, it should contain (i.e define) only functions.

     2.     Try to declare all variables intended for internal use as local.

     3.     Global variables and functions that are intended for internal use (i.e are not defined in '#EXPORT=') should begin with:
		  __<library_name>_
	    For example, internal function myfoosort of hashstash library should be named as
		  __hashstash_myfoosort
	    This helps to avoid conflicts in global name space when using libraries that come from different vendors.

     4.     See html manual for full version of this guide.

AUTHORS

     Hai Zaar <haizaar@haizaar.com>
     Gil Ran <ril@ran4.net>

SEE ALSO

     ldbash(1), ldbashconfig(8), getopts(1), hashstash(1) colors(1) messages(1) urlcoding(1) locks(1)

Linux								       Epoch								     Linux

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Redirect stdout and stderr

Discussion started by: zcurtis

2. Shell Programming and Scripting

Start subshell with different STDOUT

Discussion started by: EvilBoz

3. Shell Programming and Scripting

precedence of stderr and stdout

Discussion started by: new2ss

4. Shell Programming and Scripting

How to use tee with stdout and stderr?

Discussion started by: siegfried