Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
redirect time command output to file (cygwin bash) redirect time command output to file (cygwin bash)
Forums Search Forums Register Forum RulesMan PagesAlbums FAQ Members Calendar Search Today's Posts Mark Forums Read


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Closed Thread    
Page 1 of 2 1 2
 
Thread Tools Search this Thread Display Modes
    #1  
Old 07-13-2012
Registered User
  
Join Date: Mar 2010
Location: Bsoton
Posts: 157
Thanks: 41
Thanked 4 Times in 4 Posts
redirect time command output to file (cygwin bash)
I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file. I have set it up like,

Code:
echo "Runtimes for servlet 4, 100K structures" > test_times.txt
echo "" >> test_times.txt

echo "runs where N=10" >> test_times.txt
echo "" >> test_times.txt

echo "run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10" >> test_times.txt
   time -a test_times.txt \
   ./myProgram.exe -a arguments -i inputFile -o outputFile
echo "" >> test_times.txt

I get a bash error for -a: command not found.

Am I doing something wrong here??? I thought that time -a filename would append to the file.

LMHmedchem
Sponsored Links
    #2  
Old 07-13-2012
Mead Rotor
  
Join Date: Aug 2005
Location: Saskatchewan
Posts: 16,371
Thanks: 490
Thanked 2,534 Times in 2,417 Posts
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.

Second, you don't need to open the same file 300 times to put 300 things in it. Try this:


Code:
# The round brackets put it in a subshell.  You can redirect the output of the subshell instead
# of each individual command inside it.
(
        # You can feed one here-document into cat instead of running echo 9 times.
        # Note that the ending EOF is at the beginning of the line, not indented.  This is
        # important!  Even indenting that will prevent the here-document from ending properly.
        # Also note that variables will work inside a here-document, just like they would inside
        # double-quotes.
        cat <<EOF
Runtimes for servlet 4, 100K structures

Runs where N=10

run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF

        time ./myProgram.exe -a arguments -i inputFile -o outputFile

        echo

) > test_times.txt
The Following User Says Thank You to Corona688 For This Useful Post:
LMHmedchem (07-13-2012)
Sponsored Links
    #3  
Old 07-13-2012
Registered User
  
Join Date: Mar 2010
Location: Bsoton
Posts: 157
Thanks: 41
Thanked 4 Times in 4 Posts
Quote:
Originally Posted by Corona688 View Post
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.

Second, you don't need to open the same file 300 times to put 300 things in it. Try this:


Code:
# The round brackets put it in a subshell.  You can redirect the output of the subshell instead
# of each individual command inside it.
(
        # You can feed one here-document into cat instead of running echo 9 times.
        # Note that the ending EOF is at the beginning of the line, not indented.  This is
        # important!  Even indenting that will prevent the here-document from ending properly.
        # Also note that variables will work inside a here-document, just like they would inside
        # double-quotes.
        cat <<EOF
Runtimes for servlet 4, 100K structures

Runs where N=10

run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF

        time ./myProgram.exe -a arguments -i inputFile -o outputFile

        echo

) > test_times.txt
Thanks for the help, I always learn something new here.

I added a few things to the code you posted,

Code:
SIZE=100K
CPDS=10

(
   cat <<EOF
Runtimes for servlet 4, $SIZE structures

Runs where N=$CPDS

run1 test_ActiMol_$SIZE.sdf -n 4 -t 1000 -l 100 -d t -N $CPDS
EOF

   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS -i  'test_ActiMol_'$SIZE'.sdf' -o  'test_ActiMol_100K_'$CPDS'N_2.s'

   echo

) > test_times.txt

and this runs without error, but the the time data is still to cout and does not end up in the file. Everything else ends up in test_times.txt as expected.

I will probably end up with a double loop on SIZE and CPDS, but I need to get the single instance working first. I tried adding the same -o filename after the time command, but I still get the -o: command not found error.

LMHmedchem
    #4  
Old 07-13-2012
alister alister is offline Forum Advisor  
Registered User
  
Join Date: Dec 2009
Posts: 2,593
Thanks: 122
Thanked 716 Times in 599 Posts
time output is printed on the standard error, which hasn't been redirected.

Regards,
Alister
The Following 3 Users Say Thank You to alister For This Useful Post:
Corona688 (07-13-2012), LMHmedchem (07-13-2012), methyl (07-13-2012)
Sponsored Links
    #5  
Old 07-13-2012
Registered User
  
Join Date: Mar 2010
Location: Bsoton
Posts: 157
Thanks: 41
Thanked 4 Times in 4 Posts
This is the ugly triple loop that I set up to run this long batch,

Code:
#!/usr/bin/bash

echo "testing output" > test_times.txt
echo "" >> test_times.txt

# list directory names here
SIZE=( 100K 500K 1M )

for SIZE in ${SIZE[@]}
do

   echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
   echo "" >> test_times.txt

   CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 )

   for CPDS in ${CPDS[@]}
   do

      echo "Runs where N="$CPDS >> test_times.txt 

      REP=( 1 2 3 4 5 )

      for REP in ${REP[@]}
      do

         echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt 
         time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
         -i  'test_ActiMol_'$SIZE'.sdf' \
         -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s'  >> test_times.txt
         
         echo "   TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "   SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "   NUMBER PROCESSED=" $(wc -l 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt

      done

      echo " " >> test_times.txt 

   done

      echo " " >> test_times.txt
      echo " " >> test_times.txt 

done

This works in every respect other than printing the output of time to the file. I am very unclear as to how to utilize the here-document in this context.

This is giving me the output I need,

Code:
Runtimes for servlet 4, 100K structures

Runs where N=10
run 1 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_1.s
run 2 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_2.s
run 3 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_3.s
run 4 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_4.s
run 5 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_5.s

minus the times.

LMHmedchem

---------- Post updated at 08:44 PM ---------- Previous update was at 08:27 PM ----------

Quote:
Originally Posted by alister View Post
time output is printed on the standard error, which hasn't been redirected.

Regards,
Alister
I have tried the following to get stderr appended to my output file,

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1


Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt


Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt

the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.

There always seems to be a hangup somewhere.

LMHmedchem

---------- Post updated at 09:13 PM ---------- Previous update was at 08:44 PM ----------

Apparently, but using the following command,

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >> test_times.txt 2>&1

I am redirecting the stderr and stdout of servert.exe to test_times.txt and not the stderr/stdout of time. This was fixed by grouping the time and servert.exe commands,

Code:
   ( time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
     -i  'test_ActiMol_'$SIZE'.sdf' \
     -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

with this revision, the output is as expected,

Code:
Runtimes for servlet 4, 100K structures

Runs where N=10
run 1 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

real	0m43.656s
user	0m0.015s
sys	0m0.015s
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_1.s

run 2 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

real	0m43.203s
user	0m0.031s
sys	0m0.015s
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_2.s

run 3 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

now I think I need to try to incorporate the here-document suggested by Corona688 to minimize the number of echo commands.

LMHmedchem
Sponsored Links
    #6  
Old 07-14-2012
alister alister is offline Forum Advisor  
Registered User
  
Join Date: Dec 2009
Posts: 2,593
Thanks: 122
Thanked 716 Times in 599 Posts
There were a few things regarding your redirection woes that piqued my curiosity. I believe I now understand why you've had so much trouble with it.

The syntax for a bash pipeline is as follows (from the bash manual):

Code:
The format for a pipeline is

     [time [-p]] [!] command1 [ [| or |&] command2 ...]

The reserved word time causes timing statistics to be printed for the pipeline once it finishes.

The key here is that time is a shell keyword, not a built-in and not an external command.


Quote:
Originally Posted by LMHmedchem View Post
I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file.

Code:
<...snip...>
   time -a test_times.txt \
   ./myProgram.exe -a arguments -i inputFile -o outputFile

I get a bash error for -a: command not found.
Note that the error message says that -a is a command that cannot be found. It does not say that it's a bad option. Your script is using bash's time keyword, which does not support a -a or -o option, so that as far as bash is concerned, -a is the name of the command whose execution should be timed.

Quote:
Originally Posted by LMHmedchem View Post
I have tried the following to get stderr appended to my output file,

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1


Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt


Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt

the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.

There always seems to be a hangup somewhere.
Indeed. With a command, the first two redirection attempts would have worked. However, since time is a keyword, there is nothing to redirect. The time keyword simply instructs bash to output timing info to its own standard error. If you want to redirect the timing information, you have to modify the current shell environment with exec prior to invoking the timed pipeline...

Quote:
Originally Posted by LMHmedchem View Post
This was fixed by grouping the time and servert.exe commands,

Code:
   ( time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
     -i  'test_ActiMol_'$SIZE'.sdf' \
     -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

with this revision, the output is as expected,
[code]
... or you can run it in a subshell as you did there.

If you want to run the time command that supports the options you were initially attempting to use, you'll need to bypass the keyword using command time ./servert.exe ... or tell bash where to find the time binary, e.g., /usr/bin/time ./servert.exe ... .

Regards,
Alister
The Following User Says Thank You to alister For This Useful Post:
LMHmedchem (07-14-2012)
Sponsored Links
    #7  
Old 07-14-2012
Registered User
  
Join Date: Mar 2010
Location: Bsoton
Posts: 157
Thanks: 41
Thanked 4 Times in 4 Posts
You are right that I mistakenly thought I was running the gnu time program when I was really using the bash time argument. Am I right in guessing that this reports the time it takes the bash command to execute and not the time it takes the program to run, or is there a difference?

GNU time gives more detailed output, such a page faults, etc. I thought I tried /usr/bin/time at some point, but that didn't work. I will have to give that a try and see if I can get it working. Which says that it's there. One of the issues I as thinking of is that the log file already exists, so I would want -a, but I didn't know if I could do -a without first having used -o to create the file.

This is the current version of the script,

Code:
#!/usr/bin/bash

# t value
TIMEOUT=1000
# n value
NCLIENTS=4

echo "testing output" > test_times.txt
echo "" >> test_times.txt

# list input file sizes here (number of compounds)
SIZE=( 1K 10K 100K 500K 1M )

for SIZE in ${SIZE[@]}
do
   echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
   echo "" >> test_times.txt

#  list values for N here
   CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 75000 100000 )

   for CPDS in ${CPDS[@]}
   do
      echo "Runs where N="$CPDS >> test_times.txt 

#     list number of repetitions for each run
      REP=( 1 2 3 4 5 )

      for REP in ${REP[@]}
      do
         echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt 
         ( time ./servert.exe -n $NCLIENTS -t $TIMEOUT -l 100 -d t -N $CPDS \
           -i  'test_ActiMol_'$SIZE'.sdf' \
           -o  'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

         echo "TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "NUMBER PROCESSED=" $(wc -l 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "" >> test_times.txt

#        print something to the terminal to monitor progress
         echo 'completed run' $REP 'test_ActiMol_'$SIZE'.sdf -n' $NCLIENTS '-t' $TIMEOUT '-l 100 -d t -N' $CPDS

      done
      echo "" >> test_times.txt 

   done
      echo "" >> test_times.txt
      echo "" >> test_times.txt 

done

unix2dos test_times.txt

It seems as if some of the multiple echo statements could be reduced using the here-document that Corona688 suggested. I will look into that. It also seems that I could echo a couple of newlines at once instead of doing it several times to insert several blank lines.

LMHmedchem
Sponsored Links
Closed Thread
Page 1 of 2 1 2

« Passing gunzipped filename to a variable | awk, max value, array, row »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes
Linear Mode Linear Mode

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
How to redirect the output of a cvs command to a file as well as the console. ankitag2010 Shell Programming and Scripting 2 08-25-2011 04:27 AM
Unable to get the full content into a file when I redirect installp command output.. Sounddappan AIX 1 06-07-2010 04:35 AM
How to redirect grep command output to same file picheswa UNIX and Linux Applications 5 05-23-2010 03:11 AM
redirect to both file and std output at the same time send2sridhar UNIX for Advanced & Expert Users 2 09-23-2008 09:30 PM
how to redirect the output of a grep command to a file inside a shell script kripssmart Shell Programming and Scripting 11 06-19-2008 07:28 AM



All times are GMT -4. The time now is 11:33 PM.

The UNIX and Linux Forums - Where There is a Shell There is a Way RSS Feeds - Contact Us - The UNIX and Linux Forums - Where There is a Shell There is a Way - Archive - Botnik - Top
SEO by vBSEO
The UNIX and Linux Forums Content Copyright ©1993-2013. All Rights Reserved.
Forum Operations by The UNIX and Linux Forums
Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.