redirect time command output to file (cygwin bash)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting redirect time command output to file (cygwin bash)
# 1  
Old 07-13-2012
redirect time command output to file (cygwin bash)

I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file. I have set it up like,
Code:
echo "Runtimes for servlet 4, 100K structures" > test_times.txt
echo "" >> test_times.txt

echo "runs where N=10" >> test_times.txt
echo "" >> test_times.txt

echo "run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10" >> test_times.txt
   time -a test_times.txt \
   ./myProgram.exe -a arguments -i inputFile -o outputFile
echo "" >> test_times.txt

I get a bash error for -a: command not found.

Am I doing something wrong here??? I thought that time -a filename would append to the file.

LMHmedchem
# 2  
Old 07-13-2012
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.

Second, you don't need to open the same file 300 times to put 300 things in it. Try this:

Code:
# The round brackets put it in a subshell.  You can redirect the output of the subshell instead
# of each individual command inside it.
(
        # You can feed one here-document into cat instead of running echo 9 times.
        # Note that the ending EOF is at the beginning of the line, not indented.  This is
        # important!  Even indenting that will prevent the here-document from ending properly.
        # Also note that variables will work inside a here-document, just like they would inside
        # double-quotes.
        cat <<EOF
Runtimes for servlet 4, 100K structures

Runs where N=10

run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF

        time ./myProgram.exe -a arguments -i inputFile -o outputFile

        echo

) > test_times.txt

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 07-13-2012
Quote:
Originally Posted by Corona688
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.

Second, you don't need to open the same file 300 times to put 300 things in it. Try this:

Code:
# The round brackets put it in a subshell.  You can redirect the output of the subshell instead
# of each individual command inside it.
(
        # You can feed one here-document into cat instead of running echo 9 times.
        # Note that the ending EOF is at the beginning of the line, not indented.  This is
        # important!  Even indenting that will prevent the here-document from ending properly.
        # Also note that variables will work inside a here-document, just like they would inside
        # double-quotes.
        cat <<EOF
Runtimes for servlet 4, 100K structures

Runs where N=10

run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF

        time ./myProgram.exe -a arguments -i inputFile -o outputFile

        echo

) > test_times.txt

Thanks for the help, I always learn something new here.

I added a few things to the code you posted,
Code:
SIZE=100K
CPDS=10

(
   cat <<EOF
Runtimes for servlet 4, $SIZE structures

Runs where N=$CPDS

run1 test_ActiMol_$SIZE.sdf -n 4 -t 1000 -l 100 -d t -N $CPDS
EOF

   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS -i  'test_ActiMol_'$SIZE'.sdf' -o  'test_ActiMol_100K_'$CPDS'N_2.s'

   echo

) > test_times.txt

and this runs without error, but the the time data is still to cout and does not end up in the file. Everything else ends up in test_times.txt as expected.

I will probably end up with a double loop on SIZE and CPDS, but I need to get the single instance working first. I tried adding the same -o filename after the time command, but I still get the -o: command not found error.

LMHmedchem
# 4  
Old 07-13-2012
time output is printed on the standard error, which hasn't been redirected.

Regards,
Alister
These 3 Users Gave Thanks to alister For This Post:
# 5  
Old 07-13-2012
This is the ugly triple loop that I set up to run this long batch,
Code:
#!/usr/bin/bash

echo "testing output" > test_times.txt
echo "" >> test_times.txt

# list directory names here
SIZE=( 100K 500K 1M )

for SIZE in ${SIZE[@]}
do

   echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
   echo "" >> test_times.txt

   CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 )

   for CPDS in ${CPDS[@]}
   do

      echo "Runs where N="$CPDS >> test_times.txt 

      REP=( 1 2 3 4 5 )

      for REP in ${REP[@]}
      do

         echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt 
         time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
         -i  'test_ActiMol_'$SIZE'.sdf' \
         -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s'  >> test_times.txt
         
         echo "   TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "   SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "   NUMBER PROCESSED=" $(wc -l 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt

      done

      echo " " >> test_times.txt 

   done

      echo " " >> test_times.txt
      echo " " >> test_times.txt 

done

This works in every respect other than printing the output of time to the file. I am very unclear as to how to utilize the here-document in this context.

This is giving me the output I need,
Code:
Runtimes for servlet 4, 100K structures

Runs where N=10
run 1 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_1.s
run 2 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_2.s
run 3 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_3.s
run 4 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_4.s
run 5 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
   TIMEOUTS= 0
   SEGFAULTS= 0
   NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_5.s

minus the times.

LMHmedchem

---------- Post updated at 08:44 PM ---------- Previous update was at 08:27 PM ----------

Quote:
Originally Posted by alister
time output is printed on the standard error, which hasn't been redirected.

Regards,
Alister
I have tried the following to get stderr appended to my output file,
Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt

the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.

There always seems to be a hangup somewhere.

LMHmedchem

---------- Post updated at 09:13 PM ---------- Previous update was at 08:44 PM ----------

Apparently, but using the following command,
Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >> test_times.txt 2>&1

I am redirecting the stderr and stdout of servert.exe to test_times.txt and not the stderr/stdout of time. This was fixed by grouping the time and servert.exe commands,
Code:
   ( time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
     -i  'test_ActiMol_'$SIZE'.sdf' \
     -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

with this revision, the output is as expected,
Code:
Runtimes for servlet 4, 100K structures

Runs where N=10
run 1 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

real	0m43.656s
user	0m0.015s
sys	0m0.015s
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_1.s

run 2 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

real	0m43.203s
user	0m0.031s
sys	0m0.015s
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_2.s

run 3 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10

now I think I need to try to incorporate the here-document suggested by Corona688 to minimize the number of echo commands.

LMHmedchem
# 6  
Old 07-14-2012
There were a few things regarding your redirection woes that piqued my curiosity. I believe I now understand why you've had so much trouble with it.

The syntax for a bash pipeline is as follows (from the bash manual):
Code:
The format for a pipeline is

     [time [-p]] [!] command1 [ [| or |&] command2 ...]

The reserved word time causes timing statistics to be printed for the pipeline once it finishes.

The key here is that time is a shell keyword, not a built-in and not an external command.


Quote:
Originally Posted by LMHmedchem
I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file.
Code:
<...snip...>
   time -a test_times.txt \
   ./myProgram.exe -a arguments -i inputFile -o outputFile

I get a bash error for -a: command not found.
Note that the error message says that -a is a command that cannot be found. It does not say that it's a bad option. Your script is using bash's time keyword, which does not support a -a or -o option, so that as far as bash is concerned, -a is the name of the command whose execution should be timed.

Quote:
Originally Posted by LMHmedchem
I have tried the following to get stderr appended to my output file,
Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt

Code:
   time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
   -i  'test_ActiMol_'$SIZE'.sdf' \
   -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt

the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.

There always seems to be a hangup somewhere.
Indeed. With a command, the first two redirection attempts would have worked. However, since time is a keyword, there is nothing to redirect. The time keyword simply instructs bash to output timing info to its own standard error. If you want to redirect the timing information, you have to modify the current shell environment with exec prior to invoking the timed pipeline...

Quote:
Originally Posted by LMHmedchem
This was fixed by grouping the time and servert.exe commands,
Code:
   ( time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
     -i  'test_ActiMol_'$SIZE'.sdf' \
     -o  'test_ActiMol_100K_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

with this revision, the output is as expected,
[code]
... or you can run it in a subshell as you did there.

If you want to run the time command that supports the options you were initially attempting to use, you'll need to bypass the keyword using command time ./servert.exe ... or tell bash where to find the time binary, e.g., /usr/bin/time ./servert.exe ....

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 7  
Old 07-14-2012
You are right that I mistakenly thought I was running the gnu time program when I was really using the bash time argument. Am I right in guessing that this reports the time it takes the bash command to execute and not the time it takes the program to run, or is there a difference?

GNU time gives more detailed output, such a page faults, etc. I thought I tried /usr/bin/time at some point, but that didn't work. I will have to give that a try and see if I can get it working. Which says that it's there. One of the issues I as thinking of is that the log file already exists, so I would want -a, but I didn't know if I could do -a without first having used -o to create the file.

This is the current version of the script,
Code:
#!/usr/bin/bash

# t value
TIMEOUT=1000
# n value
NCLIENTS=4

echo "testing output" > test_times.txt
echo "" >> test_times.txt

# list input file sizes here (number of compounds)
SIZE=( 1K 10K 100K 500K 1M )

for SIZE in ${SIZE[@]}
do
   echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
   echo "" >> test_times.txt

#  list values for N here
   CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 75000 100000 )

   for CPDS in ${CPDS[@]}
   do
      echo "Runs where N="$CPDS >> test_times.txt 

#     list number of repetitions for each run
      REP=( 1 2 3 4 5 )

      for REP in ${REP[@]}
      do
         echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt 
         ( time ./servert.exe -n $NCLIENTS -t $TIMEOUT -l 100 -d t -N $CPDS \
           -i  'test_ActiMol_'$SIZE'.sdf' \
           -o  'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1

         echo "TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "NUMBER PROCESSED=" $(wc -l 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
         echo "" >> test_times.txt

#        print something to the terminal to monitor progress
         echo 'completed run' $REP 'test_ActiMol_'$SIZE'.sdf -n' $NCLIENTS '-t' $TIMEOUT '-l 100 -d t -N' $CPDS

      done
      echo "" >> test_times.txt 

   done
      echo "" >> test_times.txt
      echo "" >> test_times.txt 

done

unix2dos test_times.txt

It seems as if some of the multiple echo statements could be reduced using the here-document that Corona688 suggested. I will look into that. It also seems that I could echo a couple of newlines at once instead of doing it several times to insert several blank lines.

LMHmedchem
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to Redirect the output of telnet command on a terminal to a file ?

(/home/user1)-> more script.sh #!/bin/ksh ( echo open devicename sleep 3; echo user; sleep 2; echo password; sleep 2; echo "/info/dump"; ---------> This needs to redirect to a file .Can be number of pages sleep 2; echo "exit" ) | telnet Please use code tags next time for... (2 Replies)
Discussion started by: necro98
2 Replies

2. Shell Programming and Scripting

Strange suppression of output with bash and cygwin

Hi, although I am not expert in bash, so please forgive me if this is silly, I think that this is strange: I have this command: find . -type f -print0 |xargs -0 grep -i -e 'some rexp' and it works fine. But when I create a bash script (on cygwin) to run this command, there is no output !!!... (3 Replies)
Discussion started by: Niki999
3 Replies

3. Shell Programming and Scripting

How to redirect the output of a cvs command to a file as well as the console.

Hi can anyone tell me how to redirect the ouput of a cvs command to a file as well as the console? i tried using cvs add <filename> | tee logFile cvs add <filename> 2>logFile 2>&1 All i could get is only on console or on file. Please help Thanks (2 Replies)
Discussion started by: ankitag2010
2 Replies

4. Shell Programming and Scripting

bash command in makefile (cygwin)

Hello, In my make file (make 3.81), I use a combination of shell commands to automatically create the name of my build directory. OS := $(shell uname -s) ARCH := $(shell uname -m) KERN := $(shell uname -r | cut -d. -f 1,2) BDIR := $(OS)_$(KERN).$(ARCH)When I boot into different OSs, I... (7 Replies)
Discussion started by: LMHmedchem
7 Replies

5. AIX

Unable to get the full content into a file when I redirect installp command output..

When i use the command to check the preview of the filesets to be installed using CLI # When using this commad 'm able to see all Preview view of the filesets to be installed installp -apgX -d "." all # When I redirected the same output to a file 'm able to see only half the details... (1 Reply)
Discussion started by: Sounddappan
1 Replies

6. UNIX and Linux Applications

How to redirect grep command output to same file

Hi Everyone, Can anyone please tell me, how can I redirect the grep command output to same file. I am trying with below command but my original file contains no data after executing the command. $grep pattern file1 > file1 Kind Regards, Eswar (5 Replies)
Discussion started by: picheswa
5 Replies

7. Windows & DOS: Issues & Discussions

Output of ls command on Cygwin

what does the '+' sign stand for on the output of ls command on cygwin? -rw-------+ 1 milhan None 146783 Jun 19 12:10 schedule.pdf -rw-------+ 1 milhan None 320 Aug 15 17:14 current.txt -rw------- 1 milhan None 24576 Dec 28 2008 fdfd.txt (2 Replies)
Discussion started by: milhan
2 Replies

8. UNIX for Advanced & Expert Users

redirect to both file and std output at the same time

hello can some one please help me to redirect the output of a command to both std output and a file. this is little urgent. sridhar (2 Replies)
Discussion started by: send2sridhar
2 Replies

9. Shell Programming and Scripting

how to redirect the output of a grep command to a file inside a shell script

hi, i wat to get the output of a grep command in a file. but when i am trying out the same grep command in the unix prompt its working fine.. i am getting the output properly.. but when i am writing the same command inside my shell script , its just creating a new output file with no contents... (11 Replies)
Discussion started by: kripssmart
11 Replies

10. UNIX for Dummies Questions & Answers

Redirect time output

Hello All, I am trying to write a script that would capture the output of time command on a Sun machine. I have tried these commands $ time ls > out $ time ls 2> out I just get the ls output in the out file for the first one. Thanks for any help Chuck (2 Replies)
Discussion started by: cbeech
2 Replies
Login or Register to Ask a Question