redirect time command output to file (cygwin bash)
I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file. I have set it up like,
Code:
echo "Runtimes for servlet 4, 100K structures" > test_times.txt
echo "" >> test_times.txt
echo "runs where N=10" >> test_times.txt
echo "" >> test_times.txt
echo "run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10" >> test_times.txt
time -a test_times.txt \
./myProgram.exe -a arguments -i inputFile -o outputFile
echo "" >> test_times.txt
I get a bash error for -a: command not found.
Am I doing something wrong here??? I thought that time -a filename would append to the file.
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.
Second, you don't need to open the same file 300 times to put 300 things in it. Try this:
Code:
# The round brackets put it in a subshell. You can redirect the output of the subshell instead
# of each individual command inside it.
(
# You can feed one here-document into cat instead of running echo 9 times.
# Note that the ending EOF is at the beginning of the line, not indented. This is
# important! Even indenting that will prevent the here-document from ending properly.
# Also note that variables will work inside a here-document, just like they would inside
# double-quotes.
cat <<EOF
Runtimes for servlet 4, 100K structures
Runs where N=10
run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF
time ./myProgram.exe -a arguments -i inputFile -o outputFile
echo
) > test_times.txt
First off, don't give a filename after -a, since one isn't expected there. -a tells it to append, but you still need -o to tell it which file.
Second, you don't need to open the same file 300 times to put 300 things in it. Try this:
Code:
# The round brackets put it in a subshell. You can redirect the output of the subshell instead
# of each individual command inside it.
(
# You can feed one here-document into cat instead of running echo 9 times.
# Note that the ending EOF is at the beginning of the line, not indented. This is
# important! Even indenting that will prevent the here-document from ending properly.
# Also note that variables will work inside a here-document, just like they would inside
# double-quotes.
cat <<EOF
Runtimes for servlet 4, 100K structures
Runs where N=10
run1 test_intput.txt -n 4 -t 1000 -l 100 -d t -N 10
EOF
time ./myProgram.exe -a arguments -i inputFile -o outputFile
echo
) > test_times.txt
Thanks for the help, I always learn something new here.
I added a few things to the code you posted,
Code:
SIZE=100K
CPDS=10
(
cat <<EOF
Runtimes for servlet 4, $SIZE structures
Runs where N=$CPDS
run1 test_ActiMol_$SIZE.sdf -n 4 -t 1000 -l 100 -d t -N $CPDS
EOF
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS -i 'test_ActiMol_'$SIZE'.sdf' -o 'test_ActiMol_100K_'$CPDS'N_2.s'
echo
) > test_times.txt
and this runs without error, but the the time data is still to cout and does not end up in the file. Everything else ends up in test_times.txt as expected.
I will probably end up with a double loop on SIZE and CPDS, but I need to get the single instance working first. I tried adding the same -o filename after the time command, but I still get the -o: command not found error.
This is the ugly triple loop that I set up to run this long batch,
Code:
#!/usr/bin/bash
echo "testing output" > test_times.txt
echo "" >> test_times.txt
# list directory names here
SIZE=( 100K 500K 1M )
for SIZE in ${SIZE[@]}
do
echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
echo "" >> test_times.txt
CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 )
for CPDS in ${CPDS[@]}
do
echo "Runs where N="$CPDS >> test_times.txt
REP=( 1 2 3 4 5 )
for REP in ${REP[@]}
do
echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >> test_times.txt
echo " TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
echo " SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
echo " NUMBER PROCESSED=" $(wc -l 'test_ActiMol_100K_'$CPDS'N_'$REP'.s') >> test_times.txt
done
echo " " >> test_times.txt
done
echo " " >> test_times.txt
echo " " >> test_times.txt
done
This works in every respect other than printing the output of time to the file. I am very unclear as to how to utilize the here-document in this context.
This is giving me the output I need,
Code:
Runtimes for servlet 4, 100K structures
Runs where N=10
run 1 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_1.s
run 2 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_2.s
run 3 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_3.s
run 4 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_4.s
run 5 test_ActiMol_100K.sdf -n 4 -t 1000 -l 100 -d t -N 10
TIMEOUTS= 0
SEGFAULTS= 0
NUMBER PROCESSED= 100002 test_ActiMol_100K_10N_5.s
minus the times.
LMHmedchem
---------- Post updated at 08:44 PM ---------- Previous update was at 08:27 PM ----------
Quote:
Originally Posted by alister
time output is printed on the standard error, which hasn't been redirected.
Regards,
Alister
I have tried the following to get stderr appended to my output file,
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt
the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.
There always seems to be a hangup somewhere.
LMHmedchem
---------- Post updated at 09:13 PM ---------- Previous update was at 08:44 PM ----------
I am redirecting the stderr and stdout of servert.exe to test_times.txt and not the stderr/stdout of time. This was fixed by grouping the time and servert.exe commands,
There were a few things regarding your redirection woes that piqued my curiosity. I believe I now understand why you've had so much trouble with it.
The syntax for a bash pipeline is as follows (from the bash manual):
Code:
The format for a pipeline is
[time [-p]] [!] command1 [ [| or |&] command2 ...]
The reserved word time causes timing statistics to be printed for the pipeline once it finishes.
The key here is that time is a shell keyword, not a built-in and not an external command.
Quote:
Originally Posted by LMHmedchem
I have set up a bash script to run a long list of things that I need to time. I would like to redirect the output of time to a file.
Code:
<...snip...>
time -a test_times.txt \
./myProgram.exe -a arguments -i inputFile -o outputFile
I get a bash error for -a: command not found.
Note that the error message says that -a is a command that cannot be found. It does not say that it's a bad option. Your script is using bash's time keyword, which does not support a -a or -o option, so that as far as bash is concerned, -a is the name of the command whose execution should be timed.
Quote:
Originally Posted by LMHmedchem
I have tried the following to get stderr appended to my output file,
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' >>test_times.txt 2>&1
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' 2>>test_times.txt
Code:
time ./servert.exe -n 4 -t 1000 -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_100K_'$CPDS'N_'$REP'.s' &>>test_times.txt
the third one being a shot in the dark. None of these make any difference and I am still getting the time data printed to the terminal.
There always seems to be a hangup somewhere.
Indeed. With a command, the first two redirection attempts would have worked. However, since time is a keyword, there is nothing to redirect. The time keyword simply instructs bash to output timing info to its own standard error. If you want to redirect the timing information, you have to modify the current shell environment with exec prior to invoking the timed pipeline...
Quote:
Originally Posted by LMHmedchem
This was fixed by grouping the time and servert.exe commands,
with this revision, the output is as expected,
[code]
... or you can run it in a subshell as you did there.
If you want to run the time command that supports the options you were initially attempting to use, you'll need to bypass the keyword using command time ./servert.exe ... or tell bash where to find the time binary, e.g., /usr/bin/time ./servert.exe ....
You are right that I mistakenly thought I was running the gnu time program when I was really using the bash time argument. Am I right in guessing that this reports the time it takes the bash command to execute and not the time it takes the program to run, or is there a difference?
GNU time gives more detailed output, such a page faults, etc. I thought I tried /usr/bin/time at some point, but that didn't work. I will have to give that a try and see if I can get it working. Which says that it's there. One of the issues I as thinking of is that the log file already exists, so I would want -a, but I didn't know if I could do -a without first having used -o to create the file.
This is the current version of the script,
Code:
#!/usr/bin/bash
# t value
TIMEOUT=1000
# n value
NCLIENTS=4
echo "testing output" > test_times.txt
echo "" >> test_times.txt
# list input file sizes here (number of compounds)
SIZE=( 1K 10K 100K 500K 1M )
for SIZE in ${SIZE[@]}
do
echo "Runtimes for servlet 4," $SIZE "structures" >> test_times.txt
echo "" >> test_times.txt
# list values for N here
CPDS=( 10 100 500 1000 2500 5000 10000 25000 50000 75000 100000 )
for CPDS in ${CPDS[@]}
do
echo "Runs where N="$CPDS >> test_times.txt
# list number of repetitions for each run
REP=( 1 2 3 4 5 )
for REP in ${REP[@]}
do
echo 'run' $REP 'test_ActiMol_'$SIZE'.sdf -n 4 -t 1000 -l 100 -d t -N' $CPDS >> test_times.txt
( time ./servert.exe -n $NCLIENTS -t $TIMEOUT -l 100 -d t -N $CPDS \
-i 'test_ActiMol_'$SIZE'.sdf' \
-o 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s' ) >> test_times.txt 2>&1
echo "TIMEOUTS=" $(grep -c "TIMEOUT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
echo "SEGFAULTS=" $(grep -c "SEGFAULT" 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
echo "NUMBER PROCESSED=" $(wc -l 'test_ActiMol_'$SIZE'_'$CPDS'N_'$REP'.s') >> test_times.txt
echo "" >> test_times.txt
# print something to the terminal to monitor progress
echo 'completed run' $REP 'test_ActiMol_'$SIZE'.sdf -n' $NCLIENTS '-t' $TIMEOUT '-l 100 -d t -N' $CPDS
done
echo "" >> test_times.txt
done
echo "" >> test_times.txt
echo "" >> test_times.txt
done
unix2dos test_times.txt
It seems as if some of the multiple echo statements could be reduced using the here-document that Corona688 suggested. I will look into that. It also seems that I could echo a couple of newlines at once instead of doing it several times to insert several blank lines.
(/home/user1)-> more script.sh
#!/bin/ksh
( echo open devicename
sleep 3;
echo user;
sleep 2;
echo password;
sleep 2;
echo "/info/dump"; ---------> This needs to redirect to a file .Can be number of pages
sleep 2;
echo "exit" ) | telnet
Please use code tags next time for... (2 Replies)
Hi, although I am not expert in bash, so please forgive me if this is silly, I think that this is strange:
I have this command:
find . -type f -print0 |xargs -0 grep -i -e 'some rexp'
and it works fine. But when I create a bash script (on cygwin) to run this command, there is no output !!!... (3 Replies)
Hi
can anyone tell me how to redirect the ouput of a cvs command to a file as well as the console?
i tried using
cvs add <filename> | tee logFile
cvs add <filename> 2>logFile 2>&1
All i could get is only on console or on file.
Please help
Thanks (2 Replies)
Hello,
In my make file (make 3.81), I use a combination of shell commands to automatically create the name of my build directory.
OS := $(shell uname -s)
ARCH := $(shell uname -m)
KERN := $(shell uname -r | cut -d. -f 1,2)
BDIR := $(OS)_$(KERN).$(ARCH)When I boot into different OSs, I... (7 Replies)
When i use the command to check the preview of the filesets to be installed using CLI
# When using this commad 'm able to see all Preview view of the filesets to be installed
installp -apgX -d "." all
# When I redirected the same output to a file 'm able to see only half the details... (1 Reply)
Hi Everyone,
Can anyone please tell me, how can I redirect the grep command output to same file. I am trying with below command but my original file contains no data after executing the command.
$grep pattern file1 > file1
Kind Regards,
Eswar (5 Replies)
what does the '+' sign stand for on the output of ls command on cygwin?
-rw-------+ 1 milhan None 146783 Jun 19 12:10 schedule.pdf
-rw-------+ 1 milhan None 320 Aug 15 17:14 current.txt
-rw------- 1 milhan None 24576 Dec 28 2008 fdfd.txt (2 Replies)
hi,
i wat to get the output of a grep command in a file. but when i am trying out the same grep command in the unix prompt its working fine.. i am getting the output properly.. but when i am writing the same command inside my shell script , its just creating a new output file with no contents... (11 Replies)
Hello All,
I am trying to write a script that would capture the output of time command on a Sun machine.
I have tried these commands
$ time ls > out
$ time ls 2> out
I just get the ls output in the out file for the first one.
Thanks for any help
Chuck (2 Replies)