Shell Programming and Scripting

Hi,

I have run a lot of commands one after one,
but I can't run them simultaneous.
One command has to run and when finished second command has to run etc.
Also I would like to save all the outputs created by the commands in a file.
The commands can sometimes take hours, so it is also necessary to run them in the background or the whole script.
I am running bash shell.
The commands are very long, so I put them in these files: cmd1, cdm2, cmd3 etc
SO what I need is that the scripts runs and executes cmd1 and when finished , execute cmd2 etc.
This is what I got so far:

Code:

#! /bin/bash

# run commands in sequence

echo "Running cmd1"
sh cmd1 > output.txt 2>&1

wait

echo "Running cmd2"
sh cmd2 >> output.txt 2>&1


wait

echo "Running cmd3"
sh cmd3 >> output.txt 2>&1


echo " Scripts are finished !!"

Is it possible that the script itself fills in the number for the cmd files?
for example it will start with cmd1 and then cmd2 etc and stop until it can't find more?

Is this a homework assignment?

This is for production environment.

Maybe this is better, but how would this run all the commands :

Code:

#! /bin/bash

for i in {1..30} ;

do

echo "##########Running cmd$i ##########$(date)"  > output.txt
sh cmd$i >> output.txt 2>&1

The script that you suggested will overwrite the output of all previously executed scripts with the output of the last script to run. The wait commands in your script don't do anything. (You didn't start the cmd* scripts asynchronously and it wouldn't make any difference if you did.) If you have some long running cmd* scripts, run this shell script in the background; not the jobs started by the script. You might want to try something like:

Code:

#!/bin/bash
i=1
while [ -f cmd$i ]
do      sh cmd$i > cmd$i_stdout.txt 2> cmd$i_stderr.txt
        i=$((i + 1))
done

Note that you specified that this is a bash script, but you're executing the cmd* scripts using sh (rather than bash). If the scripts to be run are executable scripts, you could omit the "sh" in red in the above script and let a #! line in each script specify the shell to be used for any script that doesn't want to use the system's default shell. This script will run any number of cmd* scripts as long as their names have consecutive integer values (starting with "1") at the ends of the names.

PS I didn't notice that you had used >> instead of > for later jobs, so you won't be overwriting the output. You may still want to consider the redirection methods shown above to separate stdout and stderr for each script into separate files. I also skipped the echoes you had showing the progression of scripts being executed. If you want them, I assume you know where to add them. (If not; let me know.)

Thx Don for your quick reply.
Yes, I am using the bash, but should I use something else to run the commands?
I did run your script, but it didn't do anything.
Didn't get an error and no output files.
Seems like it didn't do anything.


---------- Post updated at 08:17 AM ---------- Previous update was at 07:33 AM ----------

Apologies, your command does work.
thx very much !!
but is it also possible to echo which command is going to run and date in front of every command in the log file?

I can't tell you what interpreter you should use to run your commands other than to say that the interpreter used has to be an interpreter that recognizes the commands it is being asked to run. You are the one who knows what is in the cmd* files. You are the one who knows if those files have a first line of the form #!interpreter. You are the one who knows if the cmd* files have been marked executable or are just readable. Without this information, I don't know what you should do. It just seemed strange that you were writing this script using bash, but using a Bourne shell to run the cmd* files. (Of course, you haven't said what system you're using; so sh and bash may be the same shell.)

Do you want all of the output from all of the cmd* files in one output file? Do you want the standard output from all of the cmd* files in one output file and standard error output from all of the cmd* files in a different output file? Or, do you want the standard output and standard error output for each cmd* file in separate output files (like the script I gave you currently does)?

Do you want the output showing which cmd* file is being started and the timestamp written to the same file as the standard output for the cmd* file that is about to run, to the standard output of the entire script, or to a different log file?

I am using bash.
The reason for using the sh was something I found on the internet, wasn't aware it was bourne shell.
I just want to direct all the output to one file, which I did manage by replacing " > cmd$i_stdout.txt 2> cmd$i_stderr.txt " with " > output.txt 2>&1"
I just need now need the timestamp when every cmd* commands runs in a different file (this is to see how long the commands run).

we get these commands usually in one file from a different department, they don't have enough privileges on their account to execute these command
every command creates and *.csv file, we just copy the commands one by one into a file (commands are very long) and then execute it and send the *.csv file by email and
in the beginning we just got couple of commands, but now we are getting a lot and
these commands can sometime take hours to run
this is too much effort, because sometimes one command can take hours to run and we have to wait for the command to finish, before we can move on to the next one,
I want to automate this as much as possible and the script you gave does the job. We are working on Solaris 10 and I am using the bash shell.
Thx.