UNIX for Dummies Questions & Answers

Hello all,

This maybe a dumb question,

I have a series of awk command liners, perl and C executables that I usually run in a particular order to get an output. I need 2 files as input and then , a series of outputs are produced which pipe into the next step etc.

I am usually able to paste all these steps serially into command line terminal and get my desired output...Can I safely integrate all these steps into a bash script and run that with arguments as my two input filenames and output filename?

Example of paste into terminal pipeline

Code:

#Step 1 

awk '...' infile1 infile2 > tmp

# Step 2

awk '...'  infile2 tmp > tmp2


...

# Step 5

perl '....' > tmp5
... 

# Step 12 (Final step)

awk '.....' tmp45 > final_output

Yes you can integrate them smoothly,

Code:

./script.sh input_file1 input_file2 output_file

Then you can have your script like:

Code:

awk '..' $1 $2 > tmp //$1 is input_file1 and $2 is input_file2 

awk '...' $2 tmp > tmp2

....

awk '...' tmp45 > $3 // $3 output_file which is 3rd argument

In general anything you do at the command line you can script.

This is a bit confusing and seemingly inefficient namely because you are using a great number of files and using files more than once. Can these one liner awk scripts be combined?

For example, you awk infile2 twice. The first time it is used to create file tmp. The second time you do it it is awked with a file (tmp) that it was used to create.

The answer to your question in general is yes you can script it, but why is there a need to use all these files.

I need all the intermediate steps because, the intermediate files go into different pipelines once produced, you are right,some of the awk scripts can be combined and made more streamlined..