Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Order of execution seem to be reversed with pipe

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Order of execution seem to be reversed with pipe
# 1  
Old 01-12-2012
Order of execution seem to be reversed with pipe
Hello

I am seeing a weird behavior during the execution of the shell program.
The shell programs loops through a file which has a list of files. for each file the programs cats the file, pipes the output to sed command and then on to an awk program. But while execution, the order of execution(cat,sed and awk) is reversed(to cat, awk and sed) for one file and for the next file the order seems to be correct. This behavior alternates until the end of loop.
I have resolved this issue by rewriting in awk, but just curious to know if someone had come across this weird behavior or have any clue of what's going on.Smilie

Code:
Shell Program
---------------
 
while read line
do
   cat $line | sed 's/\^ /^/g' |awk -F\^ '/\.pdf/ { print $1"|ALCV"$2"-"$1"|"$3}' > $WORK_DIR/"$PREFIX"_rename.txt
done < $FILE

debug log - ^ is the delimiter in file.
-------------------------------------
 
PREFIX=CV
+ IOLIST=/nasa-dcs-work/011112/albrfs/091244/.iolist
+ [ -s /nasa-dcs-work/011112/albrfs/091244/CV_tmpfile2.txt -a CV = CR ]
+ [ -s /nasa-dcs-work/011112/albrfs/091244/CV_tmpfile2.txt -a CV = CV ]
+ 0< /nasa-dcs-work/011112/albrfs/091244/CV_tmpfile2.txt
+ read line
+ cat 20111115_CV.txt
+ awk -F^ /\.pdf/ { print $1"|ALCV"$2"-"$1"|"$3}
+ sed s/\^ /^/g
+ 1> /nasa-dcs-work/011112/albrfs/091244/CV_rename.txt
+ read line
+ cat 20111116_CV.txt
+ sed s/\^ /^/g
+ awk -F^ /\.pdf/ { print $1"|ALCV"$2"-"$1"|"$3}
+ 1> /nasa-dcs-work/011112/albrfs/091244/CV_rename.txt

Last edited by methyl; 01-12-2012 at 11:51 AM.. Reason: please use code tags.
# 2  
Old 01-12-2012
Please use code tags when posting code or sample i/o-put.

There are several things worth noting on your script:

1. Shell loops through file lines are very ineffective. You are using filters, which are operating on one line at a time, and are optimized for this kind of processing. No need for the shell loop.
2. sed command can be gotten rid of, if the FS of awk is specified in a more general way, e.g. -F'\\^ *' would eat up the spaces after the delimiter, or even more general -F' *\\^ *', which would handle situation with spaces before or/and after the ^.
3. The strangest thing there though, is that you redirect the output into the same file, for each line, and you are using the > operator, which will truncate it to zero length. So actually you will not get anything but the last line processed.
4. Useless use of cat

Let me suggest a better way to accomplish this:
Code:
outputFile="${WORK_DIR}/${PREFIX}_rename.txt"
: > outputFile  #truncate the file to zero length, just in case it exists already
awk -F' *\\^ *' -v output=$outputFile '/\.pdf/ {print $1"|ALCV"$2"-"$1"|"$3 >> output}'  $FILE

# 3  
Old 01-12-2012
mirni, Thanks for the reply. As i mentioned, i have resolved the issue, but just curious to know why this might be happening.
# 4  
Old 01-12-2012
Can you post sample of the content of $FILE, so that I can run it and see the behavior?
From my reading your original script, I am convinced that after running that code, I would find only one line in the _rename.txt output file, assuming the $FILE contains at least one .pdf.
I don't believe that there is any "weird behavior" there, it's just doing something else than the person who wrote it intended, which is pretty much a universal law in the field of programming. The computer programs don't do what we want, but what we write.
If you create a pipe
Code:
source | program1 | program2

How could possibly program2 execute before program1? It would have to be a major bug in the shell.
To be able to determine why you believed that the order reversed, I would need to see a sample input.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Order of execution of instructions in a script shell

Hi all, Here is a portion of my script test.sh: #!/bin/bash tar -cf $name.tar $name && pbzip2 $name.tar 2-Sending .tar.bz2 to destination node Please, i have a question. The execution of the second instruction is done, but i do not see the result of pbzip2 I have to send this... (2 Replies)
Discussion started by: chercheur111
2 Replies

2. Shell Programming and Scripting

Script Execution Order

Hi, I have two scripts which are mentioned in execute.sh more execute.sh ./script1.sh //line 1 should not return error ./script2.sh //line 2 may return error ./script2.sh //line 3 should not return error Condition: I want script1.sh to complete before starting script2.sh... (1 Reply)
Discussion started by: mohtashims
1 Replies

3. Shell Programming and Scripting

How to ignore Pipe in Pipe delimited file?

Hi guys, I need to know how i can ignore Pipe '|' if Pipe is coming as a column in Pipe delimited file for eg: file 1: xx|yy|"xyz|zzz"|zzz|12... using below awk command awk 'BEGIN {FS=OFS="|" } print $3 i would get xyz But i want as : xyz|zzz to consider as whole column... (13 Replies)
Discussion started by: rohit_shinez
13 Replies

4. UNIX for Dummies Questions & Answers

upside-down reversed desktop

I got some problem at startup before login such screen splashing, startup blocking on "checking battery status" and screen going black. Now, that I've solved the above problems I see the desktop screen is upside-down n reversed as well. I tried re-installing nvidia-current, re-installing... (1 Reply)
Discussion started by: dr_mabuse
1 Replies

5. Shell Programming and Scripting

Multiple Interval cron issue with order of execution

55,0 5,6 * * * myScript This cron task will execute on 5:00AM, 5:55AM, 6:00AM and 6:55AM. Is there any possibility to make it run only in the order specified such as 5:55AM and 6:00AM (basically only on the 2 intervals) ? (1 Reply)
Discussion started by: vikram3.r
1 Replies

6. Shell Programming and Scripting

Replace pipe with Broken Pipe

Hi All , Is there any way to replace the pipe ( | ) with the broken pipe (0xA6) in unix (1 Reply)
Discussion started by: saj
1 Replies

7. UNIX for Advanced & Expert Users

How to set the SGE job execution order for nested qsub?

Dear All I am trying to write a script to set the SGE job execution order. I named each job with 'job1', 'job2' and 'job3'. I want my script to do: When 'job1' execution is complete, 'job2' is executed; when both 'job1' and 'job2' are complete, 'job3' is executed. First, I tried this... (0 Replies)
Discussion started by: cliffyiu
0 Replies

8. Shell Programming and Scripting

Expect Issue Serial Forground Execution vs Concurrent Background Execution

I have an expect script that interrogates several hundred unix servers for both access and directories therein using "ssh user@host ls -l /path". The combination of host/path are unique but the host may be interrogated multiple times if there are multiple paths to test. The expect script is run... (2 Replies)
Discussion started by: twk
2 Replies

9. Solaris

Program execution order like FIFO queue

Hi all: I have a problem with a C++ service runing on solaris 10. This service only set a signal on oracle table. When the service detect a cut off on the line (tcp/ip), trigger a cobol program for set the signal OFF. When the line is on again, the service trigger the same cobol program for set... (0 Replies)
Discussion started by: hcastellani
0 Replies
Login or Register to Ask a Question