Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with executing awk and While loop

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with executing awk and While loop
# 1  
Old 03-18-2013
Help with executing awk and While loop
Hi All,
I have a file say, sample.txt
Code:
Source Name: xxx
Department|Revenue
1001|3252
1002|3345

I am using the above file in one of my script. I need to read from Line 3 of the above the file and do some process.

My script has a code:
Code:
awk 'NR > 2' sample.txt | while read Dep; do awk -F '|' '{print $1}' ; done

The Output is:

Code:
1002   #2nd Department Name in Line 4

Expected is:
Code:
1001   #1st Department Name in Line 3
1002   #2nd Department Name in Line 4

The above command ignores line 3 and starts from line 4 even after giving as awk 'NR > 2'. Not sure why this happens.

Just want to understand why awk skips Line 3 even if it mentioned as 'NR > 2' Smilie
Last edited by machomaddy; 03-18-2013 at 07:24 AM..
# 2  
Old 03-18-2013
I did this, and got ur expected output!!

Code:
 
echo "Source Name: xxxDepartment|Revenue1001|32521002|3345" | awk -F"|" 'NR>2 {print $1}'

# 3  
Old 03-18-2013
I tried...Itz nt working. While is not reading the 1st line Smilie. I even created a temp file with Dept names alone and passed it in while loop as below
Code:
while read i
do
.
.
.
done < temp.txt

While is skipping the 1st line Smilie

---------- Post updated at 05:17 PM ---------- Previous update was at 05:09 PM ----------

OKAY...Got it!!!

WRONG CODE Smilie
Code:
awk 'NR > 2' sample.txt | while read Dep; do awk -F '|' '{print $1}' ; done

Code:
awk 'NR > 2' sample.txt | while read Dep; do 
echo $Dep | awk -F '|' '{print $1}' 
 done

[SOLVED]
# 4  
Old 03-18-2013
OK. Let's analyze what is happening. We have a file named sample.txt containing:
Code:
Source Name: xxx
Department|Revenue
1001|3252
1002|3345

and we have a script (reformatted and line numbers added for discussion):
Code:
1  awk 'NR > 2' sample.txt |
2  while read Dep;
3  do awk -F '|' '{print $1}' ;
4  done

So, first, the awk on line 1 reads and discards the first two lines from sample.txt and writes the last two lines from sample.txt into the pipe feeding line 2.

The read on line 2 sets Dep to 1001|3252 (consuming the 3rd line from sample.txt that was the 1st line output by the awk on line 1).

Then (since no other input file is specified) the awk on line 3 reads the remainder of the output from the awk on line 1 and prints 1002.

Then the next call to read on line 2 hits EOF and terminates the while loop.

If you want the awk on line 3 to process the data stored in $Dep, you could change your script to something like:
Code:
awk 'NR > 2' sample.txt | while read Dep; do echo "Processing $Dep"; echo "$Dep" | awk -F '|' '{print $1}' ; done

Note the text in red that I've added to your script. This modified script produces the output:
Code:
Processing 1001|3252
1001
Processing 1002|3345
1002

I hope this helps,
Don
# 5  
Old 03-18-2013
All this could have been done in a single command.. No need of echo, 3-4 awk and while loops Smilie

Code:
 
awk -F"|" 'NR>2 { print "Processing "$0; print $1;}' file

This User Gave Thanks to PikK45 For This Post:
machomaddy
# 6  
Old 03-18-2013
Quote:
Originally Posted by PikK45
All this could have been done in a single command.. No need of echo, 3-4 awk and while loops Smilie

Code:
 
awk -F"|" 'NR>2 { print "Processing "$0; print $1;}' file

Of course, I agree.

I assumed machomaddy plans to perform other processing on each (non-header) line of an input file and that the second awk was a placeholder for some other processing.

If all that machomaddy wants is to print up to the first vertical bar in every line after line 2 from an input file:
Code:
sed '3,$s/|.*//' sample.txt

would be much more efficient.
This User Gave Thanks to Don Cragun For This Post:
machomaddy
# 7  
Old 03-18-2013
I agree Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Storing passing and executing select statement in loop

Hi, i want to do the following: Grep the following kind of strings for the 15digit ID which is stored in filename1: "14:06:51.396 INFO BMCREMEDYSD INPUT-ACTION Failed to retrieve Remedy Incident Modification record: 000000000039047 org.apache.axis2.AxisFault: Read timed out - complete... (9 Replies)
Discussion started by: Khushbu
9 Replies

2. Shell Programming and Scripting

Issues with executing awk

I am piping some output to awk and would like to print fields $1 $2 and $3 $4 only if they exist. Note the awk begins with awk '{print $NF " " since I want the last field printed first. (7 Replies)
Discussion started by: motdman
7 Replies

3. Shell Programming and Scripting

"if" Loop not working when executing script using cron

I am facing this weird issue where the script is working fine from the command line but when I am executing it from cron though it is working fine but the "if" loop is processing else part though I know that the if part of the logic is true and ideally the loop should execute the if portion. ... (3 Replies)
Discussion started by: sk2code
3 Replies

4. Shell Programming and Scripting

awk loop using array:wish to store array values from loop for use outside loop

Here's my code: awk -F '' 'NR==FNR { if (/time/ && $5>10) A=$2" "$3":"$4":"($5-01) else if (/time/ && $5<01) A=$2" "$3":"$4-01":"(59-$5) else if (/time/ && $5<=10) A=$2" "$3":"$4":0"($5-01) else if (/close/) { B=0 n1=n2; ... (2 Replies)
Discussion started by: klane
2 Replies

5. Shell Programming and Scripting

Problem in executing For loop....

Hi frnds I trying to execute the following ksh, #!/bin/ksh file=$OBS_APP_PATH/config/com/uhg/obs/inbound/configs/ServiceFeeDetail.xml inputFileDir=$OBS_APP_PATH/config/com/uhg/obs/inbound/configs/ echo $file echo $cntWrd if then echo "Inside IF" for i in 'ls -t1... (7 Replies)
Discussion started by: balesh
7 Replies

6. Shell Programming and Scripting

cd command not executing in if else loop

Hi, I am executing below script s1=`pwd` s2=/space if then echo "done" else echo "mistake" cd /tmp fi I am not able to migrate to /tmp directory if the condition is not true.However mistake is being printed.Means cd command is not working here.All other commands except cd are... (3 Replies)
Discussion started by: d8011
3 Replies

7. Shell Programming and Scripting

error executing script in a while loop

Im unable to run scripts when i read each script thru a while loop. Is this way of execution thru while loop is wrong or is there any error in the script. I get the following error msg and i use ksh. ./vftest.ksh: ./add.ksh -customer 4875 -dim RD,TRND,TT,HS,MRKT,PRDC,ACV,CV,FCT: not found ... (7 Replies)
Discussion started by: michaelrozar17
7 Replies

8. Shell Programming and Scripting

ssh and executing a for loop

Hi all, I am trying to run a script which is expected to do: on the remote machine, There are two directories /export/home/abc1,/export/home/abc2 i am trying to do, ssh SERVERNAME "for i in `ls -l /export/home/abc*|awk '{print $9}'`; do cd $i; ls -l; done" But its not working ,iam... (11 Replies)
Discussion started by: Jartan
11 Replies

9. Shell Programming and Scripting

Script not executing second loop

I have a server that receives backup files from several servers. Each server has its own directory to scp their files into, some of the files are received as .tar files and need to be compressed before being dumped. When the scp of the tar file is complete a file named 'flag' is also sent to... (2 Replies)
Discussion started by: thumper
2 Replies

10. Shell Programming and Scripting

Print out loop index on the console after executing each sybase DB query

Hello Guys, Well, using shell script, I'm doing loop on DB query as below: isql -Usa -Ptest -I /opt/sybase/interfaces << EOF use testdb go declare @i int select @i = 1 while(@i <= 5) begin Insert into TEST values (@i,"Test","TestDesc") select @i = @i + 1 end go EOF The Issue... (2 Replies)
Discussion started by: Alaeddin
2 Replies
Login or Register to Ask a Question