Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help with shell script using multiple awks

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help with shell script using multiple awks
# 1  
Old 05-09-2012
Need help with shell script using multiple awks
I have a command lline script that works perfectly.

Code:
cat <some_filename> | awk '$9 == 200' | awk '$10 == 10623720' | awk -F\" '{print $6}' | sort | uniq -c | sort -r

I want to put this in a shell script so that we do this for multiple files(all read from a file list) and write each output to different files or folders. The shell script I am trying is as follows.

Code:
#!/bin/bash

FILENAME=$1
OUTDIR1=outdir1
OUTDIR2=outdir2
OUTDIR3=outdir3
count=0
cat $FILENAME | while read LINE
do
       let count++
       echo "$count $LINE" >> res.txt
       # Full and download of data
       cmd="cat $LINE | awk '\$9 == 200' | awk '\$10 == 10623720' | awk -F"\" '{print \$6}' | sort | uniq -c | sort -r >> $OUTDIR1/$LINE" 
       eval $cmd
done

I am not sure what exactly I am doing wrong. This is the error I get.

Code:
trends.sh: line 16: unexpected EOF while looking for matching `"'
trends.sh: line 19: syntax error: unexpected end of file

Let me know what I am doing wrong.
Last edited by Scrutinizer; 05-09-2012 at 03:47 AM.. Reason: code tags
# 2  
Old 05-09-2012
can you post the sample input data and the required output.

some suggestions:

Code:
 
dont use 
 
1) cat filename | while....... done
 
use 
 
while......done < filename
 
2) Dont use eval
 
3) dont use cat with awk
 
cat filename | awk ....
 
use 
 
awk ..... filename
 
4) you can do all comparison in one awk
 
awk '$9 == 200' | awk '$10 == 10623720' | awk -F\" '{print $6}' 
 
use 
 
awk '$9==200 && $10==10623720 {print $6}'

# 3  
Old 05-09-2012
Could you post a sample of your input file?
# 4  
Old 05-09-2012
Sample data

Code:
  213.236.258.22 tx-cdn.ops.com - [26/Mar/2012:06:21:03 -0700] "GET /pub/ops/linux/1161/ops_11.61.1250_i386.deb HTTP/1.1" 200 11455274 "http://www.ops.com/download/get.pl?&id=34384&autoupdate=1&location=385&thanks=yes&sub=yes" "Ops/9.80 (X11; Linux i686; U; en) Presto/2.10.229 Version/11.61" 24.000000 
  89.189.95.222 tx-cdn.ops.com - [26/Mar/2012:06:29:20 -0700] "GET /pub/ops/win/1161/autoupdate/Ops_11.61_int_Setup.exe HTTP/1.1" 200 96480 "-" "Ops/9.80 (Windows NT 5.1; U; en) Presto/2.10.229 Version/11.60" 6.000000 
  114.79.78.207 tx-cdn.ops.com - [26/Mar/2012:07:09:29 -0700] "GET /pub/ops/win/1161/autoupdate/Ops_1161_int_Setup.msi HTTP/1.1" 206 109200 "http://tx-cdn.ops.com/pub/ops/win/1161/autoupdate/" "Ops/9.80 (Windows NT 5.1; U; en) Presto/2.6.30 Version/10.62" 5.000000 
  39.249.19.27 tx-cdn.ops.com - [26/Mar/2012:07:07:45 -0700] "GET /pub/ops/win/1161/autoupdate/Ops_1161_int_Setup.msi HTTP/1.1" 206 176660 "http://tx-cdn.ops.com/pub/ops/win/1161/autoupdate/" "Ops/9.80 (Windows NT 6.1; U; Edition Telkomsel; id) Presto/2.6.30 Version/10.62" 21.000000 
  125.184.23.63 tx-cdn.ops.com - [26/Mar/2012:07:08:28 -0700] "GET /pub/ops/win/1161/autoupdate/Ops_11.61_int_Setup.exe HTTP/1.1" 206 68244 "http://tx-cdn.ops.com/pub/ops/win/1161/autoupdate/" "Ops/9.80 (Windows NT 6.1; U; en) Presto/2.10.229 Version/11.60" 2.000000

Last edited by Scrutinizer; 05-09-2012 at 04:10 AM.. Reason: code tags
# 5  
Old 05-09-2012
provide the expected output
# 6  
Old 05-09-2012
Try:
Code:
awk -F' *\" *' '$3=="200 11455274"{print $6}' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help getting a UNIX shell script to look at multiple files.

Hey everyone! I made a shell script that would go through a file and replace any phrase or letter with another phrase or letter. Helps update variable names or values. The following code is this: #!/bin/sh Word1="$1" Replace1="$2" File1="$3" arg=$( echo "$Word1" | sed 's:\:\\&:g' )... (3 Replies)
Discussion started by: rebmonk
3 Replies

2. Shell Programming and Scripting

Shell Script for viewing multiple logs from multiple server

I am new to Shell scripting and below is my requirement. I need to search some specific word e.g. "exception" or "transaction" from log file. We have multiple env e.g. Level1 , Level2 etc and each env have Multiple boxes e.g. For Level 1 env we have "test11.test.com" , "test12.test.com". Each... (1 Reply)
Discussion started by: peeyush
1 Replies

3. Shell Programming and Scripting

help with multiple loops in shell script

Hi Guys- I'm trying to write a script which takes date as input (mm.yy.dd) and search in the current file. If pattern doesn't exist it will then look in a backup directory and so on. being a newb i'm unable to loop over to the backup directory. hoping for some ideas, i've highlighted the... (1 Reply)
Discussion started by: Irishboy24
1 Replies

4. Shell Programming and Scripting

shell script for multiple logging

Hi All, I am preparing a script which executes following things: 1) Logs into 8 cluster one by one. 2) After logging into each cluster,it prints the cluster name & then exit from that cluster. 3) Then it logs to next cluster & peform the same task. Here is what i have written : for... (8 Replies)
Discussion started by: d8011
8 Replies

5. Shell Programming and Scripting

combine 2 awks statement into 1 liner

Using these 2 comands to concatenate both outputs into single file: cat testdata | awk 'BEGIN { FS="\n"; RS=""; } /<pattern1>/ {print}' > testdata1 cat testdata| awk '/<pattern2>/,EOF' >> testdata1 is it possible to combine both "awk" into 1-liner? pls advise and thanks in advance. (5 Replies)
Discussion started by: ux4me
5 Replies

6. Shell Programming and Scripting

How to Use Multiple if Conditions in Shell script

if -o ] then echo "Expected valid value" The above multiple if condition is NOT working in my script. I am getting the error as '-a' not expected. Can anyone help with the syntax for this? (5 Replies)
Discussion started by: dinesh1985
5 Replies

7. Shell Programming and Scripting

Multiple instances of the job in shell script.

Hi, Please let us know how to create a multiple instances of a job in the shell script. Thanks. Gangegowda K.G (1 Reply)
Discussion started by: Gangegowda
1 Replies

8. Shell Programming and Scripting

Multiple paramters to a shell script

Hi All, Can anyone tell me how to pass and handle 'N' parameters in a shell script. Thanks, Sumesh (2 Replies)
Discussion started by: sumesh.abraham
2 Replies

9. Shell Programming and Scripting

running simultaneous awks

Hello, I have an awk command that searches and replaces. I have multiple searches, but I do not want to do them one after the other. Is there a way in awk to run search/replace at the same time. thanks, (3 Replies)
Discussion started by: ctcuser
3 Replies

10. Shell Programming and Scripting

Multiple su - in Korn Shell script

i am trying to do something like this : #!/bin/ksh # Change to the userid user1 su - user1 #Issue the command to change directory and list files cd /home/user1/ ls -lrt exit #Come out of the user1 to root again #change to user 2 su - user2 cd /home/user2/ ls -lrt... (2 Replies)
Discussion started by: furrari
2 Replies
Login or Register to Ask a Question