Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk script (complex)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk script (complex)
# 8  
Old 03-11-2012
Since Solaris 8 dosn't offer may options for date calculations you will probably have to end up using perl (or c) to do this one.

If you have any control of the cron jobs, I'd suggest swapping the date format around to YYYYMMDD. This would make things a lot easier.
# 9  
Old 03-11-2012
thanks for your help but it will not be possible for me to do this as I do not have control of the cron.
# 10  
Old 03-12-2012
This might be close to what you need.

It compares the newest file against the 50 most recent and any record that appears in more than 20 files is printed:

Code:
cd /path/to/cisco/logs
files=`ls ciscostats_* | sort -t_ -k2.5 -k2.3,2.4 -k2.1,2.2 | tail -50`
first=`echo "$files" | tail -1`
 
nawk -F, 'NR==FNR{a[$1","$2","$3]++;next}
a[$1","$2","$3]{b[$1","$2","$3]++}
END{for(i in b)if(b[i]>20)print i";\t\t"b[i]}' $first $files

Last edited by Chubler_XL; 03-12-2012 at 12:21 AM..
This User Gave Thanks to Chubler_XL For This Post:
slashbash
# 11  
Old 03-12-2012
Thanks Chubler but I still need it to compare against all other files and any lines not matching it prints as new, see my second point.
# 12  
Old 03-12-2012
OK give this a try, you can change the value LOOK to control how many files are searched and MATCH for how many must match.

Code:
cd /path/to/cisco/logs
files=`ls ciscostats_* | sort -t_ -k2.5 -k2.3,2.4 -k2.1,2.2`
first=`echo "$files" | tail -1`
 
awk -F, -vLOOK=50 -vMATCH=20 '
  FNR==1{F++}F==1{a[$1","$2","$3]++;next}
  a[$1","$2","$3]&&F<LOOK{b[$1","$2","$3]++}
  a[$1","$2","$3]{c[$1","$2","$3]++}
  END{for(i in c)if(b[i]>MATCH)print i";\t\t"b[i];else if(c[i]-1)print "NEW:"i";\t\t"b[i]}' $first $files

This User Gave Thanks to Chubler_XL For This Post:
slashbash
# 13  
Old 03-12-2012
im just getting the new output with that code, no >20
# 14  
Old 03-12-2012
Are the numbers your getting on the NEW: lines bigger than 20? I'm still a bit confused about what NEW lines should be, if it's just records that only appear in the most recent file then this might work better:

Code:
cd /path/to/cisco/logs
files=`ls ciscostats_* | sort -t_ -k2.5 -k2.3,2.4 -k2.1,2.2`
first=`echo "$files" | tail -1`
 
awk -F, -vLOOK=50 -vMATCH=20 '
  FNR==1{F++}F==1{a[$1","$2","$3]++;next}
  a[$1","$2","$3]&&F<LOOK{b[$1","$2","$3]++}
  a[$1","$2","$3]{c[$1","$2","$3]++}
  END{for(i in c)if(b[i]>MATCH)print i";\t\t"b[i];else if(c[i]==a[i])print "NEW:"i";\t\t"c[i]}' $first $files

Last edited by Chubler_XL; 03-12-2012 at 01:21 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk in complex number data

Hi, I'm trying to transform my data from the following format: eps:, 0.248281687841641, -2.83539034165844e-7, 2.78042576353472+6.3505226053266e-6i to this: eps:, 0.248281687841641, -2.83539034165844e-7, 2.78042576353472, +6.3505226053266e-6 so I can plot it with GnuPlot. how do I... (4 Replies)
Discussion started by: rogeriogouvea
4 Replies

2. Shell Programming and Scripting

Building a complex xml using awk

Hi I have a complex xml to be build using awk using a lookup file values. Below is the xml <country name="xyz"> <state name="abc"> <city name="qwe" capital="yes"/> <city name="asd" capital="no"/> </state> <state name="qrq"> <city name="rthy" capital="yes"/> <state> </country> ... (1 Reply)
Discussion started by: Nevergivup
1 Replies

3. Shell Programming and Scripting

Complex awk problem

hello, i have a complex awk problem... i have two tables, one with a value (0 to 1) and it's corresponding p-value, like this: 1. table: ______________________________ value p-value ... ... 0.254 0.003 0.245 0.005 0.233 0.006 ... ... ______________________________ and a... (6 Replies)
Discussion started by: dietmar13
6 Replies

4. Shell Programming and Scripting

Complex transpose awk script

Hello to all in forum, Maybe an awk expert could help me with this complex task for me. I have the input shown below and I would like to get the output as follow: - I would like the output separated by commas. - The header is fixed and will be the same always. - For the lines containing... (22 Replies)
Discussion started by: Ophiuchus
22 Replies

5. Shell Programming and Scripting

Help with Complex Awk.

Hi, I have a file. In this file when ever the word "ABC" occurs at position from 25 and 34 I would like to replace the value at postion 100 to 5 for the first 1000 rows only. I have written the following Awk command. nawk 'substr($0,25,9)=="ABC" {print $0}' filename The above command... (4 Replies)
Discussion started by: pinnacle
4 Replies

6. Shell Programming and Scripting

Complex match of numbers between 2 files awk script

Hello to all, I hope some awk guru could help me. I have 2 input files: File1: Is the complete database File2: Contains some numbers which I want to compare File1: "NUMBERKEY","SERVICENAME","PARAMETERNAME","PARAMETERVALUE","ALTERNATENUMBERKEY"... (9 Replies)
Discussion started by: Ophiuchus
9 Replies

7. Shell Programming and Scripting

complex Awk Question

Hi, I have a file look likes this : --->start hir Trace file: pudwh_ora_9998.trc Sort options: fchela exeela ***************************************************************count = number of times OCI procedure was executed cpu = cpu time in seconds executing elapsed = elapsed... (3 Replies)
Discussion started by: yoavbe
3 Replies

8. Shell Programming and Scripting

Sorting complex file with awk

i have a file ddd.txt its delimiter is : but has , and "" within each column as below and also each line ends with ; I_EP,"29":I_US,"120":I_P_ID,"2020":I_NEW,"600":I_OLD,"400":I_POW,"4.5":I_NAME,"TOM";... (9 Replies)
Discussion started by: blackzinga80
9 Replies

9. Shell Programming and Scripting

Complex use with awk

Hi , I have file named docs.txt The content of the file look like this: DOC disk location Size ======= ===== ============= ========= TXT A /dev/dm-1 10 TXT B /dev/dm-2 10 BIN C ... (3 Replies)
Discussion started by: yoavbe
3 Replies

10. Shell Programming and Scripting

Complex Sed/Awk Question?

Hello, So i have this file called /apps/turnout which looks like that of the contents of the /etc/shadow (but not exactly) the file has a long list in it. basically, the contents of this file looks something similar to the following: jajajajalala:D#$#AFVAdfda lalabavisof:#%R@fafla#$... (3 Replies)
Discussion started by: SkySmart
3 Replies
Login or Register to Ask a Question