Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk experts please help

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk experts please help
# 1  
Old 01-16-2013
awk experts please help
I have a log file of 60 MB with 20k records which contains data like below.
Code:
this contains some data so removed


Code:
2891358271000020, 2012-12-02 23:16:17 , 2012-12-02 23:16:17 ,
378015123, 2012-12-02 23:16:19 , 2012-12-02 23:16:19 ,

Last edited by mirwasim; 01-17-2013 at 03:40 AM..
# 2  
Old 01-16-2013
I don't quite understand how you're generating this data from that input. The same date appears in multiple places, making it unclear which goes where. Not to mention there appears to be two records output for four records input, with some information repeated and some information not repeated in the output. How are they combined? What would happen if you had six of the same ID?
# 3  
Old 01-16-2013
This is first complete log(input flist and output flist).
input flist means when we receive info from upstream and output flist when we sent it to downstream

Code:
hi

so from this I am generating
Code:
2891358277113540, 2012-12-02 23:16:17 , 2012-12-02 23:16:17 ,

the script I have mentioned earlier is working fine but is taking time
Last edited by mirwasim; 01-17-2013 at 03:40 AM..
# 4  
Old 01-16-2013
Code:
awk '/PIN_FLD_UNIQUE_ID/ {
 uid=$NF;gsub(/"/,"",uid);
 print uid,dt,dt,per;
} /PIN_FLD_PERCENT/ {
 per=$NF;
} /UnknownProgramName/ {
 dt=sprintf("%s %s",$2,$3);
 gsub(/\..*/,"",dt);
}' OFS=, filename | awk '!a[$0]++'

# 5  
Old 01-16-2013
---------- Post updated at 02:19 AM ---------- Previous update was at 02:17 AM ----------

And
Code:
 #! /bin/sh or #! /bin/ksh

will work for it?

---------- Post updated at 02:22 AM ---------- Previous update was at 02:19 AM ----------

Below error I am getting
Code:
in

Code:
#! /bin/ksh
awk '/PIN_FLD_UNIQUE_ID/ {
 uid=$NF;gsub(/"/,"",uid);
 print uid,dt,dt,per;
} /PIN_FLD_PERCENT/ {
 per=$NF;
} /UnknownProgramName/ {
 dt=sprintf("%s %s",$2,$3);
 gsub(/\..*/,"",dt);
}' OFS=, $1 | awk '!a[$0]++'

Last edited by mirwasim; 01-17-2013 at 03:41 AM..
# 6  
Old 01-16-2013
It is an awk program, so you don't need a shebang: #!/bin/ksh

If you are on Solaris or SunOS use nawk or /usr/xpg4/bin/awk

For redirection:
Code:
nawk '/PIN_FLD_UNIQUE_ID/ {
 uid=$NF;gsub(/"/,"",uid);
 print uid,dt,dt,per;
} /PIN_FLD_PERCENT/ {
 per=$NF;
} /UnknownProgramName/ {
 dt=sprintf("%s %s",$2,$3);
 gsub(/\..*/,"",dt);
}' OFS=, filename | awk '!a[$0]++' > output_filename

# 7  
Old 01-16-2013
Yes I use sunOS
so tried nawk

please see error
Code:
nawk '/PIN_FLD_UNIQUE_ID/ {
 uid=$NF;gsub(/"/,"",uid);
 print uid,dt,dt,per;
} /PIN_FLD_PERCENT/ {
 per=$NF;
} /UnknownProgramName/ {
 dt=sprintf("%s %s",$2,$3);
 gsub(/\..*/,"",dt); 

}' OFS=, $1 | awk '!a[$0]++' > output_filename
pin@ssapp1025[sms]$ new.sh dm_sms.pinlog20121216231502
awk: syntax error near line 1
awk: bailing out near line 1

Last edited by mirwasim; 01-17-2013 at 03:41 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To all the awk experts out there!

Hello All, I recently encountered this difficulty in processing a File. Input File has millions of records with fields like below ID1,ID2,DATE,FLAG,VAL 123,432,0604,1,-0.5 123,432,0604,22,0.5 123,433,0604,1,-0.54 123,433,0604,22,6.77 123,543,0605,22,0.94 To put this simply, I will have... (8 Replies)
Discussion started by: PikK45
8 Replies

2. Shell Programming and Scripting

Couple of easy questions for experts on awk/sed

Hello Experts.. I have 3-4 C codes with Oracle SQL statements embedded. All the SQL statements starts with EXEC SQL keyword and ends with ;. I want to extract all the SQL statements out of these codes. I did awk '/^EXEC SQL/,/\;/' inputFile (I use this on all of the codes individually). That... (2 Replies)
Discussion started by: juzz4fun
2 Replies

3. UNIX for Advanced & Expert Users

Experts!! please help me

Hi experts, Please help me on the below: how to write a shell script to search and delete files on windows server. -script runs on unix box -it should search for specific files on windows server and delete them periodically. (1 Reply)
Discussion started by: chpradeepch
1 Replies

4. Shell Programming and Scripting

awk question for experts

Hi guys I am trying to perform a substitution using 'awk' command, but it fails. I work in ksh. Here is my code: $ line="F 18:30 10 23:00 ts1632back" $ n="ts1632back" $ m="18:45" $ echo ${line} | nawk -v a=$n -v b=$m '{if ($5==a) $2=m; print }' F 10 23:00 ts1632back $It should've... (2 Replies)
Discussion started by: aoussenko
2 Replies

5. UNIX for Advanced & Expert Users

for experts

Hi i'm working with mpi programs every thing ok but i need after i compile the mpi to calc the area of the rectangle for example (not my program) accept two arguments from the command line: mpirun -n 4 myprog 24 100 here 24 and 100 two arguments i'll pass them to the program how can... (1 Reply)
Discussion started by: Scotch
1 Replies

6. Solaris

Experts !!! Please advise

Hi, I work on sun Solaris. Am hosting few web services on my server which are accessed over the internet. Now to check whether the web service is responding or not, i first have to log in to the web service URL. If it doesn't respond there, i come back to my server box and restart the service... (4 Replies)
Discussion started by: sting672744
4 Replies

7. UNIX for Dummies Questions & Answers

Perl Experts - Need your help

Hi All, I am using ingres in perl select count(*) rec from user_tables where table_name = 'abc'; I want to use the alias variable 'rec' and check the value if >0 insert values else create table. How can I do this without using hash variables in perl. Kindly help me in this regard. Much... (1 Reply)
Discussion started by: karthickrn
1 Replies

8. UNIX for Dummies Questions & Answers

Looking for Advice from Experts

Where to start... I am a system administrator who didn't think he would ever be one. My first work was on Window 2000 as a tech(hardware, installs, stuff like that). Then we got Macs (I work in photography and Videography). The I was sent to Mac cert school. Set up a Mac Xserve and about 100... (2 Replies)
Discussion started by: Squidy P
2 Replies

9. Shell Programming and Scripting

Need help - from awk, sed experts

Hi , I have a file as below. Contents of the file are -------------------- aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa bbbbbbbbbbb ccccccccccc ddddddddddd aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa aaaaaaaaaaa bbbbbbbbbbb (4 Replies)
Discussion started by: Srini75
4 Replies
Login or Register to Ask a Question