Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

file manipulation using nawk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting file manipulation using nawk
# 1  
Old 10-16-2012
file manipulation using nawk
Legends,
I have two files (f1, f2) with below output.

Code:
file f1 contains:
TSCparser14     irons1         1 NORD_BALT_02 -- 0 gaps (0 missing messages), 0 seq no resets 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser21     irons1         1 EUREX_01    -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser22     irons1         1 EUREX_02    -- 0 gaps (0 missing messages), 0 seq no resets

file f2 contains:
TSCparser14     irons1         1 NORD_BALT_02 -- 0 gaps (0 missing messages), 0 seq no resets 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser21     irons1         1 EUREX_01    -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser22     irons1         1 EUREX_02    -- 0 gaps (0 missing messages), 0 seq no resets

f3 should be:
TSCparser14     irons1         1 NORD_BALT_02 -- 0 gaps (0 missing messages), 0 seq no resets 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser21     irons1         1 EUREX_01    -- 0 gaps (0 missing messages), 0 seq no resets
TSCparser22     irons1         1 EUREX_02    -- 0 gaps (0 missing messages), 0 seq no resets

From the last column in f3, difference (subtraction value of (col6 of f1 - col6 of f2)) should be printed along with text "times gaped in last 1 hr.")

I used nawk given by registered user "pamu" and worked well. but same awk if i modify to get the other fields, it gives me unwanted results.

Code:
nawk 'FNR==NR{a[$1,$2,$3]=$4;next}{if(a[$1,$2,$3] != ""){print $1,$2,$3,(a[$1,$2,$3]-$4)" times gapped in past 1 hr."}}' OFS="\t" f1 f2

pls help.
# 2  
Old 10-16-2012
Since the f1, f2, and f3 shown in your example are all identical; and since the string ("times gaped in last 1 hr.") that you say should appear in f3 does not appear in f3 in your example, I don't understand what you're trying to do. Subtracting 0 from 0 yielding 0 doesn't provide a good example of what you want to have happen.

I'm also not at all sure what you expect to get when you subtract text fields from each other (e.g., "NORD_BALT_02" - "NORD_BALT_02") which is $4 in the 1st record in f1 and $4 in the 1st record in f2. (You talk about column 6 in the text of your message, but you subtract values of $4 in your awk script???)
# 3  
Old 10-16-2012
Quote:
Originally Posted by Don Cragun
Since the f1, f2, and f3 shown in your example are all identical; and since the string ("times gaped in last 1 hr.") that you say should appear in f3 does not appear in f3 in your example, I don't understand what you're trying to do. Subtracting 0 from 0 yielding 0 doesn't provide a good example of what you want to have happen.

I'm also not at all sure what you expect to get when you subtract text fields from each other (e.g., "NORD_BALT_02" - "NORD_BALT_02") which is $4 in the 1st record in f1 and $4 in the 1st record in f2. (You talk about column 6 in the text of your message, but you subtract values of $4 in your awk script???)
It is not always "0". we get non-zero values too. $4 was the previous awwk, that contained 6th col value. now more columns are added so that became the 6th col in f1 and f2
# 4  
Old 10-16-2012
Quote:
Originally Posted by sdosanjh
It is not always "0". we get non-zero values too. $4 was the previous awwk, that contained 6th col value. now more columns are added so that became the 6th col in f1 and f2
You should change your script as per your input fileSmilie

try this..

Code:
$ nawk 'FNR==NR{a[$1,$2,$3]=$6;next}{if(a[$1,$2,$3] != ""){print $1,$2,$3,$4,$5,(a[$1,$2,$3]-$6)" times gapped in past 1 hr."}}' OFS="\t" file1 file2
TSCparser14     irons1  1       NORD_BALT_02    --      0 times gapped in past 1 hr.
TSCparser15     irons1  1       NORD_BALT_05    --      0 times gapped in past 1 hr.
TSCparser21     irons1  1       EUREX_01        --      0 times gapped in past 1 hr.
TSCparser22     irons1  1       EUREX_02        --      0 times gapped in past 1 hr.

Last edited by pamu; 10-16-2012 at 07:35 AM..
# 5  
Old 10-16-2012
Quote:
Originally Posted by sdosanjh
It is not always "0". we get non-zero values too. $4 was the previous awwk, that contained 6th col value. now more columns are added so that became the 6th col in f1 and f2
That doesn't alter the fact that f1, f2, and f3 in your example are identical and that f3 doesn't match the description you supply of what you want to appear in f3.

PLEASE give us sample f1, f2, and f3 where the contents of f1 and f2 are not identical and where the content of f3 is the actual data that you want to get when you process f1 and f2!

Posting an awk script that is not intended to work on the problem you're asking us to solve doesn't really help unless you show us the input that script got, the output that script produced and explain how that is related to what you want now.
# 6  
Old 10-16-2012
Quote:
Originally Posted by Don Cragun
That doesn't alter the fact that f1, f2, and f3 in your example are identical and that f3 doesn't match the description you supply of what you want to appear in f3.

PLEASE give us sample f1, f2, and f3 where the contents of f1 and f2 are not identical and where the content of f3 is the actual data that you want to get when you process f1 and f2!

Posting an awk script that is not intended to work on the problem you're asking us to solve doesn't really help unless you show us the input that script got, the output that script produced and explain how that is related to what you want now.
Sample file is something like below
Code:
f1
TSCparser14     irons1         1 NORD_BALT_02 -- 5 gaps (0 missing messages), 0 seq no resets 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets

f2
TSCparser14     irons1         1 NORD_BALT_02 -- 3 gaps (0 missing messages), 0 seq no resets 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets

f3 should be:
TSCparser14     irons1         1 NORD_BALT_02 -- 0 gaps (0 missing messages), 0 seq no resets, Total (col6 of f1 - col6 of f2 = 2) times gaped in past 1 hr 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets, Total (col6 of f1 - col6 of f2) times gaped in past 1 hr

---------- Post updated at 03:31 AM ---------- Previous update was at 03:28 AM ----------

@ pamu, but i want to print other lines too from file along with this message. i tried giving all the column values
Code:
like $5,$7,$8..

but it doesn't display the required results
# 7  
Old 10-16-2012
Quote:
Originally Posted by sdosanjh
Code:
f3 should be:
TSCparser14     irons1         1 NORD_BALT_02 -- 0 gaps (0 missing messages), 0 seq no resets, Total (col6 of f1 - col6 of f2 = 2) times gaped in past 1 hr 
TSCparser15     irons1         1 NORD_BALT_05 -- 0 gaps (0 missing messages), 0 seq no resets, Total (col6 of f1 - col6 of f2) times gaped in past 1 hr

----------
try this..

Code:
nawk 'FNR==NR{a[$1,$2,$4]=$6;next}{if(a[$1,$2,$4] != ""){s=(a[$1,$2,$4]-$6);$6=0;print $0", Total "s" times gapped in past 1 hr."}}' OFS="\t" file1 file2

This User Gave Thanks to pamu For This Post:
sdosanjh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Nawk Problem - nawk out of space in tostring on

Hi.. i am running nawk scripts on solaris system to get records of file1 not in file2 and find duplicate records in a while with the following scripts -compare nawk 'NR==FNR{a++;next;} !a {print"line"FNR $0}' file1 file2duplicate - nawk '{a++}END{for(i in a){if(a-1)print i,a}}' file1in the middle... (12 Replies)
Discussion started by: Abhiraj Singh
12 Replies

2. Shell Programming and Scripting

Populating File data with custom manipulation on file names

Hi, I am confused how to proceed firther please find the problem below: Input Files: DCIA_GEOG_DATA_OCEAN.TXT DCIA_GEOG_DATA_MCRO.TXT DCIA_GEOG_DATA_CVAS.TXT DCIA_GEOG_DATA_MCR.TXT Output File Name: MMA_RFC_GEOG_NAM_DIM_LOD.txt Sample Record(DCIA_GEOG_DATA_OCEAN.TXT):(Layout same for... (4 Replies)
Discussion started by: Arun Mishra
4 Replies

3. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

4. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

5. Shell Programming and Scripting

Reformat file using nawk

Hi all, I have a file with records that look something like this, "Transaction ID",Date,Email,"Card Type",Amount,"NETBANX Ref","Root Ref","Transaction Type","Merchant Ref",Status,"Interface ID","Interface Name","User ID" nnnnnnnnn,"21 Nov 2011 00:10:47",someone@hotmail.co.uk,"Visa... (2 Replies)
Discussion started by: dazedandconfuse
2 Replies

6. Shell Programming and Scripting

Help nawk parse file

Hello, Hope you are doing fine. I have a file in following format. I only want to process the data inside the section that comes after #DATE,CODE,VALUE #ITEMS WITH CORRECTIONS ....... #DATE,CODE,VALUE 2011-08-02, ID1, 0.30 2011-08-02, ID2, 0.40 2011-08-02, ID3, 0.50 ...... Means... (3 Replies)
Discussion started by: srattani
3 Replies

7. Shell Programming and Scripting

nawk --- can't open file ??

While i am trying to execute nawk in korn shell iam getting this error. nawk: can't open file $directory../../../filename. When the file is in home directory it is executing. But its not able to find file in other directory. Thanks (2 Replies)
Discussion started by: Diddy
2 Replies

8. Shell Programming and Scripting

Add a column to a file with nawk

Hi all, I'm new at forum, I cant find an answer to my problem so , I need a file which has pipe as a file separator and I need to add a column to a file in the third column I write this code but it s not enough , cat allproblems | nawk'\ BEGIN { FS:"|" } {print $3 $4 $5, ????} ' ... (7 Replies)
Discussion started by: circuitman06
7 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. UNIX for Advanced & Expert Users

nawk - file limits

Hi, I want to search particular pattern and splitting the file in to multiple files. (Splitted files may be more than 150). It got splitted upto 20 files after that, I got some error. nawk: filename.21 makes too many open files. input record number 654, file xxxxxxx Can u guide me to... (1 Reply)
Discussion started by: sharif
1 Replies
Login or Register to Ask a Question