Merging files with AWK filtering and counting lines - Shell Programming and Scripting

Meert · #1 (**permalink**) 01-28-2008

Hi there,

I have a couple of files I need to merge. I can do a simple merge by concatenating them into one larger file.

But then I need to filter the file to get a desired result.

The output looks like this:

Code:

TRNH 0000000010941
ORDH
OADR
OADR
ORDL
ENDT    1116399         000000003 000000001
TRLR 0000000010941 000000003 000000001
TRNH 0000000010942
ORDH
OADR
OADR
ORDL
ENDT    1116400         000000003 000000001
TRLR 0000000010942 000000003 000000001
TRNH 0000000010943
ORDH
OADR
OMSG
ORDL
ORDL
ENDT    1116399         000000004 000000001
TRLR 0000000010943 000000003 000000001
TRNH 0000000010944
ORDH
OADR
OADR
ORDL
ENDT    1116400         000000003 000000001
ORDH                                       
OADR                                       
OADR
ORDL                                       
ORDL                                       
ENDT    1116400         000000004 000000001
TRLR 0000000010944 000000007 000000002

and the filtering should leave the first and the last line with the TRNH and TRLR (giving the last TRLR the same sequence as the first TRNH). The rest of the TRNH and TRLR lines need to be omitted.

Then the final TRLR should represent the amount of ORDH lines and the amount of OADR, OMSG and ORDL lines.

I haven't gotten to deleting the extra TRNH and TRLR lines yet; this is the filter I have so far:

Code:

BEGIN {
#	 define two counters 
	ordh_cnt = 0;
	ordl_total_cnt = 0;
}

#  Start filter 

#  if line start with ORDH add 1 to counters 
$1 == "ORDH" {
	ordh_cnt++;
}

#  if line starts with TRLR, adjust line to reflect new count of ORDH in order
$1 == "TRLR" {
	printf "%s%9.9d%s\n", substr($0, 0, 31), ordh_cnt, substr($0, 39);
#	 line has been printed, next rule 
	next;
}

#  if line start with ORDL add 1 to counters 
$1 == "ORDL" {
	ordl_total_cnt++;
}

#  if line start with OADR add 1 to counters 
$1 == "OADR" {
	ordl_total_cnt++;
}

#  if line start with OMSG add 1 to counters 
$1 == "OMSG" {
	ordl_total_cnt++;
}

#  if line starts with TRLR, adjust line to reflect new total ORDL, OADR and OMSG in complete file 
$1 == "TRLR" {
	printf "%s%9.9d%s\n", substr($0, 0, 19), ordl_total_cnt, substr($0, 29);
#	 line has been printed, next rule 
	next;
}

#  Line has not changed, print normal line 
{
	print $0;
}

Now the amount of ORDH lines is output to my new file, so that seems to work. Yet the amount of OADR, OMSG and ORDL lines isn't corrected in the output.

The final result should look like this:

Code:

TRNH 0000000010941
ORDH
OADR
OADR
ORDL
ENDT    1116399         000000003 000000001
ORDH
OADR
OADR
ORDL
ENDT    1116400         000000003 000000001
ORDH
OADR
OMSG
ORDL
ORDL
ENDT    1116399         000000004 000000001
ORDH
OADR
OADR
ORDL
ENDT    1116400         000000003 000000001
ORDH                                       
OADR                                       
OADR
ORDL                                       
ORDL                                       
ENDT    1116400         000000004 000000001
TRLR 0000000010941 000000017 000000005

Any help would be greatly appreciated

radoulov · #2 (**permalink**) 01-28-2008

Code:

awk 'NR == 1 { trnh = $2; print }
!/^TR(NH|LR)/ { 
  if ($1 == "ORDH") 
    ordh ++ 
  if ($1 ~ /^O(ADR|MSG|RDL)/) 
    ordl ++
  print 
} END {
printf "TRLR %s %.9d %.9d\n", trnh, ordl, ordh
}' filename

Meert · #3 (**permalink**) 01-28-2008

Thank you very much, works like a charm!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Merging lines using AWK	senthil_is	Shell Programming and Scripting	6	03-05-2008 08:40 AM
Merging lines - Tuning the script	senthil_is	Shell Programming and Scripting	3	03-04-2008 11:24 PM
merging two lines in a file	thaduka	UNIX for Dummies Questions & Answers	6	07-11-2007 06:27 AM
Merging lines into one	Foxgard	UNIX for Dummies Questions & Answers	8	06-19-2005 03:36 AM
Counting lines and files	jorge.ferreira	UNIX for Dummies Questions & Answers	6	12-11-2003 07:24 AM

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode