Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk/nawk question to format a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk/nawk question to format a file
# 1  
Old 03-01-2009
awk/nawk question to format a file
Hi,

I am new to awk/nawk, needs help.

I want to merge the rows having emplid attribute same into a single row in the following file. In actual this kind of file will have around 50k rows.

Here is my input file

id|emplid|firstname|dep|lastname
1|001234|test|1001|1
2|002345|test|1032|2
3|001234|test|1020|1
4|123456|test|1044|4

If we see closly,lines 1 & 3 (as 001234 matches) are same but dep has different values.
I want to merge 1 & 3 lines into one line like following

id|emplid|firstname|dep|lastname
1|001234|test|1001 1020|1
2|002345|test|1032|2
3|123456|test|1044|4

Essentially I am trying to combine the rows and attribute dep where emplid is same or matching in another row(s).

Can you pl. help me how can I do in awk/nawk.

Please don't hesitate to ask if it needs more explanation.
Thanks in advance for your help.

Regards,
# 2  
Old 03-01-2009
There're plenty of similar threads - please use the 'Search' function first.
Please do come back if you have specific implementation question.
# 3  
Old 03-01-2009
hi , seems perl is a little bit easier to address your issue.
Code:
use strict;
my ($n,%hash)=(1);
open FH,"<a.txt";
while(<FH>){
  chomp;
	my @tmp=split("[|]",$_);
	if ( not exists $hash{$tmp[1]}){
		$hash{$tmp[1]}->{SEQ}=$n;
		$hash{$tmp[1]}->{VAL}=$_;
		$n++;
	}
	else{
		my @t=split("[|]",$hash{$tmp[1]}->{VAL});
		my $pre=join "|",@t[0..2];
		my $mid=$t[3]." ".$tmp[3];
		my $post=$t[4];
		$hash{$tmp[1]}->{VAL}=$pre."|".$mid."|".$post;
	}
}
close FH;
for my $key (sort { $hash{$a}->{SEQ} <=> $hash{$b}->{SEQ} } keys %hash){
	print $hash{$key}->{SEQ}."|".substr($hash{$key}->{VAL},index($hash{$key}->{VAL},"|")+1),"\n";
}

# 4  
Old 03-02-2009
Thank you for your valuable inputs. I 'll try the scripts. regards
# 5  
Old 03-03-2009
Hello Summer Cherry,

Its really a great help!! Appreciate from my heart. You are great !!

-Kumar
# 6  
Old 03-03-2009
Don't know about performance but awk solution seems more terse than perl:
Code:
awk -F'|' '{a[$2] = a[$2] " " $4} END {for (i in a) {nb += 1; print nb, i, a[i]}}' file

Although id numbering may not be what you expect.
# 7  
Old 03-03-2009
Another approach:

Code:
awk -F "|" '
NR==FNR{a[$2]=a[$2]?a[$2]" "$4:$4;next}
FNR==1{print;next}
a[$2]{$4=a[$2];a[$2]="";$1=++c;print}
' OFS="|" file file

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Date format conversion how to change this from using nawk to awk

Hi, I have a file where I need to change the date format on the nth field from DD-MM-YYYY to YYYY-MM-DD so I can accurately sort the record by dates From regex - Use sed or awk to fix date format - Stack Overflow, I found an example using nawk. Test run as below: $: cat xyz.txt A ... (2 Replies)
Discussion started by: newbie_01
2 Replies

2. Shell Programming and Scripting

Question about awk format

Hi, Guys, There is one question about AWK format. Here is the code: gawk -F: '/^Dan/ {print "Dan's phone number is ",$2}' lab3.data An syntax error will come out because the quote mark between Dan and s and first quote mark are recognized as a quote pair. I want to get the input like this:... (5 Replies)
Discussion started by: franksunnn
5 Replies

3. Shell Programming and Scripting

Using awk or nawk to convert epoch time to date format

Looking for some help and usually when I do a search this site comes up. Hopefully someone can give me a little direction as to how to use one of these two commands to achieve what I'm trying to do. What am I trying to do? I need to take the time value in epoch format returned from the... (5 Replies)
Discussion started by: minigts
5 Replies

4. Shell Programming and Scripting

NAWK: changing string-format with split

Hi all, I try to make a awk-script, which counts lines, summarized by pdf and xml. So far it works, but for sorting reasons, I'd like to change the format from the field $1 from dd-mm-yyyy to yyyy-mm-dd. This works find, but: split() and sprintf() prints its output (for no reason, the results... (2 Replies)
Discussion started by: regisl67
2 Replies

5. Shell Programming and Scripting

Nawk Format

Hi! I have a file which I want to search daily for any line that contains the work 'Reason' and I want to take that line and put the data in a certain format using awk or nawk....I do not have gawk on my machine so it would have to be awk or nawk, or sed would work as well. Here are some examples... (9 Replies)
Discussion started by: ther2000
9 Replies

6. Shell Programming and Scripting

Problem in splitiing file based on regex using awk/nawk

I have a file tmp.txt as shown below: Controller: 0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 1 Volume:c1t2d0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 2 Volume:c2t2d0 Disk: 0.2.0 Now I want to split... (4 Replies)
Discussion started by: durbam2002
4 Replies

7. Shell Programming and Scripting

how to parse the file in xml format using awk/nawk

Hi All, I have an xml file with the below format. <a>111</a><b>222</b><c>333<c><d><e>123</e><f>234</f><d><e>456</e><f>789</f> output needed is 111,222,333,123,234 111,222,333,456,789 nawk 'BEGIN{FS="<|>"} {print a,b,c,e,f a="" ... (7 Replies)
Discussion started by: natalie23
7 Replies

8. Shell Programming and Scripting

Format - Inventory Row data into Column - Awk - Nawk

Hi All, I have the following file that has computer data for various pcs in my network... Snap of the file is as follows ******************************************************************************* Serial 123456 Computer IP Address lo0:... (1 Reply)
Discussion started by: aavam
1 Replies

9. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

10. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies
Login or Register to Ask a Question