File manipulation using AWK


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File manipulation using AWK
# 1  
Old 07-04-2006
File manipulation using AWK

Hi All,
I have a file having content,
$ cat data1.txt
20060620 142 62310 959400 A 5.00
20060620 142 62310 959400 B 3.00
20060620 143 62310 959401 A 7.00
20060620 143 62310 959401 B 4.00
20060620 144 62310 959402 A 8.00
20060620 144 62310 959402 B 9.00
20060620 145 62310 959403 A 1.00
20060620 146 62310 959404 B 2.00

I need a awk file or one liner command, that will give me the output as below

20060620 142 62310 959400 A 8.00
20060620 142 62310 959400 B
20060620 143 62310 959401 A 11.00
20060620 143 62310 959401 B
20060620 144 62310 959402 A 17.00
20060620 144 62310 959402 B
20060620 145 62310 959403 A 1.00
20060620 146 62310 959404 B 2.00

Here the field separator is " ". For any two consecutive records, where $2 are same, $6 will be updated by adding rec1.$6+rec2.$6 for $5 = 'A' (first record) where as rec2.$6 will be substited by null( here $5 will be 'B' only, i.e second record). The records will remain unchange if $2 has one record only.
NOTE: For any value of $2, there can be maximum two records.

Thanks in advance,

Regards,
rin..
# 2  
Old 07-04-2006
Quote:
Originally Posted by rinku11
Hi All,
I have a file having content,
$ cat data1.txt
20060620 142 62310 959400 A 5.00
20060620 142 62310 959400 B 3.00
20060620 143 62310 959401 A 7.00
20060620 143 62310 959401 B 4.00
20060620 144 62310 959402 A 8.00
20060620 144 62310 959402 B 9.00
20060620 145 62310 959403 A 1.00
20060620 146 62310 959404 B 2.00

I need a awk file or one liner command, that will give me the output as below

20060620 142 62310 959400 A 8.00
20060620 142 62310 959400 B
20060620 143 62310 959401 A 11.00
20060620 143 62310 959401 B
20060620 144 62310 959402 A 17.00
20060620 144 62310 959402 B
20060620 145 62310 959403 A 1.00
20060620 146 62310 959404 B 2.00

Here the field separator is " ". For any two consecutive records, where $2 are same, $6 will be updated by adding rec1.$6+rec2.$6 for $5 = 'A' (first record) where as rec2.$6 will be substited by null( here $5 will be 'B' only, i.e second record). The records will remain unchange if $2 has one record only.
NOTE: For any value of $2, there can be maximum two records.

Thanks in advance,

Regards,
rin..

This should work

#! /bin/sh
while read line
do
key=`echo $line | awk '{print $2}'`
var=`echo $line | awk '{print $6}'`

awk -v key=${key} -v var=${var} 'key==$2 && var!=$6 {var2=var+$6;print $1,$2,$3,$4,$5,var2}' $1
done<$1>tempfile


sort -k 2,5 tempfile | awk '{if(NR%2!=0) print $0; else print $1,$2,$3,$4,$5}' > outputfile
rm -f tempfile
# 3  
Old 07-04-2006
Thanks Buddy
This gave me the output as
20060620 142 62310 959400 A 8
20060620 142 62310 959400 B
20060620 143 62310 959401 A 11
20060620 143 62310 959401 B
20060620 144 62310 959402 A 17
20060620 144 62310 959402 B

But the last two lines are not here ( the lines without a pair). Can you please help me out in getting my toatal output?
# 4  
Old 07-04-2006
This logic currently only adds only lines with multiple entries in the file. Will need extra logic to add remaining lines. A bit short of time, so may not be able to help you soon.
# 5  
Old 07-04-2006
nawk -f rinku.awk data1.txt

rinku.awk:
Code:
prev == "" { prev = $0; f2=$2; flast=$NF; next }
$2 != f2 { print prev; prev = $0 ; f2=$2; flast=$NF; next}
{
   match(prev, "[^ ][^ ]*$")
   printf("%s%.2f\n", substr(prev, 1, RSTART-1), $NF + flast)
   NF--
   $1=$1
   print; prev = ""
}
END {
  print prev
}

# 6  
Old 07-05-2006
#!/bin/sh

while read line
do
key=`echo $line | awk '{print $2}'`
var=`echo $line | awk '{print $6}'`
if [ `grep -c "$key" $1` -eq 1 ]; then
echo $line >> ofile
fi
awk -v key=${key} -v var=${var} 'key==$2 && var!=$6 {var2=var+$6;print $1,$2,$3,$4,$5,var2}' $1
done<$1>tempfile


sort -k 2,5 tempfile | awk '{if(NR%2!=0) print $0; else print $1,$2,$3,$4,$5}' >> ofile
rm -f tempfile

This one doesn't sorts data currently but shows all the rows. You can use
sort -k2 ofile to get the desired result.
# 7  
Old 07-05-2006
Thanks Dude.
It worked..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk manipulation

hello I have example file AA 11 BB 22 CC 33 And what I expect to have -a AA=11 -a BB=22 -a CC=33 can anyone help how I have this using awk? (1 Reply)
Discussion started by: vikus
1 Replies

2. Shell Programming and Scripting

awk manipulation

Hallo Family, I have csv file which has over a million records in it. All i want to do is to change field 2 to have the same value as field 10. sample file:Now 0860093239,Anonymous,unconditional,+27381230283,Anonymous,unconditional,y,public,,2965511477:0A Desired output: ... (2 Replies)
Discussion started by: kekanap
2 Replies

3. Shell Programming and Scripting

Awk to convert a text file to CSV file with some string manipulation

Hi , I have a simple text file with contents as below: 12345678900 971,76 4234560890 22345678900 5971,72 5234560990 32345678900 71,12 6234560190 the new csv-file should be like: Column1;Column2;Column3;Column4;Column5 123456;78900;971,76;423456;0890... (9 Replies)
Discussion started by: FreddyDaKing
9 Replies

4. UNIX for Dummies Questions & Answers

File manipulation via awk

Hello, I am having issues generating the output file below from this input file: Basically, what I want is if the ID= matches with the line below to print the first value in column 3 and the last value of column 4 for the matching ID's. The ID's can repeat more than twice, however, they... (2 Replies)
Discussion started by: verse123
2 Replies

5. Shell Programming and Scripting

File manipulation in awk

I have got a sample file below(colon(:) is the field separator) . The data is like col1:col2:col3:col4:col5:col6:col7:col8:col9:col10 11:12:012:aa:a a a:10::111:12: 311:321:320:caad::321:31:3333:: 2:22:222::bbb::cads::2222:20 :::::12:1234::12: :5:55::555:5555::::55550 Now I want to find... (9 Replies)
Discussion started by: rinku11
9 Replies

6. Shell Programming and Scripting

SED/AWK file read & manipulation

I have large number of data files, close to 300 files, lets say all files are same kind and have extension .dat , each file have mulitple lines in it. There is a unique line in each file containing string 'SERVER'. Right after this line there is another line which contain a string 'DIGIT=0',... (4 Replies)
Discussion started by: sal_tx
4 Replies

7. Shell Programming and Scripting

File manipulation with AWK and SED

Hello How do i check that correct input files are used while using AWk and SED for file manipulation? e.g awk '/bin/ {print $0 }' shell.txt sed 's/hp/samsung/' printers.txt how do i ensure that the correct input files I am working with are used? (5 Replies)
Discussion started by: Pauline mugisha
5 Replies

8. Shell Programming and Scripting

$0 manipulation in awk

OK, so if $0 represent the entire record... can I change $2 and will that be reflected back in $0? I think the following answers that YES, it does work. But is there anything I should be thinking about prior to doing this? What I am actually doing is part of 5 pages of scripting and awk... (1 Reply)
Discussion started by: joeyg
1 Replies

9. Shell Programming and Scripting

File manipulation with awk

Could you please help me to achieve the below: In a file I need to convert the multiple lines whose filed 1 and field 5 values are same into a single line but with the field 4 values comma separed as mentioned below. Fileds after 5 shall be discarded. Also here by default all other remaining... (6 Replies)
Discussion started by: dhams
6 Replies

10. Shell Programming and Scripting

awk manipulation

Hi , what a wonderful command but so hard to maintain ! i have a file like that : 03/07/2006 05:58:45 03/07/2006 06:58:45 03/07/2006 07:58:50 03/07/2006 08:58:50 and i want to read it and keep only the lines with 3rd field less than 07:00:00 writing it in a second file ! ... (2 Replies)
Discussion started by: Nicol
2 Replies
Login or Register to Ask a Question