Shell Programming and Scripting

Hi All,

i have a zip file like the format

794051400123|COM|24|0|BD|R|99.98

794051413727|COM|11|0|BD|R|28.99

794051415622|COM|23|0|BD|R|28.99

883929004676|COM|0|0|BD|R|28.99
794051400123|MOM|62|0|BD|R|99.98

794051413727|MOM|4|0|BD|R|28.99

794051415622|MOM|80|0|BD|R|28.99

883929004676|MOM|0|0|BD|R|28.99

883929017164|MOM|0|0|BD|R|39.99
794051400123|RNO|73|0|BD|R|99.98

794051413727|RNO|8|0|BD|R|28.99

794051415622|RNO|84|0|BD|R|28.99

883929004676|RNO|0|0|BD|R|28.99
794051400123|SOM|25|0|BD|R|99.98

794051415622|SOM|80|0|BD|R|28.99

883929004676|SOM|0|0|BD|R|28.99

883929017164|SOM|0|0|BD|R|39.99
.................................

i need concate all duplicate line like

794051400123|COM|24|MOM|62|SOM|25|RNO|73
794051413727|COM|11||MOM|4|RNO|8
............
...

the file size is nearly 30 MB.So it takes lot of time.I have to do it with in 15 min.

please help me.

Thanks!
vaskar

What happened to the dollar amount at the end of the line?

Tested and working.

Code:

#!/usr/bin/env python

import sys

input = open(sys.argv[1], 'r')

dict = {}

for line in input:

    line = line.rstrip()

    if line.count("|"):
        line = line.split("|")
        if dict.has_key(line[0]):
            dict[line[0]].extend(line[1:3])
        else:
            dict[line[0]] = line[1:3]
            
input.close()

for key, value in dict.items():

    print("%s|%s" % (key, "|".join(value)))

I don't know python script.Is it possible write simple shell script? I have done it but it takes lot of time.

Gift horse and all that ...

Assuming you mean take all values with an identical first field, and paste together fields 2 and 3 from all those lines, something like

Code:

sort file |
awk -F '|' '$1 == prev { collect=collect "|" $2 "|" $3; next }
{ if (collect) print collect; prev = $1; collect = $1 "|" $2 "|" $3; next }
END { if (collect) print collect }'

The sort might kill you if it's a very small or old system, but having sorted input makes the awk script very simple.

Thanks!! It is working....

Hi,

I am facing a big problem...It gives me wrong result

like

|COM|24|MOM|62|SOM|25|RNO|73
794051413727|COM|11||MOM|4|RNO|8

pls help me.

Thanks!
vaskar