Shell Programming and Scripting

Hi all,

I need some help to remove duplicates from a file before merging.

I have got 2 files:

file1 has data in format
4300 23456
4301 2357

the 4 byte values on the right hand side is uniq, and are not repeated anywhere in the file

file 2 has data in same format but is not in sorted order

3999 2349
4305 1234
4300 3459

the script i am using is not able to remove duplicate values after spaces, and the output is coming in the following manner:-

3999 2349
4300 233445569
4301 2357
4305 1234

The expected output is:-
3999 2349
4300 234569
4301 2357
4305 1234

The script i am using is:-

Code:

export INPUT1=File1
export REQ=File2
export OUTPUT1=Final_File


sort $INPUT1 $REQ |
           awk     '$1 == SV1              {SV2=SV2  $2;next}
           NR>1 && $1 != SV1                {print SV1, SV2}
                                            {SV1=$1;SV2=$2}
           END                             {print  SV1,SV2}
        ' > $OUTPUT1

I don't see how the 'expected' is different from 'produced'...
Could you provide a better (more representative) set of sample files to explain what you're after.....

For the value 4300

produced value is:
4300 233445569

expected value is:-
4300 234569

You mean you want to change 233445569 to this 234569? and not the records but the repeated values in the fields?

yes....thats correct....

We need to see your script. How is the value from first file 4300 23456 and second file 4300 3459 ending up as 4300 233445569?

The script is given in the 1st post.......................