Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Strange situation of file sorting and merging

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Strange situation of file sorting and merging
# 15  
Old 08-11-2012
This proposal hopefully will fulfill all your requirements, file1 being larger, smaller than, or identical to file2. It will compare on field 6 only; you may extend it to comparing field 5 by adding ai[5] and bi[5] to all occurrences of ai[6] and bi[6]. But then, expect bizarre results! Here we go:
Code:
awk 'BEGIN {fmt="%10.10s"; aOK=bOK=needa=needb=1;}
        { while (aOK || bOK)
                {if (needa) aOK=getline a<filea;
                 if (needb) bOK=getline b<fileb;
                 if (!(aOK||bOK)) exit;

                 if (aOK) {split(a,ai)} else {ai[6]="NA"};
                 if (bOK) {split(b,bi)} else {bi[6]="NA"};

                 if(ai[6]==bi[6])       {needa=needb=1}
                 else   if(ai[6]>bi[6])
                                        {needa=0; needb=1; ai[4]=ai[5]=ai[6]=ai[7]=ai[8]=ai[9]="NA";}
                        else            {needa=1; needb=0; bi[4]=bi[5]=bi[6]=bi[7]=bi[8]=bi[9]="NA";}
                for (i=1;i<=9;i++) printf fmt, ai[i]; for(i=4;i<=9;i++) printf fmt, bi[i]; print "";
                }
         exit
        }
        ' FS=" " filea=today fileb=yday today

Please adapt the format string fmt in the BEGIN part and the field separator according to your requirements. Give it a thorough test and come back with the results!
It still may require some polishing, e.g. on the repeated assignment of "NA" to the single fields, but I've run out of ideas here...
Last edited by RudiC; 08-11-2012 at 08:59 AM..
# 16  
Old 08-13-2012
Code:
 awk 'BEGIN{OFS="\t"}
 NR==FNR{
	a[$6]=$0
	}
 NR!=FNR{
	if(a[$6])
		{	
			split(a[$6],b);
			if(!c[$6])
				{
					c[$6]=1;
					print $0,b[4],b[7],b[8],b[9]
				}
			else
				{
					print $0,b[4],"NA","NA","NA"
				}
		}
	else
		{
			print $0,"NA","NA","NA","NA"
		}
	}' YEST.CSV TODAY.CSV

This is working for all scenario. For the same row count output is
Code:
36000807        A       123     78      0       1       0.1      0.2    0.3     76      0.56    0.47    0.39
36000807        A       123     78      0       5       0.1      0.2    0.3     76      -0.34   0.27    -0.38
36000807        A       123     79      0       10      -0.1     0.2    -0.3    76      -0.14   0.25    -0.53

which is as desired by you
# 17  
Old 08-15-2012
Here's the polished version of my above proposal. Pls. give it a try and comment:
Code:
awk -F, 'BEGIN  {OFS=FS;fmt="%s"OFS;
                 aOK=bOK=needa=needb=1;
                 na[4]=na[5]=na[6]=na[7]=na[8]=na[9]="NA";
                 filea=ARGV[1];
                 fileb=ARGV[2];
                }

         function prt (A, B, C) {for (i=1;i<=3;i++) printf fmt, A[i];
                                 for (i=4;i<=9;i++) printf fmt, B[i];
                                 for (i=4;i<=9;i++) printf fmt, C[i];
                                 print "";}

        BEGIN \
         {while (aOK||bOK)
                {if (needa) aOK=getline a<filea;
                 if (needb) bOK=getline b<fileb;

                 if (!(aOK||bOK)) exit;

                 if (aOK) {split(a,ai)} else {ai[6]="NA"};
                 if (bOK) {split(b,bi)} else {bi[6]="NA"};

                 if(ai[6]==bi[6])               {needa=1; needb=1; prt (ai, ai, bi);}
                 else   if(ai[6]>bi[6])         {needa=0; needb=1; prt (ai, na, bi);}
                        else                    {needa=1; needb=0; prt (ai, ai, na);}
                }
         }' today.csv yday.csv

This User Gave Thanks to RudiC For This Post:
manas_ranjan
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging and sorting files

I have the following files: file A Col1 Col2 A 1 B 2 C 3 D 4 file B Col1 Col2 A 1 Aa 1 B 2 C 3 D 4 file C Col1 Col2 A 1 (1 Reply)
Discussion started by: ramky79
1 Replies

2. Shell Programming and Scripting

merging two file

Dear All, I have two file like this: file1: a1234 b1235 c4678 d7859 file2 : e4575 f7869 g7689 h9687 I want output like this: a1234 b1235 c4678 (2 Replies)
Discussion started by: attila
2 Replies

3. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

4. UNIX for Dummies Questions & Answers

Sorting and merging files.

Hi I’m new to scripting and have only had about two days experience with this. I have questions about a bash/gawk script. Problem: I have 27 files, which needs to get merged into one, the files are separated into 8 subdivisions containing a 3 row data description. Example of data File.1 ... (7 Replies)
Discussion started by: Bateman1001
7 Replies

5. Programming

Help in sorting and merging lists

Hi everyone, need your help in sorting and merging two numerical lists Example: I have one list 1 2 3 4 5 7 and the other 4 6 8, then the final output should be 1 2 3 4 5 6 7 8 Requesting your kind help in this Regards, RB :) (1 Reply)
Discussion started by: ramakanth_burra
1 Replies

6. Shell Programming and Scripting

Extracting a column from a file and merging with other file using awk

Hi All: I have following files: File 1: <header> text... text .. text .. text .. <\header> x y z ... File 2: <header> text... text .. text .. (4 Replies)
Discussion started by: mrn006
4 Replies

7. UNIX for Dummies Questions & Answers

merging 2 file

I have 2 files file1.txt a 123 aqsw c 234 sfdr fil2.txt b 345 hgy d 4653 jgut I want to merger in such a manner the the output file should be outfile.txt a 123 aqsw b 345 hgy c 234 sfdr d 4653 jgut Do we have any command to achive this? (8 Replies)
Discussion started by: siba.s.nayak
8 Replies

8. Shell Programming and Scripting

How to Sort a file for given situation?

Hi All, How can you sort a file that is doubled space ( where even number lines are blank lines) and still preserves the blank lines? You can use grep,sed and regular expression. Thanks Vishal (4 Replies)
Discussion started by: vishalpatel03
4 Replies

9. Programming

strange situation in file

Hi All, I am writing some data's into a file from C++ program. The files which i am writing is of fixed length . say 232 in length per line. I am writing as . my c code is as ... (0 Replies)
Discussion started by: arunkumar_mca
0 Replies

10. UNIX for Dummies Questions & Answers

strange situation with nslookup on Linux

Hey, I have a problem with nslookup under the newly installed mandrake 9.1, and as I see now also under Redhat 8.0 I have a pc called evo with the ip 10.0.0.1 which entered correctly in the /etc/hosts file. Connection to the internet is via an adsl-router. I have the nameservers from my... (2 Replies)
Discussion started by: mod
2 Replies
Login or Register to Ask a Question