Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Strange situation of file sorting and merging

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Strange situation of file sorting and merging
# 8  
Old 08-09-2012
thank you all for all your suggestions .
I'm gonna try it. hopefully it's going to work, fingers are crossed...

can I make a combination columns like
Code:
$5 and $6

not
Code:
$6

from the file
# 9  
Old 08-09-2012
yes it can be done with combination also
This User Gave Thanks to raj_saini20 For This Post:
manas_ranjan
# 10  
Old 08-09-2012
Quote:
Originally Posted by raj_saini20
yes it can be done with combination also
thanks
the one which you provided works fine or rather as charm if both files have different counts.

But it's doesn't work, if both y'day and today file have same count??

can you help me.
I made little change can you please check what I'm missing for same count.

Code:
awk -F"," 'BEGIN{OFS=","}
         NR==FNR{a[$5 FS $6]=$0 } NR!=FNR{
        if(a[$5 FS $6]){split(a[$5 FS $6],b);
        if(!c[$5 FS $6])
        {       c[$5 FS $6]=1;
        print $0,b[4],b[7],b[8],b[9]
        } else {
        print $0,b[4],"NA","NA","NA" }
        } else {
        print $0,"NA","NA","NA","NA"
        }}' yday.csv today.csv

today.csv
Code:
36000807	A	123 	78	0	1	0.1	 0.2 	0.3
36000807	A	123 	78	0	5	0.1	 0.2 	0.3
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3

yday.csv
Code:
36000807	A	123 	76	0	1	0.56	 0.47 	0.39
36000807	A	123 	76	0	5	-0.34 	 0.27 	-0.38
36000807	A	123 	76	0	10	-0.14 	 0.25 	-0.53

final.csv
Code:
f1.1		f1.2	f1.3	f1.4	f1.5	f1.6	f1.7	f1.8	f1.9	f2.4	f2.7	f2.8	f2.9
36000807	A	123 	78	0	1	0.1	 0.2 	0.3	76	0.56	 0.47 	0.39
36000807	A	123 	78	0	5	0.1	 0.2 	0.3	76	-0.34 	 0.27 	-0.38
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3	76	-0.14 	 0.25 	-0.53
--------------------------today.csv----------------------------------------****-----------yday.csv------------

# 11  
Old 08-09-2012
Any comments on / errors with my proposal? Works fine on my machine...
Last edited by RudiC; 08-09-2012 at 10:17 AM.. Reason: mod'd wording
# 12  
Old 08-09-2012
yes, Rudi I have tried your both option taking both file as same count and different count .

in different record count it works fine as expected
but when both of record counts are same then it kind of messed up.

I think some where in the same count, we need little tweak keeping the logic for different record count untouched.
# 13  
Old 08-09-2012
OK, then be little more specific. yday was half the lines as today in your example. This should not be assumed, I understand; so what be the number of lines in yday? Anything between today and today/2? Or today and zero? And what to do if yday has the same line count for e.g. 3 records and only half the line count starting at record 4? Is it possible yday has more lines for certain records?
# 14  
Old 08-10-2012
Hi Rudi,

That's exactly I'm trying to say.

sometimes it could be possible that

yday.csv's record count is exactly equals to today.csv
the file I used as an example have same record count 3 and 3
it's purely coincidence that my example of yday and today has 3 records which is exactly half of not same record count file.
There could be possible that yday and today file counts could be 4 and 4.
This is the scenario for same record count.
and for not same record count it could be 6 and 2 or could be 6 and 3 .

please refer example case one and two are example of not same record count
case one
today.csv 6 count


Code:
36000807	A	123 	78	0	1	0.1	 0.2 	0.3
36000807	A	123 	79	0	1	-0.1 	 0.2 	-0.3
36000807	A	123 	78	0	5	0.1	 0.2 	0.3
36000807	A	123 	79	0	5	-0.1 	 0.2 	-0.3
36000807	A	123 	78	0	10	0.1	 0.2 	0.3
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3

yday.csv - 3 record count

Code:
36000807	A	123 	76	0	1	0.56	 0.47 	0.39
36000807	A	123 	76	0	5	-0.34 	 0.27 	-0.38
36000807	A	123 	76	0	10	-0.14 	 0.25 	-0.53

merged file which matched column 5 and 6 of both file and print exactly 1st row with the value and 2nd row with NA NA

Code:
f1.1		f1.2	f1.3	f1.4	f1.5	f1.6	f1.7	f1.8	f1.9	f2.4	f2.7	f2.8	f2.9
36000807	A	123 	78	0	1	0.1	 0.2 	0.3	76	0.56	 0.47 	0.39
36000807	A	123 	79	0	1	-0.1 	 0.2 	-0.3	76	NA	NA	NA
36000807	A	123 	78	0	5	0.1	 0.2 	0.3	76	-0.34 	 0.27 	-0.38
36000807	A	123 	79	0	5	-0.1 	 0.2 	-0.3	76	NA	NA	NA
36000807	A	123 	78	0	10	0.1	 0.2 	0.3	76	-0.14 	 0.25 	-0.53
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3	76	NA	NA	NA
--------------------------today.csv------------------------------------------****-----------yday.csv------------

option two
today.csv 6 reocrd count
Code:
36000807	A	123 	78	0	1	0.1	 0.2 	0.3
36000807	A	123 	79	0	1	-0.1 	 0.2 	-0.3
36000807	A	123 	78	0	5	0.1	 0.2 	0.3
36000807	A	123 	79	0	5	-0.1 	 0.2 	-0.3
36000807	A	123 	78	0	10	0.1	 0.2 	0.3
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3

yday.csv - 2 record count missing column 5 and 6 (0 and 1) values
Code:
36000807	A	123 	76	0	5	-0.34 	 0.27 	-0.38
36000807	A	123 	76	0	10	-0.14 	 0.25 	-0.53

final merged file have 6 rows and since yday column 5 and 6 missed it's values
so f2.7 f2.8 f2.9 have values NA NA NA
Code:
f1.1		f1.2	f1.3	f1.4	f1.5	f1.6	f1.7	f1.8	f1.9	f2.4	f2.7	f2.8	f2.9
36000807	A	123 	78	0	1	0.1	 0.2 	0.3	76	NA	NA	NA
36000807	A	123 	79	0	1	-0.1 	 0.2 	-0.3	76	NA	NA	NA
36000807	A	123 	78	0	5	0.1	 0.2 	0.3	76	-0.34 	 0.27 	-0.38
36000807	A	123 	79	0	5	-0.1 	 0.2 	-0.3	76	NA	NA	NA
36000807	A	123 	78	0	10	0.1	 0.2 	0.3	76	-0.14 	 0.25 	-0.53
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3	76	NA	NA	NA
--------------------------today.csv-----------------------------------------***-----------yday.csv------------

and incase of same record count , both files have same record count and each column 5 and 6 have corresponding value for coulmn 5 and 6 in other file
today.csv
Code:
36000807	A	123 	78	0	1	0.1	 0.2 	0.3
36000807	A	123 	78	0	5	0.1	 0.2 	0.3
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3

yday.csv
Code:
36000807	A	123 	76	0	1	0.56	 0.47 	0.39
36000807	A	123 	76	0	5	-0.34 	 0.27 	-0.38
36000807	A	123 	76	0	10	-0.14 	 0.25 	-0.53

and final mergerd file
Code:
f1.1		f1.2	f1.3	f1.4	f1.5	f1.6	f1.7	f1.8	f1.9	f2.4	f2.7	f2.8	f2.9
36000807	A	123 	78	0	1	0.1	 0.2 	0.3	76	0.56	 0.47 	0.39
36000807	A	123 	78	0	5	0.1	 0.2 	0.3	76	-0.34 	 0.27 	-0.38
36000807	A	123 	79	0	10	-0.1 	 0.2 	-0.3	76	-0.14 	 0.25 	-0.53
--------------------------today.csv----------------------------------------****-----------yday.csv------------

f1=today.csv
f2=yday.csv
f1.X today.csv's X column number
f2.X yday.csv's X column number
Last edited by manas_ranjan; 08-10-2012 at 05:16 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging and sorting files

I have the following files: file A Col1 Col2 A 1 B 2 C 3 D 4 file B Col1 Col2 A 1 Aa 1 B 2 C 3 D 4 file C Col1 Col2 A 1 (1 Reply)
Discussion started by: ramky79
1 Replies

2. Shell Programming and Scripting

merging two file

Dear All, I have two file like this: file1: a1234 b1235 c4678 d7859 file2 : e4575 f7869 g7689 h9687 I want output like this: a1234 b1235 c4678 (2 Replies)
Discussion started by: attila
2 Replies

3. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

4. UNIX for Dummies Questions & Answers

Sorting and merging files.

Hi I’m new to scripting and have only had about two days experience with this. I have questions about a bash/gawk script. Problem: I have 27 files, which needs to get merged into one, the files are separated into 8 subdivisions containing a 3 row data description. Example of data File.1 ... (7 Replies)
Discussion started by: Bateman1001
7 Replies

5. Programming

Help in sorting and merging lists

Hi everyone, need your help in sorting and merging two numerical lists Example: I have one list 1 2 3 4 5 7 and the other 4 6 8, then the final output should be 1 2 3 4 5 6 7 8 Requesting your kind help in this Regards, RB :) (1 Reply)
Discussion started by: ramakanth_burra
1 Replies

6. Shell Programming and Scripting

Extracting a column from a file and merging with other file using awk

Hi All: I have following files: File 1: <header> text... text .. text .. text .. <\header> x y z ... File 2: <header> text... text .. text .. (4 Replies)
Discussion started by: mrn006
4 Replies

7. UNIX for Dummies Questions & Answers

merging 2 file

I have 2 files file1.txt a 123 aqsw c 234 sfdr fil2.txt b 345 hgy d 4653 jgut I want to merger in such a manner the the output file should be outfile.txt a 123 aqsw b 345 hgy c 234 sfdr d 4653 jgut Do we have any command to achive this? (8 Replies)
Discussion started by: siba.s.nayak
8 Replies

8. Shell Programming and Scripting

How to Sort a file for given situation?

Hi All, How can you sort a file that is doubled space ( where even number lines are blank lines) and still preserves the blank lines? You can use grep,sed and regular expression. Thanks Vishal (4 Replies)
Discussion started by: vishalpatel03
4 Replies

9. Programming

strange situation in file

Hi All, I am writing some data's into a file from C++ program. The files which i am writing is of fixed length . say 232 in length per line. I am writing as . my c code is as ... (0 Replies)
Discussion started by: arunkumar_mca
0 Replies

10. UNIX for Dummies Questions & Answers

strange situation with nslookup on Linux

Hey, I have a problem with nslookup under the newly installed mandrake 9.1, and as I see now also under Redhat 8.0 I have a pc called evo with the ip 10.0.0.1 which entered correctly in the /etc/hosts file. Connection to the internet is via an adsl-router. I have the nameservers from my... (2 Replies)
Discussion started by: mod
2 Replies
Login or Register to Ask a Question