How to merge two files with unique values matching.?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to merge two files with unique values matching.?
# 1  
Old 11-06-2014
How to merge two files with unique values matching.?

I have one script as below:
Code:
#!/bin/ksh
Outputfile1="/home/OutputFile1.xls"
Outputfile2="/home/OutputFile2.xls"
InputFile1="/home/InputFile1.sql"
InputFile2="/home/InputFile2.sql"
echo "Select hobby, class, subject, sports, rollNumber from Student_Table" >> InputFile1
echo "Select rollNumber ,First_name, Last_name from Name_Table" >>InputFile2
isql -Uusername -Ppassword -Ddatabase -Sserver_name -iInputFile1 -oOutputfile1
isql -Uusername -Ppassword -Ddatabase -Sserver_name -iInputFile2 -oOutputfile2

So after running the script i have two output files OutputFile1 and OutputFile2,one feild is common between both the files which is rollNumber

OutputFile1.xls:
Code:
hobby  class  subject  sports  rollNumber
dance  11     science  cricket   1
dance  12     science  cricket   3
dance  10     science  cricket   1
dance  11     science  cricket   2
dance  11     science  cricket   1
dance  11     science  cricket   2

OutputFile2.xls:
Code:
rollNumber First_name          last_name
1               sonali           kumari
2               anni             Das
3               Rini             kumari

Now i want a third file which merges both the OutputFile1 and OutputFile2 with the unique rollNumbers and gets the below output:
Code:
hobby  class  subject  sports  rollNumber   First_name        last_name
dance  11     science  cricket   1              sonali        kumari
dance  12     science  cricket   3              Rini          kumari        
dance  10     science  cricket   1              sonali        kumari
dance  11     science  cricket   2              anni          Das
dance  11     science  cricket   1              sonali        kumari
dance  11     science  cricket   2              anni          Das

So it matches the second file with the first file and finds the unique number and merges the first_name and last_name with the appeopriate roll number.
# 2  
Old 11-06-2014
Hello Sharma331,

Following may help you in same.

Code:
awk 'FNR==NR{A[$1]=$2 OFS $3;next} ($5 in A){print $0 OFS A[$5]}' outputfile2.xls OFS="\t" outputfile1.xls

Output will be as follows.
Code:
hobby  class  subject  sports  rollNumber       First_name last_name
dance  11     science  cricket   1      sonali kumari
dance  12     science  cricket   3      Rini kumari
dance  10     science  cricket   1      sonali kumari
dance  11     science  cricket   2      anni Das
dance  11     science  cricket   1      sonali kumari
dance  11     science  cricket   2      anni Das


Thanks,
R. Singh
# 3  
Old 11-06-2014
Merging the two output files on the shell level should not be considered the smartest approach as SQL has the means - and is designed to do exactly that - to do the merge immediately in a query. Try like (my SQL has become somewhat rusty, so there may be smarter queries):
Code:
Select hobby, class, subject, sports, S.rollNumber, First_name, Last_name 
from Student_Table S, Name_Table N where S.rollnumber=N.rollnumber

# 4  
Old 11-06-2014
Hi Rudi,

Actually the query which i provided was dummy query in my actual query i am using the output of the first query in to the second one. The exact query is given below:
Code:
select substr(OBJECT_NAME,3,9) as NUMBER, ID,
CREATION_DATE, CONTENT_TYPE, CREATOR_NAME from DOCUMENT where 
CONTENT_TYPE in ('mp3','ra', 'wma', 'wav', 'wave') and 
CREATION_DATE between 
'2014-05-01 00:00:00.0' and '2014-05-31 23:59:59.0'


select pin_num, frst_nme, last_nme from rsrc_t where pin_num in (pin number list from above query which is CREATOR_NAME )

Note:- both the queries are done from two different servers as well as DBs (one being DB2 and one Sybase)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge 4 bim files by keeping only the overlapping variants (unique rs values )

Dear community, I am facing a problem and I kindly ask your help: I have 4 different data sets consisted from 3 different types of array. On each file, column 1 is chromosome position, column 2 is SNP id etc... Lets say I have the following (bim) datasets: x2014: 1 rs3094315... (4 Replies)
Discussion started by: fondan
4 Replies

2. Linux

Merge two files based on matching criteria

Hi, I am trying to merge two csv files based on matching criteria: File description is as below : Key_File : 000|ÇÞ|Key_HF|ÇÞ|Key_FName 001|ÇÞ|Key_11|ÇÞ|Sort_Key22|ÇÞ|Key_31 002|ÇÞ|Key_12|ÇÞ|Sort_Key23|ÇÞ|Key_32 003|ÇÞ|Key_13|ÇÞ|Sort_Key24|ÇÞ|Key_33 050|ÇÞ|Key_15|ÇÞ|Sort_Key25|ÇÞ|Key_34... (3 Replies)
Discussion started by: PK29
3 Replies

3. Shell Programming and Scripting

Count Unique values from multiple lists of files

Looking for a little help here. I have 1000's of text files within a multiple folders. YYYY/ /MM /1000's Files Eg. 2014/01/1000 files 2014/02/1237 files 2014/03/1400 files There are folders for each year and each month, and within each monthly folder there are... (4 Replies)
Discussion started by: whegra
4 Replies

4. Shell Programming and Scripting

How to merge two or more fields from two different files where there is non matching column?

Hi, Please excuse for often requesting queries and making R&D, I am trying to work out a possibility where i have two files field separated by pipe and another file containing only one field where there is no matching columns, Could you please advise how to merge two files. $more... (3 Replies)
Discussion started by: karthikram
3 Replies

5. UNIX for Dummies Questions & Answers

Grep to find matching patern and return unique values

Request: grep to find given matching patern and return unique values, eliminate the duplicate values I have to retrieve the unique folder on the below file contents like; /app/oracle/build_lib/pkg320.0_20120927 /app/oracle/build_lib/pkg320.0_20121004_prof... (5 Replies)
Discussion started by: Siva SQL
5 Replies

6. Shell Programming and Scripting

Merge two files matching columns

Hi! I need to merge two files when col1 (x:x:x) matching and adds second column from file1.txt. # cat 1.txt aaa;a12 bbb;b13 ccc;c33 ddd;d55 eee;e11 # cat 2.txt bbb;b55;34444;d55 aaa;a15;35666;a44 I try with this awk and I get succesfully first column from 1.txt: # awk -F";"... (2 Replies)
Discussion started by: fhluque
2 Replies

7. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

8. Shell Programming and Scripting

merge files with same row values

Hi everyone, I'm just wondering how could I using awk language merge two files by comparison of one their row. I mean, I have one file like this: file#1: 21/07/2009 11:45:00 100.0000000 27.2727280 21/07/2009 11:50:00 75.9856644 25.2492676 21/07/2009 11:55:00 51.9713287 23.2258072... (4 Replies)
Discussion started by: tonet
4 Replies

9. Shell Programming and Scripting

comparing 2 text files to get unique values??

Hi all, I have got a problem while comparing 2 text files and the result should contains the unique values(Non repeatable). For eg: file1.txt 1 2 3 4 file2.txt 2 3 So after comaping the above 2 files I should get only 1 and 4 as the output. Pls help me out. (7 Replies)
Discussion started by: smarty86
7 Replies

10. UNIX for Dummies Questions & Answers

Need to find only unique values for a given tag across the files

Need to find only unique values for a given tag across the files: For eg: Test1: <Tag1>aaa</Tag1> <Tag2>bbb</Tag2> <Tag3>ccc</Tag3> Test2: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test3: <Tag1>aaa</Tag1> <Tag2>ddd</Tag2> <Tag3>eee</Tag3> Test4: (8 Replies)
Discussion started by: sudheshnaiyer
8 Replies
Login or Register to Ask a Question