Sponsored Content
Top Forums Shell Programming and Scripting Help to retrieve data from two files matching a string Post 302558278 by hartz on Friday 23rd of September 2011 08:05:45 AM
Old 09-23-2011
join the files

I think join is the easiest (and most elegant) solution for this. Join requires that the files be sorted by the joining column, so ... some pre-work.

Code:
#!/bin/sh
sort input > sorted.input
sort -t, -k6 file1 > sorted.file1
tr = , <file2 | sort -t, -k6 > sorted.file2
join -t, -o 1.1,2.2,2.5 -1 1 -2 6 sorted.input sorted.file1 | sort -t, -k2 > sorted.temp1
echo "Number\tData1\tData2"
join -t, -o 1.1,1.2,2.2 -1 2 -2 6 sorted.temp1 sorted.file2 | sed 's/ //g; s/,/\t/g'

The question is: how do you want to handle lines which do not match? You can handle this in different ways, depending on whether there is a line in "input" with no line in file1, and then again the other way round if there is no line in input but some lines in file1 or file2. If you are sure there will always be pairs, then my work here is done.

But you have already hinted at lines that wasn't atched, so have a look at the options for -e as well as -a 1 and/or -a 2 in the man page of the join command to see what is possible. It is called UNPAIRABLE lines.

If the options confuse you, explain what you need and someone will surely help.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Retrieve data from a file

Hello guys I want to retrieve two data from a file, like this: bash-2.03$ cat numtest 123456 123457 bash-2.03$ more ./test_num #!/bin/bash num1= num2= cnt=1 while read x do num${cnt}=$x cnt=$(($cnt+1)) done <$1 echo $num1 "\n" $num2 But when i executed this script, error... (2 Replies)
Discussion started by: tpltp
2 Replies

2. Shell Programming and Scripting

matching string in two files of different length

Dear all, I have the following problem (it originates in the domain of bio-inf, but it is a general problem). I have two files of one column each and of different length: a.txt and b.txt. a.txt contains alphanumeric strings (around 30 digit) and there are 300 rows b.txt contains alphanumeric... (2 Replies)
Discussion started by: ad_meis
2 Replies

3. Programming

How to find the matching data b/w 2 files in perl?

Hi friends,, i have find the matching data between 2files. My file1 have a data like rs3001336 rs3984736 rs2840532 File2 have a data like rs3736330 1 2359237 A G 0.28 1.099 0.010 rs2840532 1 2359977 G A 0.363 0.3373 1.123 rs3001336 1 2365193 G A 0.0812 0.07319 1.12 ... (1 Reply)
Discussion started by: sureshraj
1 Replies

4. Shell Programming and Scripting

How to find the matching data b/w 2 files in perl?

Hi friends,, i have find the matching data between 2files. My file1 have a data like rs3001336 rs3984736 rs2840532 File2 have a data like rs3736330 1 2359237 A G 0.28 1.099 0.010 rs2840532 1 2359977 G A 0.363 0.3373 1.123 rs3001336 1 ... (4 Replies)
Discussion started by: sureshraj
4 Replies

5. Shell Programming and Scripting

search string in a file and retrieve 10 lines including string line

Hi Guys, I am trying to write a perl script to search a string "Name" in the file "FILE" and also want to create a new file and push the searched string Name line along with 10 lines following the same. can anyone of you please let me know how to go about it ? (8 Replies)
Discussion started by: sukrish
8 Replies

6. Shell Programming and Scripting

Matching string on two files based on match rules.

Hi, How to check if a string on file2 exactly matches with a part or complete string on file1, and return a match indicator based on some match rules. 1) only records on file1 with category A should be matched. for other category, the output match indicator should default to 'N' 2) on file2... (13 Replies)
Discussion started by: effay
13 Replies

7. Shell Programming and Scripting

Compare two files and get matching data

Hi, Can anyone help me to compare two files and get the matching data... say i have file1 and file2 ... file1 has 300 unique data with that i need to match with file2 to see how may are matching.. file2 have 1000 records. (4 Replies)
Discussion started by: zooby
4 Replies

8. Shell Programming and Scripting

Compare columns of two files and retrieve data

Hi guys, I need your help. I have two files: file1 1 3 5 file2 1,XX 2,AA 3,BB 4,CC 5,DD I would like to compare the first column and where they are equal to write that output in a new file: 1,XX 3,BB (7 Replies)
Discussion started by: apenkov
7 Replies

9. Shell Programming and Scripting

How can I retrieve the matching records from data file mentioned?

XYZNA0000778800Z 16123000012300321000000008000000000000000 16124000012300322000000007000000000000000 17234000012300323000000005000000000000000 17345000012300324000000004000000000000000 17456000012300325000000003000000000000000 9 XYZNA0000778900Z 16123000012300321000000008000000000000000... (8 Replies)
Discussion started by: later_troy
8 Replies

10. Shell Programming and Scripting

Field matching in two data files

Hello, I am looking to output all of the lines from file2 whose 11th field is present in the first field in file1. Then the second field from file1 should be appended as such: file1: 2222 0.35 4444 0.25 5555 0.75 file2: col1 col2 col3 col4 col5 col6 col7 col8 col9 col10 1111 col1 col2... (4 Replies)
Discussion started by: palex
4 Replies
JOIN(1) 						      General Commands Manual							   JOIN(1)

NAME
join - relational database operator SYNOPSIS
join [ options ] file1 file2 DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard input is used. File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in each line. There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con- sists of the common field, then the rest of the line from file1, then the rest of the line from file2. Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis- carded. These options are recognized: -an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2. -e s Replace empty output fields by string s. -jn m Join on the mth field of file n. If n is missing, use the mth field in each file. -o list Each output line comprises the fields specifed in list, each element of which has the form n.m, where n is a file number and m is a field number. -tc Use character c as a separator (tab character). Every appearance of c in a line is significant. SEE ALSO
sort(1), comm(1), awk(1) BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort. The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous. JOIN(1)
All times are GMT -4. The time now is 04:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy