Merge two file data together based on specific pattern match


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge two file data together based on specific pattern match
# 1  
Old 11-19-2009
Merge two file data together based on specific pattern match

My input:
File_1:
2000_t
g1110.b1
abb.1
2001_t
g1111.b1
abb.2
abb.2
g1112.b1
abb.3
2002_t
.
.

File_2:
2000_t Ali england 135
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
abb.2
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
.
.

My desired output file:
2000_t Ali england 135
g1110.b1
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
abb.2
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
2002_t
.
.

My main purpose is merge both file_1 and file_2 data together. All the file_1 data must be in output file. File_2 data will be append to the output file based on those same data of column 1 in both file. In between, some of the file_1 content might be appear twice. Thanks a lot.

Last edited by patrick87; 11-19-2009 at 05:58 AM..
# 2  
Old 11-19-2009
can the key fields (2000_t, g1110.b1, etc.) exist in file1 or file 2 more than once?
# 3  
Old 11-19-2009
Erm...
It just based on the column 1 content of file_1.
So far all the column 1 content of file_1 only appear once and no repeat.
you got any idea to solve my trouble?
thanks first Smilie

---------- Post updated at 04:54 AM ---------- Previous update was at 04:46 AM ----------

Hi, jsmithstl
sad to said that some might be appear twice as well in file_1's column 1.
I want all the file_1 content must exist in the output file. File_2 content just combined with it once match.
Sorry if bring you any inconvenience.
# 4  
Old 11-19-2009
Hi patrick,

i have tried your scenario, pelase find the script and its working for your scenario.

t1 - File_1
t2 - File_2

script:

$more t3.sh
#!/bin/bash
exec<t1
while read line
do
tes=`grep -w $line t2`
if [ -z "$tes" ]
then
echo "$line"
else
echo "$tes"
fi
done

-------------------
o/p

$sh t3.sh
2000_t Ali england 135
g1110.b1
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
2002_t
# 5  
Old 11-19-2009
Check if it solves your purpose:
Code:
awk 'NR==FNR{a[$0]=$0;next}{a[$1]=$0}END{for (i in a) print a[i]}' f1 f2

# 6  
Old 11-19-2009
hi,

I just try your script.
Unfortunately, it will face problem like this at the output file:
2000_t Ali england 135
g1110.b1
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
abb.2
abb.2 Zoe british 500
abb.2
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
2002_t

abb.2 repeat twice Smilie
Do I got do any mistakes?

---------- Post updated at 06:34 AM ---------- Previous update was at 06:31 AM ----------

hi,

I just try the awk code that you suggested.
It give the output like this:
2001_t Ali england 305
2002_t
g1110.b1
g1111.b1 Lucy russia 126
g1112.b1 Lucy russia 180
abb.1 Zoe british 150
abb.2
abb.3 Zoe british 700
2000_t Ali england 135

It is a bit different with my desired output result.
Do you know what is the problem causing it?
Thanks ya.

---------- Post updated at 06:47 AM ---------- Previous update was at 06:34 AM ----------

Hi uthay85,
Your script work perfectly if the column 1 at file_1 appear only once.
Do you have any idea if some of the content of column 1 at file_1 appear twice?
Thanks ya.
# 7  
Old 11-23-2009
Does anybody got idea to archive this goal?!
My input:
File_1:
2000_t
g1110.b1
abb.1
2001_t
g1111.b1
abb.2
abb.2
g1112.b1
abb.3
2002_t
.
.

File_2:
2000_t Ali england 135
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
abb.2
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
.
.

My desired output file:
2000_t Ali england 135
g1110.b1
abb.1 Zoe british 150
2001_t Ali england 305
g1111.b1 Lucy russia 126
abb.2 Zoe british 500
abb.2
g1112.b1 Lucy russia 180
abb.3 Zoe british 700
2002_t
.
.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to move specific files to directory based on match to file

I am trying to mv each of the .vcf files in the variants folder to the folder in /home/cmccabe/f2 that the .vcf id is found in file. $2 in file will always have the id of a .vcf in the variants folder. The line in blue staring with R_2019 in file up to the -v5.6 will always be an exact match to a... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

awk to update value based on pattern match in another file

In the awk, thanks you @RavinderSingh13, for the help in below, hopefully it is close as I am trying to update the value in $12 of the tab-delimeted file2 with the matching value in $1 of the space delimeted file1. I have added comments for each line as well. Thank you :). awk awk '$12 ==... (10 Replies)
Discussion started by: cmccabe
10 Replies

3. Shell Programming and Scripting

Match all lines in file where specific text pattern is less than

In the below file I am trying to grep or similar, all lines where only AF= is less than 0.4.. Thank you :). grep grep "AF=" ,+ .4 file file 12 112036782 . T C 34.0248 PASS ... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

How to merge variable data from another file into specific place?

Hello, I'm trying to create multiple commands using a variable input from another file but am not getting any successful results. Basically, file1.txt contains multiple lines with single words: <file1.txt> yellow blue black white I want to create multiple echo commands with these... (8 Replies)
Discussion started by: demmel
8 Replies

5. Shell Programming and Scripting

Help with ksh-to read ip file & append lines to another file based on pattern match

Hi, I need help with this- input.txt : L B white X Y white A B brown M Y black Read this input file and if 3rd column is "white", then add specific lines to another file insert.txt. If 3rd column is brown, add different set of lines to insert.txt, and so on. For example, the given... (6 Replies)
Discussion started by: prashob123
6 Replies

6. Shell Programming and Scripting

Help with replace line based on specific pattern match

Input file data20714 7327 7366 detail data20714 7327 7366 main data250821 56532 57634 detail data250821 57527 57634 main data250821 57359 57474 main data250821 57212 57301 main data250821 57140 57159 detail data250821 56834 57082 main data250821 56708 56779 main ... (3 Replies)
Discussion started by: perl_beginner
3 Replies

7. Shell Programming and Scripting

fetch last line no form file which is match with specific pattern by grep command

Hi i have a file which have a pattern like this Nov 10 session closed Nov 10 Nov 9 08:14:27 EST5EDT 2010 on tty . Nov 10 Oct 19 02:14:21 EST5EDT 2010 on pts/tk . Nov 10 afrtetryytr Nov 10 session closed Nov 10 Nov 10 03:21:04 EST5EDT 2010 Dec 8 Nov 10 05:03:02 EST5EDT 2010 ... (13 Replies)
Discussion started by: Himanshu_soni
13 Replies

8. Shell Programming and Scripting

Merge lines in text file based on pattern

Hello, I have searched forum trying to find a solution to my problem, but could not find anything or I did not understand the examples.... I should say, I am very inexperienced with text processing. I have a text file with approx 60k lines in it. I need to merge lines based on the number... (8 Replies)
Discussion started by: Bertik
8 Replies

9. Shell Programming and Scripting

Paste two file side by side together based on specific pattern match problem

Input file_1: P78811 P40108 O17861 Q6NTW1 P40986 Q6PBK1 P38264 Q6PBK1 Q9CZ49 Q1GZI0 Input file_2: (6 Replies)
Discussion started by: patrick87
6 Replies

10. Shell Programming and Scripting

Concatenating and appending string based on specific pattern match

Input #GEO-1-type-1-fwd-Initial 890 1519 OPKHIJEFVTEFVHIJEFVOPKHIJTOPKEFVHIJTEFVOPKOPKHIJHIJHIJTTOPKHIJHIJEFVEFVOPKHIJOPKHIJOPKEFVEFVOPKHIJHIJEFVHIJHIJEFVTHIJOPKOPKTEFVEFVEFVOPKHIJOPKOPKHIJTTEFVEFVTEFV #GEO-1-type-2-fwd-Terminal 1572 2030... (7 Replies)
Discussion started by: patrick87
7 Replies
Login or Register to Ask a Question