Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace line in file with line in another file based on matching string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace line in file with line in another file based on matching string
# 1  
Old 01-09-2012
Replace line in file with line in another file based on matching string
Hi

I am not the best scripter in the world and have run into a issue which you might be able to guide me on...

I have two files.

File1 :
Code:
A123, valueA, valueB
B234, valueA, valueB
C345, valueA, valueB
D456, valueA, valueB
E567, valueA, valueB
F678, valueA, valueB

File2:
Code:
C345, valueX, valueY
D456, valueX, valueY

I basically want to match the lines in the file1 and file2 based on the value of the first col in both files. If these values match then replace the entry in file1 with the entry in file2

Output:

Code:
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB


Please note file1 is a million lines long so loops and greps take a long time!

Thanks a mil in advance

KB
Last edited by Franklin52; 01-09-2012 at 07:24 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 01-09-2012
Hi luckycharm,

One way using perl:
Code:
$ cat file1
A123, valueA, valueB
B234, valueA, valueB
C345, valueA, valueB
D456, valueA, valueB
E567, valueA, valueB
F678, valueA, valueB
$ cat file2
C345, valueX, valueY
D456, valueX, valueY
$ cat script.pl
use warnings;
use strict;

die qq[Usage: perl $0 <file1> <file2>\n] unless @ARGV == 2;

my %file2_key;

open my $file2, "<", pop @ARGV or die qq[ERROR: Cannot open input file\n];

while ( <$file2> ) {
        my @f = split /\s*,\s*/, $_, 2;
        $file2_key{ $f[0] } = $_;
}

while ( <> ) {
        my @f = split /\s*,\s*/, $_, 2;
        if ( exists $file2_key{ $f[0] } ) {
                print $file2_key{ $f[0] };
                next;
        }

        print;
}
$ perl script.pl file1 file2
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB

Regards,
Birei
# 3  
Old 01-09-2012
here is your code :-)
input:
Code:
# cat file1
A123, valueA, valueB
B234, valueA, valueB
C345, valueA, valueB
D456, valueA, valueB
E567, valueA, valueB
F678, valueA, valueB
# cat file2
C345, valueX, valueY
D456, valueX, valueY

code:
Code:
while read line
do
        column1=$( echo $line | awk -F' ' '{print $1}'| sed 's/^[ \t]*//g;s/[ \t]*$//g;s/,$//g' )
        linematch=$( cat file2 | grep $column1 )
        if [[ "$linematch" != "" ]] ; then
                lineno=$( cat file1 | grep -n $column1 | cut -d ':' -f 1 )
                echo "`sed "${lineno}d" file1`" > file1
                input=$( cat file2 | grep -i $column1 )
                echo "`sed $lineno" i\$input" file1`" > file1
        fi
 
done < file1

output after execution:
Code:
 cat file1
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB

# 4  
Old 01-09-2012
with awk
code:

awk -F, 'NR==FNR{a[$1]=$0;next;}a[$1]{$0=a[$1]}1' file2 file1
# 5  
Old 01-10-2012
Hi Agrawal,

Please explain how the above awk command works.
# 6  
Old 01-10-2012
1. awk first buffer the file2 data in the form of key, value pair (associative array).
and I have used key as col1 and value as entire line.
2. then awk will check whether array key is exist for column 1 in file1 (joining columns)
if yes, then replace file1 current value (i.e $0) with current value in array, if not skip it.
3. finally print the data.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Search for a string,delete the line and replace with new string in a file

Hi Everyone, I have a requirement in ksh where i have a set of files in a directory. I need to search each and every file if a particular string is present in the file, delete that line and replace that line with another string expression in the same file. I am very new to unix. Kindly help... (10 Replies)
Discussion started by: Pradhikshan
10 Replies

2. Shell Programming and Scripting

Replace line in file with line in another file based on matching string

HI Can any one guide me how to achieve this task. I have 2 files env.txt #Configuration.Properties values identity_server_url = http://identity.test-hit.com:9783/identity/service/user/register randon_password_length = 6 attachment_file_path = /pass/temp/attachments/... (1 Reply)
Discussion started by: nikilbr86
1 Replies

3. Shell Programming and Scripting

Replace a string with each line from another file repeatedly

I don't know if it's been asked before but seems i gave up seeking. i have 2 files : file1.txt Monday XXXX Tuesday XXXX XXXX Wednesday Thursday XXXX XXXX is in every lines of file1.txt and i want to replace them with each line in file2.txt: home school cinema so output file is: ... (19 Replies)
Discussion started by: perseous
19 Replies

4. Shell Programming and Scripting

Replace and add line in file with line in another file based on matching string

Hi, I want to achieve something similar to what described in another post: The difference is I want to add the line if the pattern is not found. File 1: A123, valueA, valueB B234, valueA, valueB C345, valueA, valueB D456, valueA, valueB E567, valueA, valueB F678, valueA, valueB ... (11 Replies)
Discussion started by: jyu3
11 Replies

5. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

6. Shell Programming and Scripting

replace (sed?) a single line/string in file with multiple lines (string) from another file??

Can someone tell me how I can do this? e.g: Say file1.txt contains: today is monday the 22 of NOVEMBER 2010 and file2.txt contains: the 11th month of How do i replace the word NOVEMBER with (5 Replies)
Discussion started by: tuathan
5 Replies

7. Shell Programming and Scripting

Retrieve string from each line in a file based on a pattern in Unix

I have a file which contains several lines. Sample content of the file is as below. OK testmessage email<test@123> NOK receivemessage email<123@test> NOK receivemessage email(123@test123) NOK receivemessage email<abc@test> i would like to know by scripting will... (10 Replies)
Discussion started by: ramasar
10 Replies

8. Shell Programming and Scripting

replace string in file.1 with line from file.2

Hello all, the title makes this sound simple, and maybe it should be. This is by code: #!/bin/sh cp ch25.txt ch25.fn.tex n=`grep -c '^\' ch25_footnotes.txt > temp` r=`awk -F] '{print $2}' temp` `sed 's/\/\\footnote{$r}/' ch25.fn.tex` done This is what I am trying to... (6 Replies)
Discussion started by: ccox85
6 Replies

9. Shell Programming and Scripting

Replace string in a file within a range of line

Hi, I want to replace the srting '; with ABCD'; in a file from line 1 to line 65. Is there any single command to do it without using awk Thanks for quick reply https://www.unix.com/images/misc/progress.gif (3 Replies)
Discussion started by: tosattam
3 Replies

10. UNIX for Dummies Questions & Answers

how can search a String in one text file and replace the whole line in another file

i am very new to UNIX plz help me in this scenario i have two text files as below file1.txt name=Rajakumar. Discipline=Electronics and communication. Designation=software Engineer. file2.txt name=Kannan. Discipline=Mechanical. Designation=CADD Design Engineer. ... (6 Replies)
Discussion started by: kkraja
6 Replies
Login or Register to Ask a Question