Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace and add line in file with line in another file based on matching string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace and add line in file with line in another file based on matching string
# 8  
Old 06-13-2013
request
Code:
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB
G789, value A, valueB


Replacing file1 and file2 in the awk seems to give correct result.
Code:
awk -F, 'NR==FNR{A[$1]++;print;next} !A[$1]++' file2 file1 | sort
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB
G789, value A, valueB

This User Gave Thanks to Jotne For This Post:
pamu
# 9  
Old 06-13-2013
Messed up with requirement Smilie
Code:
awk -F, 'NR==FNR{A[$1]=$0;next}{if(A[$1]){print A[$1];delete A[$1]}else{print}}
    END{for(i in A){if(A[i]){print A[i]}}}' file2 file1

Thanks for changing the code Jotne, which also works with changing file sequence. Smilie
Last edited by pamu; 06-13-2013 at 05:42 AM..
This User Gave Thanks to pamu For This Post:
jyu3
# 10  
Old 06-13-2013
Thanks so much, guys. You guys are genius!!
All the solutions work. But I need to study each of them to see how they work.
Awk code is so hard to understand. It's like a magic to me.
# 11  
Old 06-13-2013
awk Tutorial

PS I have changed !A[$1]++ to !A[$1], there are no need to imcrement value for the second part

awk -F, 'NR==FNR{A[$1]++;print;next} !A[$1]' f2 f1 | sort

Some more easy to read
Code:
awk -F, '
	NR==FNR {
		A[$1]++
		print
		next}
	!A[$1]
	' f1 f2 | sort

Code:
cat f2	
C345, valueX, valueY
D456, valueX, valueY
G789, value A, valueB

Code:
cat f1
A123, valueA, valueB
B234, valueA, valueB
C345, valueA, valueB
D456, valueA, valueB
E567, valueA, valueB
F678, valueA, valueB

When you run awk with multiple files, it will run once for every line in all files.
NR is line nuber that increase all the time trough all files
FNR is the line number for one single file and is reset on the next file
To show this run
Code:
awk -F, '{print $0,NR,FNR}' f2 f1
C345, valueX, valueY 1 1
D456, valueX, valueY 2 2
G789, value A, valueB 3 3
A123, valueA, valueB 4 1
B234, valueA, valueB 5 2
C345, valueA, valueB 6 3
D456, valueA, valueB 7 4
E567, valueA, valueB 8 5
F678, valueA, valueB 9 6

Hence when run NR==FNR it will only do some finction on the first file in the line f2
So this code will be run in f2 {A[$1]++;print;next}
The reason that only this code is run on f2 and no other code is the next.

Part 1
Now we look at the array.
A[$1]++
It stores every value of column 1 in f2 into an array named A
The ++ ads a value to every array, and since all value of column 1 in f2 is unique the ++ can be change to =1
So A[$1]++ can be replaced by A[$1]=1
This gives
Code:
A[C345]=1
A[D456]=1
A[G789]=1

print
Here we print all post of the f2, and this is what we want. Print value of f2, even if there are same index in f1
Code:
C345, valueX, valueY
D456, valueX, valueY
G789, value A, valueB

next
Stop doing more on this record and go to next record and start code from start.

Part 2
When the 3 record of f2 are done, start working on file f1
Since NR==FNR is no more true on f1, skip this part and run
!A[$1]
We continue working on same array A
the $1 now handles column 1 in f1
This is test part, and its tru only if array is 0 due to the !
Run in this on f1 we get this, since we do not add any value to it.
Code:
A[A123]=0
A[B234]=0
A[C345]=1 That it got form f2
A[D456]=1 That it got form f2
A[E567]=0
A[F678]=0

! neglect the value and print only those with value 0
the !A[$1] has no action {}, so it does the default, print record $0 from f1 - {print $0}
Code:
A123, valueA, valueB
B234, valueA, valueB
E567, valueA, valueB
F678, valueA, valueB

After awk
The final sort gives
Code:
A123, valueA, valueB
B234, valueA, valueB
C345, valueX, valueY
D456, valueX, valueY
E567, valueA, valueB
F678, valueA, valueB
G789, value A, valueB

I do hope this give some light on what awk does Smilie
This User Gave Thanks to Jotne For This Post:
jyu3
# 12  
Old 06-13-2013
Jotne,

This is an excellent excellent tutorial of awk! This the first time I actually understand how the code works instead of just copy and paste it in my script. I couldn't be more thankful.

Thanks for all the effort that you spent to come up with such a nice explanation.

You are the hero!!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Search for a string,delete the line and replace with new string in a file

Hi Everyone, I have a requirement in ksh where i have a set of files in a directory. I need to search each and every file if a particular string is present in the file, delete that line and replace that line with another string expression in the same file. I am very new to unix. Kindly help... (10 Replies)
Discussion started by: Pradhikshan
10 Replies

2. Shell Programming and Scripting

Replace line in file with line in another file based on matching string

HI Can any one guide me how to achieve this task. I have 2 files env.txt #Configuration.Properties values identity_server_url = http://identity.test-hit.com:9783/identity/service/user/register randon_password_length = 6 attachment_file_path = /pass/temp/attachments/... (1 Reply)
Discussion started by: nikilbr86
1 Replies

3. Shell Programming and Scripting

Replace a string with each line from another file repeatedly

I don't know if it's been asked before but seems i gave up seeking. i have 2 files : file1.txt Monday XXXX Tuesday XXXX XXXX Wednesday Thursday XXXX XXXX is in every lines of file1.txt and i want to replace them with each line in file2.txt: home school cinema so output file is: ... (19 Replies)
Discussion started by: perseous
19 Replies

4. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

5. Shell Programming and Scripting

Replace line in file with line in another file based on matching string

Hi I am not the best scripter in the world and have run into a issue which you might be able to guide me on... I have two files. File1 : A123, valueA, valueB B234, valueA, valueB C345, valueA, valueB D456, valueA, valueB E567, valueA, valueB F678, valueA, valueB File2: C345,... (5 Replies)
Discussion started by: luckycharm
5 Replies

6. Shell Programming and Scripting

replace (sed?) a single line/string in file with multiple lines (string) from another file??

Can someone tell me how I can do this? e.g: Say file1.txt contains: today is monday the 22 of NOVEMBER 2010 and file2.txt contains: the 11th month of How do i replace the word NOVEMBER with (5 Replies)
Discussion started by: tuathan
5 Replies

7. Shell Programming and Scripting

Read file and for each line replace two variables, add strings and save output in another file

Hi All, I have a file, let's call it "info.tmp" that contains data like this .. ABC123456 PCX333445 BCD789833 I need to read "info.tmp" and for each line add strings in a way that the final output is put /logs/ua/dummy.trigger 'AAA00001.FTP.XXX.BLA03A01.xxxxxx(+1)' where XXX... (5 Replies)
Discussion started by: Andy_ARG
5 Replies

8. Shell Programming and Scripting

replace string in file.1 with line from file.2

Hello all, the title makes this sound simple, and maybe it should be. This is by code: #!/bin/sh cp ch25.txt ch25.fn.tex n=`grep -c '^\' ch25_footnotes.txt > temp` r=`awk -F] '{print $2}' temp` `sed 's/\/\\footnote{$r}/' ch25.fn.tex` done This is what I am trying to... (6 Replies)
Discussion started by: ccox85
6 Replies

9. Shell Programming and Scripting

Replace string in a file within a range of line

Hi, I want to replace the srting '; with ABCD'; in a file from line 1 to line 65. Is there any single command to do it without using awk Thanks for quick reply https://www.unix.com/images/misc/progress.gif (3 Replies)
Discussion started by: tosattam
3 Replies

10. UNIX for Dummies Questions & Answers

how can search a String in one text file and replace the whole line in another file

i am very new to UNIX plz help me in this scenario i have two text files as below file1.txt name=Rajakumar. Discipline=Electronics and communication. Designation=software Engineer. file2.txt name=Kannan. Discipline=Mechanical. Designation=CADD Design Engineer. ... (6 Replies)
Discussion started by: kkraja
6 Replies
Login or Register to Ask a Question