two file merge with awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting two file merge with awk
# 1  
Old 04-01-2010
two file merge with awk

Help
I read a file that has 2 fields.
look for in a second file the first field and update it with the second field of first file.

file1
1131518fat11416.txt ../newaod/2001/04/2001-04-00129233-1.txt

file2
INSERT INTO tabric
VALUES
('00566605','00499181','ZPPLGU50S20F158O','1131518fat11416.txt','2000',1)

risultato
INSERT INTO tabric
VALUES
('00566605','00499181','ZPPLGU50S20F158O','../newaod/2001/04/2001-04-00129233-1.txt','2000',1)

I can not do it with awk.
Thanks to all

Last edited by jim mcnamara; 04-01-2010 at 03:11 PM.. Reason: remove url/trash at file end
# 2  
Old 04-01-2010
Assuming your files don't contain pipes ('|')
Code:
#!/bin/bash
while read A B
do
	sed "s|$A|$B|" file2
done < file1

# 3  
Old 04-01-2010
Quote:
Originally Posted by frans
Assuming your files don't contain pipes ('|')
Code:
#!/bin/bash
while read A B
do
	sed "s|$A|$B|" file2
done < file1

Assuming $A doesn't contain "|", "\", ".", "*", "[", "]", "^", "$" and assuming $B doesn't contain "|", "\", "&". (If it's gnu sed, then there are a few other special characters to worry about.)

Regards,
Alister

---------- Post updated at 05:55 PM ---------- Previous update was at 04:31 PM ----------



The following attempts to backslash escape characters that are special in posix basic regular expressions when they appear in the first field of file1, and attempts to backslash escape characters that are special in sed replacement text when they appear in the second field of file1. I say "attempts" because I only tested it against a few contrived lines of text, and only using one sed implementation (not gnu, so any additional metacharacters in gnu sed will leak through unescaped and possibly break the solution), and because it was concocted for the fun of it.

The (hopefully correctly) sanitized input from file1 is used to build a stream of ed commands which modify file2:
Code:
{ sed 'h; s| .*||; s|[][^$/\.*]|\\&|g; s|.*|/&/|; x; s|.* ||; s|[\&/]|\\&|g; s|.*|s//&/|; H; g; s|\n||' file1; printf 'w\nq\n'; } | ed -s file2

For important tasks of this type, I would suggest using a tool that can match strings literally, like AWK's index/substr functions. Don't expand sh parameters into sed commands unless you are sure the incoming text is free of special characters.

Cheers,
Alister

Last edited by alister; 04-01-2010 at 05:56 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk can't open file; file merge attempt

Dear all, I have an AWK related issue. I have two data files; the first, FileA has fewer lines, the second, FileB has more. FileA is a subset of FileB . Both files are tab delimited. What I want to do? When the first two columns for FileA match the first two columns of FileB, I want to... (4 Replies)
Discussion started by: A_Human_Person
4 Replies

2. Shell Programming and Scripting

Merge values from multiple directories into one file in awk or bash

I am trying to merge or combine all $1 values in validation.txt from multiple directories into one new file and output it here tab-delimited:/home/cmccabe/Desktop/20x/total/total.txt. Each $2 value and the header would then be a new field in total.txt. I am not sure how to go about this as cat is... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

How to merge fields in a single file using awk ?

Hi, From a file, using: awk -F" " '{ if (NF == 6) print $1, $2, $3, $4, $5, $6; if (NF == 5) print $1, $2, $3, $4, $5; }' i printed out the required output. But i'm trying to merge the columns. Please look at the desired output. Any suggestions? Thanks Output: 00015 PSA1 ... (5 Replies)
Discussion started by: web2moha
5 Replies

4. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

5. UNIX for Dummies Questions & Answers

merge files along with file names (awk)?

Dear programmers, I have a question about conditionally merging multiple files and having their file names in the first column. Input files: file.1.extension file.2.extension file.3.extension file.4.extension ... file.1000.extension where each file looks like this (with multiple lines):... (5 Replies)
Discussion started by: wei.deng
5 Replies

6. UNIX for Dummies Questions & Answers

file merge using awk

Hi All, I have 2 csv files. 1st file has 10 columns and the 2nd file has 12 columns. The requirement is, if the 4th column of file1 matches with the 4th column of file2, then append the 12th column of file2 with file1. Both files have equal number of lines and the 4th column values are... (1 Reply)
Discussion started by: ganesh_248
1 Replies

7. Shell Programming and Scripting

Another awk merge

File1 01C6,039E,RDF1+TDEV,120(2),1507_3RAID5,33 01C8,0E46,RDF1+TDEV,200(8),1507_3RAID5,33 01D4,045E,RDF1+TDEV,8,1507_3RAID5,33 01D5,045F,RDF1+TDEV,8,1507_3RAID5,33 File2 01C6,AA,BB,CC 01C8,,,NN 01D5,BB,KK, 0122,GG,, Ouput ... (2 Replies)
Discussion started by: greycells
2 Replies

8. Shell Programming and Scripting

please help to merge file with awk or sed

hi experts please help me,thanks in advance file1 arch : x86 install : pass make os : pass make build kernel : pass=100 failed=45 usb storage pass : The Linux Kernel Archives file2 arch : ppc install : failed make os : http://kernel.org (6 Replies)
Discussion started by: yanglei_fage
6 Replies

9. Shell Programming and Scripting

Merge lines in a file with Awk - incorrect output

Hi, I would like: FastEthernet0/0 is up, line protocol is up 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 output errors, 0 collisions, 0 interface resets Serial1/0:0 is up, line protocol is up 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0... (14 Replies)
Discussion started by: mv652
14 Replies

10. Shell Programming and Scripting

merge two files into one file use awk

Hi, guys. I have one question: I have two files: passwd and shadow (the number of records in these files are not equal)the contents of them are below: passwd: ************** ftp:x:24:24: sshd:x:71:65: uucp:x:10:14: brownj:x:5005:1000: sherrys: x :5006:1000: ... ************* ... (2 Replies)
Discussion started by: daikeyang
2 Replies
Login or Register to Ask a Question