Sponsored Content
Top Forums Shell Programming and Scripting Match value in two files and replace values in selected columns Post 303031003 by jiam912 on Wednesday 20th of February 2019 10:44:06 AM
Old 02-20-2019
Match value in two files and replace values in selected columns

The purpose is to check if values for column 3 and 4 in file1 match with column 1 in file2.

If any value match do:

1) Replace values in file2 for column 2 and 3 using the information of file1 columns 5 and 6

2) Replace string ($1,1,5) and string ($1,6,5) in file2 with values of columns 7 and 8 from file1

3) Add character R for lines replaced and O for lines not replaced.

file1

Code:
2,100,31431,37131,999991.70,0000000.30,11111,22222,3
3,100,31431,37471,111113.20,1111111.30,22222,33333,4

file2

Code:
3143137113 318512.50 2334387.50 100
3143137131 318737.50 2334387.50 100
3143137201 319612.50 2334387.50 100
3143137471 322987.50 2334387.50 100
3143137491 323237.50 2334387.50 100

Desired output:

Code:
31431,37113,318512.50,2334387.50,100,O
11111,22222,999991.70,0000000.30,100,R
31431,37201,319612.50,2334387.50,100,O
22222,33333,111113.20,1111111.30,100,R
31431,37491,323237.50,2334387.50,100,O

I tried

Code:
awk -F, 'NR==FNR{a[$3 $4]=substr($0,length($3 FS)+1);next} $1 in a{print a[$1],$NF;next} {$1=substr($1,1,5) OFS substr($1,6,5);} 1' OFS=, file1 FS=' ' file2


Last edited by jiam912; 02-20-2019 at 01:57 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare selected columns of two files and print whole line with mismatch

hi! i researched about comparing two columns here and got an answer. but after examining my two files, i found out that the first columns of the two files are not unique with each other. all i want to compare is the 2nd and 3rd column. FILE 1: ABS 456 315 EBS 923 163 JYQ3 654 237 FILE 2:... (1 Reply)
Discussion started by: engr.jay
1 Replies

2. Shell Programming and Scripting

compare two files, selected columns only

hi! i have two files that looks like this file 1: ABS 123 456 BCDG 124 542 FGD 459 762 file 2: ABS 132 456 FGD 459 762 output would be: from file1: ABS 132 456 BCDG 124 542 from file 2: ABS 132 456 (4 Replies)
Discussion started by: kingpeejay
4 Replies

3. Shell Programming and Scripting

Get values from different columns from file2 when match values of file1

Hi everyone, I have file1 and file2 comma separated both. file1 is: Header1,Header2,Header3,Header4,Header5,Header6,Header7,Header8,Header9,Header10 Code7,,,,,,,,, Code5,,,,,,,,, Code3,,,,,,,,, Code9,,,,,,,,, Code2,,,,,,,,,file2... (17 Replies)
Discussion started by: cgkmal
17 Replies

4. Shell Programming and Scripting

Replace duplicate columns with values from first occurrence

I've a text file with below values viz. multiple rows with same values in column 3, 4 and 5, which need to be considered as duplicates. For all such cases, the rows from second occurrence onwards should be modified in a way that their values in first two columns are replaced with values as in first... (4 Replies)
Discussion started by: asyed
4 Replies

5. Shell Programming and Scripting

Match on columns and replace other columns

Hi Friends, I have the following input file cat input chr1 100 200 0.1 0.2 na 1 na nd chr1 105 200 0.1 0.2 1 1 na 98 chr1 110 290 nf 1 na nd na 1 chr2 130 150 12 3 na 1 na 1 chr3 450 600 nf nf na 10 na nd chr4 300 330 1 1 10 11 23 34 My requirement is 1. If $6 is na make $7 nd and... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

Replace values in columns

I have the following input format: AA00000712000 -0.17 0.90 -1.04 -0.37 -1.45 -1.13 -0.22 -0.34 -0.55 2.37 0.67 -0.48 -0.48 AA00000712001 0.15 0.03 0.47 0.62 2.04 1.14 0.29 -0.81 0.85 0.53 1.00 -0.10 -0.48 BB00000712000 1.32 -0.47 0.44 0.00 0.98 ... (4 Replies)
Discussion started by: ncwxpanther
4 Replies

7. Shell Programming and Scripting

Reporting where values match and mismatch across three columns

Hello, I have values in three columns. Some values occur in all three columns, other values are present in only one or two columns. I would like to be able to see where the matches and mismatches occur. Thanks in advance for any advice! I have: A B C 1 1 2 2 3 3 3 ... (6 Replies)
Discussion started by: MDeBiasse
6 Replies

8. Shell Programming and Scripting

Multiple columns replace with null values.

I am trying to replace the partcular columns(Col3,col5,col20,col44,col55,co56,col59,col60,col61,col62,col74,col75,col88,col90,col91,col93,col94,col95) with empty Input file Col1,col2,col3,col4,col5------,col100 1,2,3,4,5,---------,100 3,4,5,6,7,---------,300 Output : ... (3 Replies)
Discussion started by: onesuri
3 Replies

9. UNIX for Dummies Questions & Answers

Read in Multiple log files and output selected variables and values to cvs file

I have several problems with my problems: I hope you can help me. 1) the If else statement I am getting an error message. My syntax must be incorrect because the entire statement is throwing an error. For example in filew.log if these items don't exist Memsize, SASFoundation and also if... (0 Replies)
Discussion started by: dellanicholson
0 Replies

10. UNIX for Beginners Questions & Answers

Data match 2 files based on first 2 columns matching only and join if match

Hi, i have 2 files , the data i need to match is in masterfile and i need to pull out column 3 from master if column 1 and 2 match and output entire row to new file I have tried with join and awk and i keep getting blank outputs or same file is there an easier way than what i am... (4 Replies)
Discussion started by: axis88
4 Replies
comm(1) 						      General Commands Manual							   comm(1)

NAME
comm - Compares two sorted files. SYNOPSIS
comm [-123] file1 file2 STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: command: XCU5.0 Refer to the standards(5) reference page for more information about industry standards and associated tags. OPTIONS
Suppresses output of the first column (lines in file1 only). Suppresses output of the second column (lines in file2 only). Suppresses output of the third column (lines common to file1 and file2). The command comm -123 produces no output. OPERANDS
A pathname of the first file to be compared. If file1 is a hyphen (-), the standard input is used. A pathname of the second file to be compared. If file2 is a hyphen (-), the standard input is used. If both file1 and file2 refer to standard input or to the same FIFO special, block special or character special file, the results are unde- fined. DESCRIPTION
The comm command reads file1 and file2 and writes three columns to standard output, showing which lines are common to the files and which are unique to each. The leftmost column of standard output includes lines that are in file1 only. The middle column includes lines that are in file2 only. The rightmost column includes lines that are in both file1 and file2. If you specify a hyphen (-) in place of one of the file names, comm reads standard input. Generally, file1 and file2 should be sorted according to the collating sequence specified by the LC_COLLATE environment variable. (See sort(1).) If the input files are not sorted properly, the output of comm might not be useful. EXIT STATUS
Successful completion. Error occurred. EXAMPLES
In the following examples, file1 contains the following sorted list of North American cities: Anaheim Baltimore Boston Chicago Cleveland Dallas Detroit Kansas City Milwaukee Minneapolis New York Oakland Seattle Toronto The second file, file2, contains this sorted list: Atlanta Chicago Cincinnati Houston Los Angeles Montreal New York Philadelphia Pittsburgh San Diego San Francisco St. Louis To display the lines unique to each file and common to the two files, enter: comm file1 file2 This command results in the following output: Anaheim Atlanta Baltimore Boston Chicago Cincinnati Cleveland Dal- las Detroit Houston Kansas City Los Angeles Milwaukee Minneapolis Montreal New York Oakland Philadel- phia Pittsburgh San Diego San Francisco Seattle St. Louis Toronto The leftmost column contains lines in file1 only, the middle column contains lines in file2 only, and the rightmost column contains lines common to both files. To display any one or two of the three output columns, include the appropriate flags to suppress the columns you do not want. For example, the following command displays columns 1 and 2 only: comm -3 file1 file2 Anaheim Atlanta Baltimore Boston Cincinnati Cleveland Dallas Detroit Houston Kansas City Los Angeles Milwaukee Minneapolis Montreal Oakland Philadelphia Pittsburgh San Diego San Francisco Seattle St. Louis Toronto The following command displays output from only the second column: comm -13 file1 file2 Atlanta Cincinnati Houston Los Angeles Montreal Philadelphia Pittsburgh San Diego San Francisco St. Louis The following command displays output from only the third column: comm -12 file1 file2 Chicago New York SEE ALSO
Commands: cmp(1), diff(1), sdiff(1), sort(1), uniq(1) comm(1)
All times are GMT -4. The time now is 02:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy