Compare two files, if the columns are same then replace next column with some value


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compare two files, if the columns are same then replace next column with some value
# 1  
Old 04-10-2020
Compare two files, if the columns are same then replace next column with some value

Hi Team,

As i'm new to the shell scripting, i stuck in a place to replace a column valuein a file when compared with other file. Could you please help me to work on this?
I have 2 files test1.csv and test2.csv. I need to replace empdep column in test1.csv if it has the value as "sign*". I have second file test2.csv which have the value need to replace that" sign*".

Note : I'm using ksh and test1.csv have around 2,048,576 rows and test2.csv have 10000 rows

test1.csv
Code:
empname,place,empdep
aaaa ,city1,001
bbbb,city2,sign-1
dddd,city1,005
ffff,city5,sign-2
hhhh,city7,sign-1

test2.csv
Code:
empname ,new
aaaa ,001 
bbbb,002
cccc,003
dddd ,005
eeee,006
ffff,007
gggg,008
hhhh,009

Expected Result:
Code:
empname,place,empdep
aaaa ,city1,001
bbbb,city2,002
dddd,city1,005
ffff,city5,007
hhhh,city7,009

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!

Last edited by vgersh99; 04-10-2020 at 10:58 AM.. Reason: code tags, please!
# 2  
Old 04-10-2020
Try
Code:
awk -F, 'NR==FNR {T[$1]=$2; next} $3 ~ /^sign/ {$3=T[$1]} 1' OFS=, file2 file1
empname,place,empdep
aaaa ,city1,001
bbbb,city2,002
dddd,city1,005
ffff,city5,007
hhhh,city7,009

This User Gave Thanks to RudiC For This Post:
# 3  
Old 04-10-2020
Thanks a lot.!!!

Hi Rudic,
I just tried it in a sample file... Its working....


Thanks a lot Smilie

Could you please explain the command, it will help me to understand?
# 4  
Old 04-10-2020
When working on the first input (file2), NR == FNR, it collects field 2's values into an array indexed by field 1.
When working on the second, it checks if field 3 begins with sign, and replaces this with the respective array value (agin indexed by field 1).
The final 1 represents a "TRUE" result and starts the default action which is print which by default prints $0.
This User Gave Thanks to RudiC For This Post:
# 5  
Old 04-10-2020
Question

I have tried for "I" separated file with below command got error as below... could u please help on this?

Code:
awk -F| 'NR==FNR {T[$1]=$2; next} $7 ~ /^Sync/ {$7=T[$1]} 1' OFS=| file2.csv file1.csv

Code:
awk: A flag requires a parameter: F
Usage: awk [-u] [-F Character][-v Variable=Value][-f File|Commands][Variable=Value|File ...]
ksh: NR==FNR {T[$1]=$2; next} $7 ~ /^Sync/ {$7=T[$1]} 1:  not found.

# 6  
Old 04-10-2020
Try escaping / quoting the pipe character to use it for the field separator; else it will be interpreted by the shell for a command pipeline (and fail).
# 7  
Old 04-10-2020
tried, but empty space instead of data

I tried the below command, but got empt space as a result

Code:
awk 'BEGIN{ FS=OFS="|" }; NR==FNR {T[$1]=$2; next} $7 ~ /^sign {$7=T[$1]} 1' OFS="|" file1.csv file2.csv> final.csv

wrong result please check below

Code:
 xfsgdh|gdjshd|bbbb|20191228|230||sign-1|14420|44766|||||||20191228 ---  original 

xfsgdh|gdjshd|bbbb|20191228|230|||14420|44766|||||||20191228 -  result after code

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

2. Shell Programming and Scripting

Compare and replace two columns from two files

Hello, I have two text tab delimited files File 1 has 30 columns. I am pasting only first 9 Chr Position Ref Alt Score Gene HGVS_C HGVS_P Coding_Consequence dbSNP 1 17312743 C T 1 Gene1 - ... (2 Replies)
Discussion started by: nans
2 Replies

3. Shell Programming and Scripting

Compare two columns and replacing it with value from third column!!

Hi, I am new to Unix and I am finding it hard to fix a particular logic. The context is as below. File 1 : This contains of 5 fields : Type | Bank Code | Account | Name | Date/Time 60|ABC123|TX123456|XXXX|YYYYMMDDHH:MM:SS 72|ABC123|MYD34561|XXXX|YYYYMMDDHH:MM:SS... (4 Replies)
Discussion started by: DJose
4 Replies

4. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

5. Shell Programming and Scripting

Compare first column of 2 files and replace

Hi All, I have 2 files in the following format : File 1 S00999999|BHANU|TEST|007 JOHN DOE APT 999||VENGA HIGHWAY|MA|09566|SCO DUAL|20140201|20140331|20140401|20140630|20140327| S00888888|BU|TES|009 JOHN DOE APT 909||SENGA HIGHWAY|MA|08566|SCO... (1 Reply)
Discussion started by: nua7
1 Replies

6. Shell Programming and Scripting

Combine columns from many files but keep them aligned in columns-shorter left column issue

Hello everyone, I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble. I have many files each having two columns and hundreds of rows. first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
Discussion started by: isildur1234
5 Replies

7. Shell Programming and Scripting

awk compare column n replace with in one file

hi Friends need to compare columns in one file where the data looks like below laptop,IBM phone,samsung car,rental user1,laptop user2,laptop user3,phone want to get output as laptop,IBM phone,samsung car,rental user1,IBM user2,IBM user3,samsung need to seach $2 in array of $1 and... (4 Replies)
Discussion started by: arun1401
4 Replies

8. Shell Programming and Scripting

Compare Two Files(Column By Column) In Perl or shell

Hi, I am writing a comparator script, which comapre two txt files(column by column) below are the precondition of this comparator 1)columns of file are not seperated Ex. file1.txt 8888812341181892 1243548895685687 8945896789897789 1111111111111111 file2.txt 9578956789567897... (2 Replies)
Discussion started by: kumar96877
2 Replies

9. Ubuntu

How to compare two columns and fetch the common data with additional column

Dear All, I am new to this forum and please ignore my little knowledge :p I have two types of data (a subset is given below) data version 1: 439798 2 1 451209 1 2 508696 2 1 555760 2 1 582757 1 2 582889 1 2 691827... (2 Replies)
Discussion started by: evoll
2 Replies

10. UNIX for Dummies Questions & Answers

find common lines using just one column to compare and result with all columns

Hi. If we have this file A B C 7 8 9 1 2 10 and this other file A C D F 7 9 2 3 9 2 3 4 The result i´m looking for is intersection with A B C D F so the answer here will be (10 Replies)
Discussion started by: alcalina
10 Replies
Login or Register to Ask a Question