Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Compare two columns of one file to two columns in another file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compare two columns of one file to two columns in another file
# 1  
Old 05-29-2019
Compare two columns of one file to two columns in another file
I Have two files as below,

first file:

Code:
Start State |Next State |Session Count |Transition%

LA_product_view |home |694 |28.660%

LA_product_view | searchresults |54 |2.230%

home | 1101260 | 2 | 0.050%

second file:

Code:
Start State Next State Session Count Transition%

LA_product_view |home | 618 | 27.560%

LA_product_view |searchresults | 59 | 2.630%

home | price | 25 | 0.360%

i need to compare first two coloumns of first file and first two coloumns of second file and if they both match it should print 3,4 coloums of both file.if both the files has unique they should be listed with NA from the both files. my output should be like:

Code:
LA_product_view | home | 694 |28.660% |618 |27.560%

LA_product_view | searchresults |54 | 2.230% | 59 |2.630%

home | 1101260 | 2 |0.050% |N/A | N/A

home | price | N/A | N/A | 25 | 0.360%

can someone please help me in this..!
Last edited by vgersh99; 05-29-2019 at 11:52 AM.. Reason: Code tags, please!
# 2  
Old 05-29-2019
If you got a recent bash or ksh with "process substitution", and

Code:
join --version
join (GNU coreutils) 8.30

, try
Code:
join  -t"|" -a1 -a2 -e"N/A" -o auto --nocheck-order --header  <(sed  's/|/#/' file1) <(sed 's/|/#/' file2) | sed 's/#/|/'
Start State|Next State Session | Count |Transition%|Session Count|Transition%
LA_product_view|home|694|28.660%|618|27.560%
LA_product_view|searchresults|54|2.230%|59 |2.630%
home|price |N/A|N/A|25|0.360%
home|view|2|0.050%|N/A|N/A

# 3  
Old 05-29-2019
@RudiC
mine is join (GNU coreutils) 8.22
i tried above it didnt work...any awk solution if possible?

--- Post updated at 01:16 PM ---

@RudiC
please help me if my files are in comma separated can u check will this work
file1.txt:
Code:
Start State,Next State,Session Count,Transition%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/home,1126,28.600%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/LA_product_view,818,20.780%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/orders/price,21,0.530%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/login/,997,25.330%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/organizations,31,0.780%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/partsVisualizer,167,4.240%

file2.txt:
Code:
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/LA_product_view,894,13.100%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/home,1184,17.350%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/orders/price,25,0.360%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/searchresults,226,3.310%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/inventories/LA_entry,3,0.040%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/cart,51,0.740%

can u check if this is like this
Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 05-29-2019 at 11:23 AM.. Reason: Code tags, please!
# 4  
Old 05-29-2019
What keeps you from checking it yourself, reading the join man page if need be?
# 5  
Old 05-29-2019
@rudic
i checked with this
join -t"," -a1 -a2 -e"N/A" -o auto --nocheck-order --header <(sed 's/,/#/' file1.txt) <(sed 's/,/#/' file2.txt) | sed 's/#/,/'

it didnt worked...!
# 6  
Old 05-29-2019
You may need to pre-process your files; sorting them but keeping the header in line 1. Then, join on field 2.
# 7  
Old 05-29-2019
a bit verbose, but try: awk -f rag.awk file1 file2 where rag.awk is:
Code:
BEGIN {
  FS=OFS=","
}
{idx=$1 OFS $2; _34=$3 OFS $4}
FNR==NR {if (FNR>1) f1[idx]=_34; next}
idx in f1 { print idx, f1[idx], _34; delete f1[idx];next}
{print idx, "NA","NA", _34}
END {
  for (i in f1)
     print i, f1[i], "NA", "NA"
}

You didn't provide the expected output for your latest sample files, but... the above produces - not sure if it's what you'd expect:
Code:
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/LA_product_view,818,20.780%,894,13.100%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/home,1126,28.600%,1184,17.350%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/orders/price,21,0.530%,25,0.360%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/searchresults,NA,NA,226,3.310%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/inventories/LA_entry,NA,NA,3,0.040%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/cart,NA,NA,51,0.740%
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/organizations,31,0.780%,NA,NA
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/pages/layout/partsVisualizer,167,4.240%,NA,NA
/ccstoreui/v1/pages/layout/home,/ccstoreui/v1/login/,997,25.330%,NA,NA

This User Gave Thanks to vgersh99 For This Post:
Raghuram717
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: compare values in two columns of the same file

I'm trying to learn awk, but I've hit a roadblock with this problem. I have a hierarchy stored in a file with 3 columns: id name parentID 4 D 2 2 B 1 3 C 1 1 A 5 I need to check if there are any values in column 3 that are not represented anywhere in column 1. I've tried this: awk '{arr;}... (7 Replies)
Discussion started by: kaktus
7 Replies

2. Shell Programming and Scripting

Compare 2 columns from the same file and print a value depending on the result

Hello Unix gurus, I have a file with this format (example values): label1 1 0 label2 1 0 label3 0.4 0.6 label4 0.5 0.5 label5 0.1 0.9 label6 0.9 0.1 in which: column 1 is a row label column 2 and 3 are values I would like to do a simple operation on this table and get the... (8 Replies)
Discussion started by: ksennin
8 Replies

3. Shell Programming and Scripting

Compare columns in a single file

i have the following files (all separated by tabs): file 1.txt 1 yes 2 no 3 yes 4 yes file 2.txt a no b no c yes d no i combine the above files in file 3 which looks like file 3.txt 1 yes a no 2 no b no 3 yes c yes 4 yes d no now, i need to compare the values between column 2... (3 Replies)
Discussion started by: msonoth
3 Replies

4. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

5. Shell Programming and Scripting

Compare two date columns in same file

Hi All, Need to compare two date columns from the filname FinalDate.txt. My data's are like below D_OT_START D_EXP_STR Amount 1/3/2012 1/3/2012 5000 6/21/2011 6/25/2011 6000 2/28/2011 2/28/2011 7000 7/16/2010 8/16/2010 8000 7/14/2010 10/26/2010 9000 ... (3 Replies)
Discussion started by: suresh_target
3 Replies

6. Shell Programming and Scripting

Compare Multiple Columns in one file

Hello guys, I am quite new to Shell Scripting and I need help for this I have a CSV file like this: Requisition,Order,RequisitionLineNumber,OrderLineNumber REQ1,Order1,1,1 REQ1,Order1,1,3 REQ2,Order2,1,5 Basically what I want to do is compare the first 3 fields If all 3 fields are the same... (5 Replies)
Discussion started by: jeffreybsu
5 Replies

7. Shell Programming and Scripting

script to compare two columns in a file

Dear everyone, I need any sort of shell script or perl script would do the following. I have a txt file as follows: ;Stretnumber Resident Resdient (not in file) 16 John Mary 16 Mary Parker 16 Nancy Smith 16 Mary John 18 Trey ... (5 Replies)
Discussion started by: sasharma
5 Replies

8. UNIX for Dummies Questions & Answers

To compare first two columns in an excel file

Hi All, i have a excel sheet with two columns as below. column1 column2 100 100 200 300 300 400 400 400 500 600 i need to compare the values these two columns and the output should be printed in the third column...if these values are equal the output should be green and if these... (2 Replies)
Discussion started by: arunmanas
2 Replies

9. Shell Programming and Scripting

Compare selected columns from a file and print difference

I have learned file comparison from my previous post here. Then, it is comparing the whole line. Now, i have a new problem. I have two files with 3 columns separated with a "|". What i want to do is to compare the second and third column of file 1, and the second and third column of file 2. And... (4 Replies)
Discussion started by: kingpeejay
4 Replies

10. Shell Programming and Scripting

compare file columns

I need help in file comparision. I have two files in below format: FILE_A: ------- COL1 COL2 COL3 COL4 COL5 FILE_B: ------- COL1A COL1B COL1C COL1D COL1E i want to compare for a for each row in FILE_A and FILE_B COL1 of FILE_A with COL1B of FILE_B COL3 of FILE_A with COL1E of... (1 Reply)
Discussion started by: learnoutmore99
1 Replies
Login or Register to Ask a Question