Sponsored Content
Top Forums Shell Programming and Scripting awk - Matching columns between 2 files and reordering results Post 302464561 by ambroze on Wednesday 20th of October 2010 12:14:58 PM
Old 10-20-2010
awk - Matching columns between 2 files and reordering results

I am trying to match 4 colums (first_name,last_name,dob,ssn) between 2 files and when there is an exact match I need to write out these matches to a new file with a combination of fields from file1 and file2. I've managed to come up with a way to match these 2 files based on the columns (see below) but I'm not sure how to control which columns from either file1 or file2 I can output to a final results file.

Code:
nawk -F "|" 'FNR==NR{ x[$2$3$5$6]; next} $1$2$4$3 in x' file1 file2

Here's an example of my files below.

file1: Min List
Code:
id|first_name|last_name|gender|ssn|dob
10000|Maria|Bellow|F|111-15-1111|01/21/1957
10001|Brian|Hala|M|123-45-6789|01/21/1940
10002|Amy|Jones|F|123-45-6789|01/19/1972
10003|Jimmy|Thomas|M|222-45-2222|05/25/1952
10004|John|Smith|M||02/22/1970
10005|Atkins|Lucent|F|123-45-6789|02/27/1955
10006|Shawn|Batman|F|123-45-6789|03/21/1936
10007|Karen|Rockefeller|F|123-45-6789|02/11/1957

file2: Master List
Code:
first_name|last_name|dob|ssn|value1|value2|value3|value4|value5|value6|value7

Maria|Bellow|01/21/1957|111-15-1111|value1|value2|value3|value4|value5|value6|value7
Jimmy|Thomas|05/25/1952|222-45-2222|value1|value2|value3|value4|value5|value6|value7Maria|Bellow|01/21/1957|111-15-1111|value1|value2|value3|value4|value5|value6|value7
Jimmy|Thomas|05/25/1952|222-45-3333|value1|value2|value3|value4|value5|value6|value7
James|Thomas|01/21/1957|123-45-6789|value1|value2|value3|value4|value5|value6|value7

file3: Results List
Code:
id|value1|value2|last_name|first_name|value3|value4|value5|value6|value7
10000|value1|value2|Bellow|Maria|value3|value4|value5|value6|value7
10000|value1|value2|Bellow|Maria|value3|value4|value5|value6|value7
10003|value1|value2|Thomas|Jimmy|value3|value4|value5|value6|value7

Notice that the "id" column from file1 is in file3 the results file and all others are from file2 in different columns. Any help on this is greatly appreciated.

Last edited by Scott; 10-20-2010 at 01:23 PM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

matching columns from two files

Hey, I have two files that have exactly the same format. They are both tab-delimited and contain 12 columns. However the # of rows vary. What I want to do is match columns # 5,6 and 7 between the two files. If they do match exactly (based on numbers) then I want the whole row from file 2 to... (1 Reply)
Discussion started by: phil_heath
1 Replies

2. UNIX for Dummies Questions & Answers

Matching corresponding columns in two different files

Hi to all, I have two separated files: FILE1 "V1" "V2" "V3" Mary James Nicole Robert Francisco Sophie Nancy Antony Matt Josephine Louise Rose Mark Simon Charles FILE2 "V1" "V2" "V3"... (2 Replies)
Discussion started by: eleonoral
2 Replies

3. Shell Programming and Scripting

Help with awk Matching columns from two files

Hello, I have two files as following: #bin chrom chromStart chromEnd name score strand observed 585 chr2 29442 29443 rs4637157 0 + C/T 585 chr2 33011 33012 rs13423995 0 + A/G 585 chr2 34502 34503 rs13386087 0 + ... (2 Replies)
Discussion started by: Homa
2 Replies

4. Shell Programming and Scripting

Merge two files matching columns

Hi! I need to merge two files when col1 (x:x:x) matching and adds second column from file1.txt. # cat 1.txt aaa;a12 bbb;b13 ccc;c33 ddd;d55 eee;e11 # cat 2.txt bbb;b55;34444;d55 aaa;a15;35666;a44 I try with this awk and I get succesfully first column from 1.txt: # awk -F";"... (2 Replies)
Discussion started by: fhluque
2 Replies

5. Shell Programming and Scripting

Join two files with matching columns

Hi, I need to join two files together with one common value in a column. I think I can use awk or join or a combination but I can't quite get it. Basically my data looks like this, with the TICKER columns matching up in each file File1 TICKER,column 1, column, 2, column, 3, column 4 ... (6 Replies)
Discussion started by: unkleruckus
6 Replies

6. Shell Programming and Scripting

awk to copy previous line matching a particular columns

Hello Help, 2356798 7689867 999 000 123678 20385907 9797 666 17978975 87468976 968978 98798 I am trying to have out put which actually look for the third column value of 9797 and then it insert line there after with first, second column value exactly as the previous line and replace the third... (3 Replies)
Discussion started by: Indra2011
3 Replies

7. Shell Programming and Scripting

awk - matching on 2 columns for differents lines

Given this file (I separated them in block to make my explanation clearer): 92157768877;Sof_deme_Fort_Email_am_%yyyy%%mm%%dd%;EMAIL;20/02/2015;1;0;0 92157768877;Sof_trav_Fort_Email_am_%yyyy%%mm%%dd%;EMAIL;20/02/2015;1;0;0 91231838895;Sof_deme_faible_Email_am;EMAIL;26/01/2015;1 0;0... (1 Reply)
Discussion started by: Andy_K
1 Replies

8. Shell Programming and Scripting

awk merge matching columns

I know I'm not the first one asking this but my code still does not work: File 1: gi|1283| tRNAscan exon 87020 88058 . - . transcript_id "Parent=tRNA-Tyr5.r01"; gi|3283| tRNAscan exon 97020 97058 . + . transcript_id "Parent=tRNA-Tyr6.r01"; gi|4283| rRNAscan exon 197020 197058 . - . transcript_id... (5 Replies)
Discussion started by: 0sMoses
5 Replies

9. Shell Programming and Scripting

Joining Two Files Matching Two Columns

Hi All, I am looking to join two files where column 1 of file A matches with column 1 of file B and column 5 of files A matches with column 2 of file B. After joining the files based on above condition, out should contain entire line of file A and column 3, 4 and 5 of file B. Here is sample... (8 Replies)
Discussion started by: angshuman
8 Replies

10. Shell Programming and Scripting

awk Matching Columns - Am I missing something?

I am using awk to match columns and output based on those matches. For some reason it is not printing matching columns, am I missing something? Operating system - windows with cygwin. Command that I am using: sed 's/]*,]*/,/g' $tempdir/file1 > $tempdir/file1.$$ && awk -F, 'FNR==NR{f2=$2... (7 Replies)
Discussion started by: dis0wned
7 Replies
QwtScaleArithmetic(3)						 Qwt User's Guide					     QwtScaleArithmetic(3)

NAME
QwtScaleArithmetic - Arithmetic including a tolerance. SYNOPSIS
#include <qwt_scale_engine.h> Static Public Member Functions static double ceil125 (double x) static double ceilEps (double value, double intervalSize) static int compareEps (double value1, double value2, double intervalSize) static double divideEps (double interval, double steps) static double floor125 (double x) static double floorEps (double value, double intervalSize) Detailed Description Arithmetic including a tolerance. Member Function Documentation double QwtScaleArithmetic::ceil125 (doublex) [static] Find the smallest value out of {1,2,5}*10^n with an integer number n which is greater than or equal to x Parameters: x Input value double QwtScaleArithmetic::ceilEps (doublevalue, doubleintervalSize) [static] Ceil a value, relative to an interval Parameters: value Value to ceil intervalSize Interval size See also: floorEps() int QwtScaleArithmetic::compareEps (doublevalue1, doublevalue2, doubleintervalSize) [static] Compare 2 values, relative to an interval. Values are 'equal', when : $ Parameters: value1 First value to compare value2 Second value to compare intervalSize interval size Returns: 0: if equal, -1: if value2 > value1, 1: if value1 > value2 double QwtScaleArithmetic::divideEps (doubleintervalSize, doublenumSteps) [static] Divide an interval into steps. $stepSize = (intervalSize - intervalSize * 10e^{-6}) / numSteps$.PP Parameters: intervalSize Interval size numSteps Number of steps Returns: Step size double QwtScaleArithmetic::floor125 (doublex) [static] Find the largest value out of {1,2,5}*10^n with an integer number n which is smaller than or equal to x. Parameters: x Input value double QwtScaleArithmetic::floorEps (doublevalue, doubleintervalSize) [static] Floor a value, relative to an interval Parameters: value Value to floor intervalSize Interval size See also: floorEps() Author Generated automatically by Doxygen for Qwt User's Guide from the source code. Version 5.2.2 Mon Aug 1 2011 QwtScaleArithmetic(3)
All times are GMT -4. The time now is 05:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy