Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help in comparing multiple columns from two files.
Top Forums UNIX for Advanced & Expert Users Need help in comparing multiple columns from two files. Post 302922290 by DGPickett on Thursday 23rd of October 2014 02:41:21 PM
Old 10-23-2014
  1. The elegant solution is to use tools that make delimited files act like SQL tables and do an SQL join.
  2. You can scan one file putting the concatenated keys into an associative array (awk, bash, ksh, etc.), then process the second file looking up the keys in the array. Associative arrays are hash lookup tables, keyed with strings.
  3. A more UNIX flavored solution is to cut out the columns to compare into two files or bash/ksh pipe streams '<(...)', sort them unique and match them with 'comm'. Now you have the key values to filter the input file. Sort the input file on the same columns so matching is a simple merge.
  4. If you create modified input files with the keys copied or moved on the front and then a delimiter and sort them, you can use 'join'. Since join only handles one join column, you need a different delimiter to make it join on multiple columns. If concatenating columns, make sure each column has values formatted all to the same length.
Make sure $LC_ALL=C during sort for binary order.
Last edited by Scrutinizer; 10-24-2014 at 06:07 AM..
This User Gave Thanks to DGPickett For This Post:
sivarajb
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Comparing Columns of two FIles

Dear all, I have two files in UNIX File1 and File2 as below: File1: 1,1234,.,67.897,,0 1,4134,.,87.97,,4 0,1564,.,97.8,,1 File2: 2,8798,.,67.897,,0 2,8879,.,77.97,,4 0,1564,.,97.8,,1 I want to do the following: (1) Make sure that both the files have equal number of columns and if... (1 Reply)
Discussion started by: ggopal
1 Replies

2. Shell Programming and Scripting

Comparing Columns of two FIles

Dear all, I have two files in UNIX File1 and File2 as below: File1: 1,1234,.,67.897,,0 1,4134,.,87.97,,4 0,1564,.,97.8,,1 File2: 2,8798,.,67.897,,0 2,8879,.,77.97,,4 0,1564,.,97.8,,1 I want to do the following: (1) Make sure that both the files have equal number of columns and if... (4 Replies)
Discussion started by: ggopal
4 Replies

3. Shell Programming and Scripting

comparing 2 columns from 2 files

Hey, I have 2 files that have a name and then a number: File 1: dog 21 dog 24 cat 33 cat 27 dog 76 cat 65 File 2: dog 109 dog 248 cat 323 cat 207 cat 66 (2 Replies)
Discussion started by: dcfargo
2 Replies

4. UNIX for Dummies Questions & Answers

Comparing 2 columns from 2 files

Hi, I have two files with the same number of columns. Basically I want to print the 2 columns that match between the two files. File1 looks like this: dr12 12 6 abn dr14 12 7 abn File2 looks something like this: dr12 12 8 abn dr12 14 7 abn So basically if the first... (1 Reply)
Discussion started by: kylle345
1 Replies

5. Shell Programming and Scripting

comparing the values of repeated keys in multiple columns

Hi Guyz The 1st column of the input file has repeated keys like x,y and z. The ist task is if the 1st column has unique key (say x) and then need to consider 4th column, if it is + symbol then subtract 2nd column value with 3rd column value (we will get 2(10-8)) or if it is - symbol subtract 3rd... (3 Replies)
Discussion started by: repinementer
3 Replies

6. UNIX for Dummies Questions & Answers

Comparing columns in two files

Hi, I have two files. File1.txt has 2 columns and looks like: 458739 122345 4456 122657 34200 122600 File2.txt has many columns with column 1 the same as column2 of File1.txt, but with lot more rows: 122786 abcdefg user1@email 122778 uuhjeufh user2@email... (1 Reply)
Discussion started by: ursaan
1 Replies

7. Shell Programming and Scripting

comparing two columns from two different files

Hello, I have two files as 1.txt and 2.txt with number as columns. 1.txt 0 53.7988 1 -30.0859 2 20.1632 3 14.2135 4 14.6366 5 -37.6258 . . . 31608 -8.57333 31609 -2.58554 31610 -24.2857 2.txt (1 Reply)
Discussion started by: AKD
1 Replies

8. Shell Programming and Scripting

Comparing two columns from two different files

Hi, I have a single-column file1 having records like: 00AB01/11 43TG22/00 78RC09/34 ...... ...... and a second file , file 2 having two columns like 78RC09/34 1 45FD11/11 2 00AB01/11 3 43TG22/00 4 ...... ...... (8 Replies)
Discussion started by: amarn
8 Replies

9. Shell Programming and Scripting

awk arrays comparing multiple columns across two files.

Hi, I'm trying to use awk arrays to compare values across two files based on multiple columns. I've attempted to load file 2 into an array and compare with values in file 1, but success has been absent. If anyone has any suggestions (and I'm not even sure if my script so far is on the right lines)... (4 Replies)
Discussion started by: hubleo
4 Replies

10. Shell Programming and Scripting

Comparing multiple columns using awk

Hello All; I have two files with below conditions: 1. Entries in file A is missing in file B (primary is field 1) 2. Entries in file B is missing in file A (primary is field 1) 3. Field 1 is present in both files but Field 2 is different. Example Content: File A ... (4 Replies)
Discussion started by: mystition
4 Replies

LEARN ABOUT DEBIAN

mdb-tables

MDBTools(1)															       MDBTools(1)

NAME

       mdb-tables - Get listing of tables in an MDB database

SYNOPSIS

       mdb-tables [-S] [-1 | -d delimiter] database

DESCRIPTION

       mdb-tables is a utility program distributed with MDB Tools.

       It produces a list of tables contained within an MDB database in a format suitable for use in shell scripts.

OPTIONS

       -S     Show system tables. System tables are generally those beginning with 'MSys'

       -1     specifies that the tables should be listed 1 per line.

       -d delimiter
	      specifies an alternative delimiter. If no delimiter is specified, table names will be delimited by a tab character, or by newline if
	      the -1 option was specified.

NOTES

ENVIRONMENT

       MDB_JET3_CHARSET
	      Defines the charset of the input JET3 (access 97) file. Default is CP1252. See iconv(1).

       MDBICONV
	      Defines the output charset. Default is UTF-8. mdbtools must have been compiled with iconv.

       MDBOPTS
	      semi-column separated list of options:

	      o  use_index

	      o  no_memo

	      o  debug_like

	      o  debug_write

	      o  debug_usage

	      o  debug_ole

	      o  debug_row

	      o  debug_props

	      o  debug_all is a shortcut for all debug_* options

HISTORY

       mdb-tables first appeared in MDB Tools 0.3.

SEE ALSO

       gmdb2(1) mdb-export(1) mdb-hexdump(1) mdb-prop(1) mdb-sql(1) mdb-ver(1) mdb-array(1) mdb-header(1) mdb-parsecsv(1) mdb-schema(1)

AUTHORS

       The mdb-tables utility was written by Brian Bruns.

BUGS

       Access allows for tables to have spaces embeded in the table name.
	      You must specify a delimiter (-d) if you intend on piping the output of mdb-tables to a program such as awk or cut.

0.7								   13 July 2013 						       MDBTools(1)
All times are GMT -4. The time now is 06:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy