08-30-2007
file Lookup using awk
Hi All,
I have two files file1 and file2(lookup file).I need to map more than one keyfields of file1 with file2.how can we achieve it using awk.
file1(max 2.2 million records)
--------------------------
680720|680721|077
680720|680721|978
680721|680722|090
file2(no idea about the record count)
---------------------------------------
680721|A|680720
680722|B|680721
Assumptions:
1)file 2 will not have any duplicate records.
2)no idea about the number of records.
Keyfields:1,2 for file1 and 3,1 for file2
Output
---------
680720|680721|077|A
680720|680721|978|A
680721|680722|090|B
Please help me how to go about.
I can try with arrays but as file2 record is not fixed, i cant take in array.please suggest me.
regards,
sukumar
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Write a quick shell snippet to find all of the IPV4 IP addresses
in any and all of the files under /var/lib/output/*, ignoring
whatever else may be in those files. Perform a reverse lookup on
each, and format the output neatly, like "IP=192.168.0.1,
... (0 Replies)
Discussion started by: choco4202002
0 Replies
2. Shell Programming and Scripting
I want to lookup filea with fileb,filec and filed.
If entry in filea exist in fileb and filec mark Y and then if entry in filea exist in filed mark as Y.
Final output should have all the entries from filea.
This prints only matching entries from file a in fileb i want all entries from... (9 Replies)
Discussion started by: pinnacle
9 Replies
3. UNIX for Dummies Questions & Answers
I have the below 2 files:
1) Third field from file1.txt should be compared to the first field of lookup.txt.
2) If match found then third field, file1.txt should be substituted with the second field from lookup.txt.
3)Else just print the line from file1.txt.
File1.txt:... (4 Replies)
Discussion started by: venalla_shine
4 Replies
4. Shell Programming and Scripting
have written a combined sed+awk to perform a lookup operation which works but looking to enhance it.
looking to match a record using any of the comma separated values + return selected fields from the record - including the field header. so:
cat foo
make,model,engine,trim,value... (6 Replies)
Discussion started by: jack.bauer
6 Replies
5. Shell Programming and Scripting
I need to add new ID to the file with old ID (column 7), I collected old ID / new ID pairs in a lookup file and I am trying to use awk to do the job, but something is not clicking.
My input file
ABC| 107|1440589221| -118.117167| 33.986333|10| 497476|1
ABC| 125|1440591215| -118.181000| ... (4 Replies)
Discussion started by: migurus
4 Replies
6. Shell Programming and Scripting
The below awk uses $3 and $4 in search as the min and max, then takes each $2 value in lookup and compares it. If the value in lookupfalls within the range in searchthen it prints the entire line in lookup/ICODE]. What I can't seem to figure out is how to print the matching $5 from search on that... (4 Replies)
Discussion started by: cmccabe
4 Replies
7. Shell Programming and Scripting
In the below, I am trying to lookup $1 and $2 from file1, in a range search using $1 $2 $3 of file2. If the search key from file1 is found in file2, then the word low is printed in the last field of that line in the updated file1. Only the last section of file1 needs to be searched, but I am not... (6 Replies)
Discussion started by: cmccabe
6 Replies
8. Shell Programming and Scripting
I am trying to update the below awk, kindly provided by @RavinderSingh13, to update each line of file1 with either Low or No Low based on matching $2 of file1 to a range in $2 and $3 of file2. If the $2 value in file1 matches the range in file2 then that line is Low, otherwise it is No Low in the... (3 Replies)
Discussion started by: cmccabe
3 Replies
9. Shell Programming and Scripting
The bash bash below extracts the oldest folder from a directory and stores it in filename
That result will match a line in bold in input. In the matching line there is an_xxx digit in italics that
(once the leading zero is removed) will match a line in link. That is the lint to print in output.... (2 Replies)
Discussion started by: cmccabe
2 Replies
10. Shell Programming and Scripting
I use conditional operators alot in AWK to print rows from large text files based on values in a certain column. For example:
awk -F '\t' '{ if ($1 == "A" || $1 == "C" ) print $0}' OFS="\t" file1.txt > file2.txt
In this case every row is printed from file1 to file2 for which the column 1... (5 Replies)
Discussion started by: Geneanalyst
5 Replies
LEARN ABOUT DEBIAN
combine
COMBINE(1) COMBINE(1)
NAME
combine - combine sets of lines from two files using boolean operations
SYNOPSIS
combine file1 and file2
combine file1 not file2
combine file1 or file2
combine file1 xor file2
_ file1 and file2 _
_ file1 not file2 _
_ file1 or file2 _
_ file1 xor file2 _
DESCRIPTION
combine combines the lines in two files. Depending on the boolean operation specified, the contents will be combined in different ways:
and Outputs lines that are in file1 if they are also present in file2.
not Outputs lines that are in file1 but not in file2.
or Outputs lines that are in file1 or file2.
xor Outputs lines that are in either file1 or file2, but not in both files.
"-" can be specified for either file to read stdin for that file.
The input files need not be sorted, and the lines are output in the order they occur in file1 (followed by the order they occur in file2
for the two "or" operations). Bear in mind that this means that the operations are not commutative; "a and b" will not necessarily be the
same as "b and a". To obtain commutative behavior sort and uniq the result.
Note that this program can be installed as "_" to allow for the syntactic sugar shown in the latter half of the synopsis (similar to the
test/[ command). It is not currently installed as "_" by default, but you can alias it to that if you like.
SEE ALSO
join(1)
AUTHOR
Copyright 2006 by Joey Hess <joey@kitenet.net>
Licensed under the GNU GPL.
moreutils 2012-04-09 COMBINE(1)