04-30-2009
awk program for file comparison
Hello there,
I'm trying to write an awk program in bash shell with the following three input files:
File 1
1001 1 2 3
1002 4 5 6
1003 7 8 9
1004 10 11 12
File 2
1001 11 22 33
1002 44 55 66
1004 100 111 122
File 3
1001 111 222 333
1004 130 141 152
I would like to compare the first field of File3 with the first field of File 2. If a particular first field in File 2 doesn't exist in File 3, then I want to delete that entire row in File 2.
For example, first field on Row 2 of File 2 (=1002) doesn't exist in File 3. Hence I want to delete this row in File 2 and save it.
Now I want to compare File 3 and File 1. We see that the first field of second row (=1002) and the first field of the third row (=1003) of File 1 doesn't exist in File 3. I want to delete these two rows from File 1 and save it.
After the above steps, the three files would result as the following:
File 1
1001 1 2 3
1004 10 11 12
File 2
1001 11 22 33
1004 100 111 122
File 3
1001 111 222 333
1004 130 141 152
Any help is greatly appreciated.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have some .dat files that I cannot open and read the data. It is an awk program file, and my question would be to you all is there a way to convert this awk file to ascii text?
Thanks (10 Replies)
Discussion started by: ryangfm
10 Replies
2. Shell Programming and Scripting
- I am looking for different kind of awk solution which I don't think is mentioned before in these forums.
Number of rows in the file are fixed
Their are two columns in file1.txt
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
I am looking for 3... (1 Reply)
Discussion started by: softwarekids23
1 Replies
3. Shell Programming and Scripting
I have two files which I would like to compare and then manipulate in a way.
File1:
pictures.txt 1.1 1.3
dance.txt 1.2 1.4
treehouse.txt 1.3 1.5
File2:
pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244
dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies
4. Shell Programming and Scripting
Hello all,
Probably a very simple question, I am stuck with a small part of a code:
I am trying to do a comparison to get the maximum value of column 6 if columns 1, 4 and 5 of two or more rows match. Here is what I am doing:
awk -F'\t' '{if ($6 > a)a=$6}END{for (i in a) print i"\t"a}'
... (4 Replies)
Discussion started by: jaysean
4 Replies
5. Shell Programming and Scripting
Hi all,
I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example
File1
ABA
ABC
ABE
ABF
File 2
ABA 123
ABB 124
ABD 125
ABC 126
So what I would like printed to a... (6 Replies)
Discussion started by: pcg
6 Replies
6. Shell Programming and Scripting
Hello All,
I am having a below requirement.
File1 contains
KEY|VIN|SEQUENCE|COST
101 | XXX111 | 1 | 234.22
234 | XXX111 | 2 | 134.32
444 | ABC234 | 1 | 100.22
555 | DFF611 | 1 | 734.82
FILE 2 Contains only VINs
XXX111
DFF611
Now if the VIN from file 1 is present in... (8 Replies)
Discussion started by: dinesh1985
8 Replies
7. Shell Programming and Scripting
Hello,
I couldn't find anything on the Forum that would help me to solve this problem. Could any body help me process below data using awk?
I have got two files:
file1:
Worker1: Thomas
Position: Manager
Department: Sales
Salary: $5,000
Worker2: Jason
Position: ... (5 Replies)
Discussion started by: killerbee
5 Replies
8. Shell Programming and Scripting
my files are as follows
fileA sepearated by tab /t
00 lieferungen
00 attractiop
01 done
02 forness
03 rasp
04 alwaysisng
04 funny
05 done1
fileB
funnymou120112
funnymou234470
mou3raspnhdhv
rddfgmoudone1438748
so all those record which are greater than 3 and which are not... (6 Replies)
Discussion started by: rajniman
6 Replies
9. Shell Programming and Scripting
Hi All,
i have two files file1 ,file 2
file 1
col1|col2|col3|col4|col5|col6|col7|col8
11346925|0|2009-09-20|9999-12-31|100|0
11346925|0|2009-09-20|9999-12-31|120|0
12954311|0|2009-09-11|9999-12-31|100|0
12954311|0|2009-07-23|2999-12-31|120|0
12954312|0|2009-09-11|9999-12-31|100|0... (9 Replies)
Discussion started by: mohanalakshmi
9 Replies
10. Shell Programming and Scripting
Hi Guys,
I am having two requirement in one of my scripts. please help out to find a fast solution using AWK (since there is lot of data to be processed)
1) First snippet -
File1 has two columns and file2 has three columns
If any value of column 1 of file1 matches with column 1... (4 Replies)
Discussion started by: stormfield
4 Replies
LEARN ABOUT FREEBSD
xo_close_list_hd
LIBXO(3) BSD Library Functions Manual LIBXO(3)
NAME
xo_emit -- emit formatted output based on format string and arguments
LIBRARY
library ``libxo''
SYNOPSIS
#include <libxo/xo.h>
LIBXO(3) BSD Library Functions Manual LIBXO(3)
NAME
xo_open_list
xo_open_list_h
xo_open_list_hd
xo_open_list_d
xo_open_instance
xo_open_instance_h
xo_open_instance_hd
xo_open_instance_d
xo_close_instance
xo_close_instance_h
xo_close_instance_hd
xo_close_instance_d
xo_close_list
xo_close_list_h
xo_close_list_hd
xo_close_list_d -- open and close lists and instances
LIBRARY
library ``libxo''
SYNOPSIS
int
xo_open_list_h(xo_handle_t *xop, const char *name);
int
xo_open_list(const char *name);
int
xo_open_list_hd(xo_handle_t *xop, const char *name);
int
xo_open_list_d(const char *name);
int
xo_open_instance_h(xo_handle_t *xop, const char *name);
int
xo_open_instance(const char *name);
int
xo_open_instance_hd(xo_handle_t *xop, const char *name);
int
xo_open_instance_d(const char *name);
int
xo_close_instance_h(xo_handle_t *xop, const char *name);
int
xo_close_instance(const char *name);
int
xo_close_instance_hd(xo_handle_t *xop);
int
xo_close_instance_d(void);
int
xo_close_list_h(xo_handle_t *xop, const char *name);
int
xo_close_list(const char *name);
int
xo_close_list_hd(xo_handle_t *xop);
int
xo_close_list_d(void);
DESCRIPTION
Lists are sequences of instances of homogeneous data objects. Two distinct levels of calls are needed to represent them in our output
styles. Calls must be made to open and close a list, and for each instance of data in that list, calls must be make to open and close that
instance.
The name given to all calls must be identical, and it is strongly suggested that the name be singular, not plural, as a matter of style and
usage expectations.
A list is a set of one or more instances that appear under the same parent. The instances contain details about a specific object. One can
think of instances as objects or records. A call is needed to open and close the list, while a distinct call is needed to open and close
each instance of the list:
xo_open_list("item");
for (ip = list; ip->i_title; ip++) {
xo_open_instance("item");
xo_emit("{L:Item} '{:name/%s}':0, ip->i_title);
xo_close_instance("item");
}
xo_close_list("item");
Getting the list and instance calls correct is critical to the proper generation of XML and JSON data.
EXAMPLE:
xo_open_list("user");
for (i = 0; i < num_users; i++) {
xo_open_instance("user");
xo_emit("{k:name}:{:uid/%u}:{:gid/%u}:{:home}0,
pw[i].pw_name, pw[i].pw_uid,
pw[i].pw_gid, pw[i].pw_dir);
xo_close_instance("user");
}
xo_close_list("user");
TEXT:
phil:1001:1001:/home/phil
pallavi:1002:1002:/home/pallavi
XML:
<user>
<name>phil</name>
<uid>1001</uid>
<gid>1001</gid>
<home>/home/phil</home>
</user>
<user>
<name>pallavi</name>
<uid>1002</uid>
<gid>1002</gid>
<home>/home/pallavi</home>
</user>
JSON:
user: [
{
"name": "phil",
"uid": 1001,
"gid": 1001,
"home": "/home/phil",
},
{
"name": "pallavi",
"uid": 1002,
"gid": 1002,
"home": "/home/pallavi",
}
]
LEAF LISTS
In contrast to a list of instances, a "leaf list" is list of simple values. To emit a leaf list, call the xo_emit() function using the ""l""
modifier:
for (ip = list; ip->i_title; ip++) {
xo_emit("{Lwc:Item}{l:item}0, ip->i_title);
}
The name of the field must match the name of the leaf list.
In JSON, leaf lists are rendered as arrays of values. In XML, they are rendered as multiple leaf elements.
JSON:
"item": "hammer", "nail"
XML:
<item>hammer</item>
<item>nail</item>
ADDITIONAL DOCUMENTATION
Complete documentation can be found on github:
http://juniper.github.io/libxo/libxo-manual.html
libxo lives on github as:
https://github.com/Juniper/libxo
The latest release of libxo is available at:
https://github.com/Juniper/libxo/releases
SEE ALSO
xo_emit(3)
HISTORY
The libxo library was added in FreeBSD 11.0.
AUTHOR
Phil Shafer
BSD
December 4, 2014 BSD