Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Filter lines common in two files Post 302830431 by latsyrc on Monday 8th of July 2013 07:06:30 PM
Old 07-08-2013
Filter lines common in two files

Thanks everyone. I got that problem solved.

I require one more help here. (Yes, UNIX definitely seems to be fun and useful, and I WILL eventually learn it for myself. But I am now on a different project and don't really have time to go through all the basics. So, I will really appreciate some help.)

I've got two data files.
One has: ID lat lon data1 data2; for ex:
Code:
1001   23.5  -6.45  3.2  14.68
1002   48.2  -35.6  8.5  21.67
1003   -5.6   23.6   3.5  3.56
...
...

And the other has: ID data3 data4 data5
For example:
Code:
1001   C   16   US
1002   D    32   US
1004   E    13   US
...
...


There are approximately 2500 IDs but neither has all of them. file1 has few missing, so does file2, but the missing IDs are not necessarily common.

Now I want to make a new file with: ID lat lon data4 ; but ONLY for the IDs that are common in both.
So, for above files it would be:

Code:
1001   23.5  -6.45  16
1002   48.2  -35.6  32
...
...

I searched the forums and there are similar problems which are solved using awk but I could not comprehend the scripts well enough to create my own solution.
Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To find all common lines from 'n' no. of files

Hi, I have one situation. I have some 6-7 no. of files in one directory & I have to extract all the lines which exist in all these files. means I need to extract all common lines from all these files & put them in a separate file. Please help. I know it could be done with the help of... (11 Replies)
Discussion started by: The Observer
11 Replies

2. Shell Programming and Scripting

Drop common lines at head/tail of a large set of files

Hi! I have a large set of pairs of text files (each pair in their own subdirectory) and each pair shares head/tail (a couple of first and last lines) but differs in the middle part. I need to delete the heads/tails and keep only the middle portions in which they differ. The lengths of heads/tails... (1 Reply)
Discussion started by: dobryden
1 Replies

3. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 3rd column.(tab separated columns) Sample input: file1: 111 222 0.1 333 444 0.5 555 666 0.4 file 2: 111 222 0.7 555 666... (5 Replies)
Discussion started by: jaysean
5 Replies

4. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 5th column.(tab separated columns) . 3rd and 4th columns corresponds to the row which has highest value for the 5th column. Sample... (2 Replies)
Discussion started by: jaysean
2 Replies

5. Shell Programming and Scripting

Get common lines from multiple files

FileA chr1 31237964 NP_001018494.1 PUM1 M340L chr1 31237964 NP_055491.1 PUM1 M340L chr1 33251518 NP_037543.1 AK2 H191D chr1 33251518 NP_001616.1 AK2 H191D chr1 57027345 NP_001004303.2 C1orf168 P270S FileB chr1 ... (9 Replies)
Discussion started by: genehunter
9 Replies

6. Shell Programming and Scripting

Find common lines between multiple files

Hello everyone A few years Ago the user radoulov posted a fancy solution for a problem, which was about finding common lines (gene variation names) between multiple samples (files). The code was: awk 'END { for (R in rec) { n = split(rec, t, "/") if (n > 1) dup = dup ?... (5 Replies)
Discussion started by: bibb
5 Replies

7. Shell Programming and Scripting

Finding out the common lines in two files using 4 fields with the help of awk and UNIX

Dear All, I have 2 files. If field 1, 2, 4 and 5 matches in both file1 and file2, I want to print the whole line of file1 and file2 one after another in my output file. File1: sc2/80 20 . A T 86 F=5;U=4 sc2/60 55 . G T ... (1 Reply)
Discussion started by: NamS
1 Replies

8. Shell Programming and Scripting

Find common lines with one file and with all of the files in another folder

Hi! I would like to comm -12 with one file and with all of the files in another folder that has a 100 files or more (that file is not in that folder) to find common text lines. I would like to have each case that they have common lines to be written to a different output file and the names of the... (6 Replies)
Discussion started by: Eve
6 Replies

9. Shell Programming and Scripting

Find common lines between all of the files in one folder

Could it be possible to find common lines between all of the files in one folder? Just like comm -12 . So all of the files two at a time. I would like all of the outcomes to be written to a different files, and the file names could be simply numbers - 1 , 2 , 3 etc. All of the file names contain... (19 Replies)
Discussion started by: Eve
19 Replies

10. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches... (2 Replies)
Discussion started by: beca123456
2 Replies
LIBXO(3)						   BSD Library Functions Manual 						  LIBXO(3)

NAME
xo_emit -- emit formatted output based on format string and arguments LIBRARY
library ``libxo'' SYNOPSIS
#include <libxo/xo.h> LIBXO(3) BSD Library Functions Manual LIBXO(3) NAME
xo_open_list xo_open_list_h xo_open_list_hd xo_open_list_d xo_open_instance xo_open_instance_h xo_open_instance_hd xo_open_instance_d xo_close_instance xo_close_instance_h xo_close_instance_hd xo_close_instance_d xo_close_list xo_close_list_h xo_close_list_hd xo_close_list_d -- open and close lists and instances LIBRARY
library ``libxo'' SYNOPSIS
int xo_open_list_h(xo_handle_t *xop, const char *name); int xo_open_list(const char *name); int xo_open_list_hd(xo_handle_t *xop, const char *name); int xo_open_list_d(const char *name); int xo_open_instance_h(xo_handle_t *xop, const char *name); int xo_open_instance(const char *name); int xo_open_instance_hd(xo_handle_t *xop, const char *name); int xo_open_instance_d(const char *name); int xo_close_instance_h(xo_handle_t *xop, const char *name); int xo_close_instance(const char *name); int xo_close_instance_hd(xo_handle_t *xop); int xo_close_instance_d(void); int xo_close_list_h(xo_handle_t *xop, const char *name); int xo_close_list(const char *name); int xo_close_list_hd(xo_handle_t *xop); int xo_close_list_d(void); DESCRIPTION
Lists are sequences of instances of homogeneous data objects. Two distinct levels of calls are needed to represent them in our output styles. Calls must be made to open and close a list, and for each instance of data in that list, calls must be make to open and close that instance. The name given to all calls must be identical, and it is strongly suggested that the name be singular, not plural, as a matter of style and usage expectations. A list is a set of one or more instances that appear under the same parent. The instances contain details about a specific object. One can think of instances as objects or records. A call is needed to open and close the list, while a distinct call is needed to open and close each instance of the list: xo_open_list("item"); for (ip = list; ip->i_title; ip++) { xo_open_instance("item"); xo_emit("{L:Item} '{:name/%s}':0, ip->i_title); xo_close_instance("item"); } xo_close_list("item"); Getting the list and instance calls correct is critical to the proper generation of XML and JSON data. EXAMPLE: xo_open_list("user"); for (i = 0; i < num_users; i++) { xo_open_instance("user"); xo_emit("{k:name}:{:uid/%u}:{:gid/%u}:{:home}0, pw[i].pw_name, pw[i].pw_uid, pw[i].pw_gid, pw[i].pw_dir); xo_close_instance("user"); } xo_close_list("user"); TEXT: phil:1001:1001:/home/phil pallavi:1002:1002:/home/pallavi XML: <user> <name>phil</name> <uid>1001</uid> <gid>1001</gid> <home>/home/phil</home> </user> <user> <name>pallavi</name> <uid>1002</uid> <gid>1002</gid> <home>/home/pallavi</home> </user> JSON: user: [ { "name": "phil", "uid": 1001, "gid": 1001, "home": "/home/phil", }, { "name": "pallavi", "uid": 1002, "gid": 1002, "home": "/home/pallavi", } ] LEAF LISTS
In contrast to a list of instances, a "leaf list" is list of simple values. To emit a leaf list, call the xo_emit() function using the ""l"" modifier: for (ip = list; ip->i_title; ip++) { xo_emit("{Lwc:Item}{l:item}0, ip->i_title); } The name of the field must match the name of the leaf list. In JSON, leaf lists are rendered as arrays of values. In XML, they are rendered as multiple leaf elements. JSON: "item": "hammer", "nail" XML: <item>hammer</item> <item>nail</item> ADDITIONAL DOCUMENTATION
Complete documentation can be found on github: http://juniper.github.io/libxo/libxo-manual.html libxo lives on github as: https://github.com/Juniper/libxo The latest release of libxo is available at: https://github.com/Juniper/libxo/releases SEE ALSO
xo_emit(3) HISTORY
The libxo library was added in FreeBSD 11.0. AUTHOR
Phil Shafer BSD
December 4, 2014 BSD
All times are GMT -4. The time now is 03:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy