Sponsored Content
Top Forums Shell Programming and Scripting Awk+Grep Input file needs to match a column and print the entire line Post 302297513 by poliver on Friday 13th of March 2009 04:22:52 PM
Old 03-13-2009
Awk+Grep Input file needs to match a column and print the entire line

I'm having problems since few days ago, and i'm not able to make it works with a simple awk+grep script (or other way to do this).

For example, i have a input file1.txt:

cat inputfile1.txt

218299910417
1172051195
1172070231
1172073514
1183135117
1183135118
1183135119
1281440202


And i need to match these numbers, on another file on two specific columns, for example the $3 and $4 column, using the pipe delimiter

cat inputfile2.txt

AAAAA|DISTHOR1_U2|6981258207|218299910417|END
BBBBB|DISTHOR1_U2|6981118022|6981259131|END
FARFAR|DISTHOR1_U2|6981119404|1172070231|END
CCCCC|DISTHOR1_U2|1172073514|6981258793|END
BBBBB|DISTHOR1_U2|698515487|489498131|END

The expected result, is a output file that matches the elements from the first file, with the third and forth column from the second file, in this case, the output file will be:

cat outputfile1.txt

AAAAA|DISTHOR1_U2|6981258207|218299910417|END
FARFAR|DISTHOR1_U2|6981119404|1172070231|END
CCCCC|DISTHOR1_U2|1172073514|6981258793|END

I was able to do this, with this command, but he is looking for the whole file, not a specific column:

grep -f inputfile1.txt inputfile2.txt > outputfile1.txt

But this command is taking over an hour, because my input file1.txt has over 1600 records and the inputfile2.txt has over one million of records with 190 characters on each line, divided in 43 columns

Can someone help me with this?

Thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Print entire line based on value in a column

Friends, File1.txt abc|0|xyz 123|129|opq def|0|678 890|pqw|sdf How do I print the entire line where second column has value is 0? Expected Result: abc|0|xyz def|0|678 Thanks, Prashant ---------- Post updated at 02:14 PM ---------- Previous update was at 02:06 PM ---------- ... (1 Reply)
Discussion started by: ppat7046
1 Replies

2. Shell Programming and Scripting

grep N lines after match and then print them on 1 line each

Hello I need some help with this job. file.txt ----- cut ---- TARGET 13/11/08 20:43:21 POINT 1 MOVE 8 772102y64312417771 TARGET 13/11/08 21:10:01 POINT 2 MOVE 5 731623jjd12njhd ----- cut ---- this is the example. i need to grep for the word TARGET and print next 4 lines like... (1 Reply)
Discussion started by: alekkz
1 Replies

3. UNIX for Dummies Questions & Answers

grep N lines after match and then print them on 1 line each

Hello I have a silly question. I need to grep a match in text file and then print 5 lines after it. grep -A 5 .... do it. OK The next thing I can not handle is I need each output to be on 1 line match line2 line3 line4 line5 match line2 line3 line4 line5 etc.. I will really... (10 Replies)
Discussion started by: alekkz
10 Replies

4. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

5. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

How to print the entire line if the mentioned match is found?

Hello Everyone, I have a file with 5 fields in each line just like mentioned below. Also the 4th field is time elapsed(hh:mm:ss) since the process is running xyz abc status 23:00:00 idle abc def status 24:00:00 idle def gji status 27:00:02 idle fgh gty status 00:00:00 idle Here I... (8 Replies)
Discussion started by: rahul2662
8 Replies

7. Shell Programming and Scripting

Input file needs to match a column and print the entire line

I have a file with class c IP addresses that I need to match to a column and print the matching lines of another file. I started playing with grep -if file01.out file02.out but I am stuck as to how to match it to a column and print the matching lines; cat file01.out 10.150.140... (5 Replies)
Discussion started by: lewk
5 Replies

8. Shell Programming and Scripting

Print next line beside preceding line on column match

Hi, I have some data like below: John 254 Chris 254 Matt 123 Abe 123 Raj 487 Moh 487 How can i print it using awk to have: 254 John,Chris 123 Matt,Abe 487 Raj,Moh Thanks. (4 Replies)
Discussion started by: james2009
4 Replies

9. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies

10. UNIX for Beginners Questions & Answers

Compare 1st column from 2 file and if match print line from 1st file and append column 7 from 2nd

hi I have 2 file with more than 10 columns for both 1st file apple,0,0,0...... orange,1,2,3..... mango,2,4,5..... 2nd file apple,2,3,4,5,6,7... orange,2,3,4,5,6,8... watermerlon,2,3,4,5,6,abc... mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies
Template::Plugin::Table(3pm)				User Contributed Perl Documentation			      Template::Plugin::Table(3pm)

NAME
Template::Plugin::Table - Plugin to present data in a table SYNOPSIS
[% USE table(list, rows=n, cols=n, overlap=n, pad=0) %] [% FOREACH item IN table.row(n) %] [% item %] [% END %] [% FOREACH item IN table.col(n) %] [% item %] [% END %] [% FOREACH row IN table.rows %] [% FOREACH item IN row %] [% item %] [% END %] [% END %] [% FOREACH col IN table.cols %] [% col.first %] - [% col.last %] ([% col.size %] entries) [% END %] DESCRIPTION
The "Table" plugin allows you to format a list of data items into a virtual table. When you create a "Table" plugin via the "USE" directive, simply pass a list reference as the first parameter and then specify a fixed number of rows or columns. [% USE Table(list, rows=5) %] [% USE table(list, cols=5) %] The "Table" plugin name can also be specified in lower case as shown in the second example above. You can also specify an alternative variable name for the plugin as per regular Template Toolkit syntax. [% USE mydata = table(list, rows=5) %] The plugin then presents a table based view on the data set. The data isn't actually reorganised in any way but is available via the "row()", "col()", "rows()" and "cols()" as if formatted into a simple two dimensional table of "n" rows x "n" columns. So if we had a sample "alphabet" list contained the letters '"a"' to '"z"', the above "USE" directives would create plugins that represented the following views of the alphabet. [% USE table(alphabet, ... %] rows=5 cols=5 a f k p u z a g m s y b g l q v b h n t z c h m r w c i o u d i n s x d j p v e j o t y e k q w f l r x We can request a particular row or column using the "row()" and "col()" methods. [% USE table(alphabet, rows=5) %] [% FOREACH item = table.row(0) %] # [% item %] set to each of [ a f k p u z ] in turn [% END %] [% FOREACH item = table.col(2) %] # [% item %] set to each of [ m n o p q r ] in turn [% END %] Data in rows is returned from left to right, columns from top to bottom. The first row/column is 0. By default, rows or columns that contain empty values will be padded with the undefined value to fill it to the same size as all other rows or columns. For example, the last row (row 4) in the first example would contain the values "[ e j o t y undef ]". The Template Toolkit will safely accept these undefined values and print a empty string. You can also use the IF directive to test if the value is set. [% FOREACH item = table.row(4) %] [% IF item %] Item: [% item %] [% END %] [% END %] You can explicitly disable the "pad" option when creating the plugin to returned shortened rows/columns where the data is empty. [% USE table(alphabet, cols=5, pad=0) %] [% FOREACH item = table.col(4) %] # [% item %] set to each of 'y z' [% END %] The "rows()" method returns all rows/columns in the table as a reference to a list of rows (themselves list references). The "row()" methods when called without any arguments calls "rows()" to return all rows in the table. Ditto for "cols()" and "col()". [% USE table(alphabet, cols=5) %] [% FOREACH row = table.rows %] [% FOREACH item = row %] [% item %] [% END %] [% END %] The Template Toolkit provides the "first", "last" and "size" virtual methods that can be called on list references to return the first/last entry or the number of entries in a list. The following example shows how we might use this to provide an alphabetical index split into 3 even parts. [% USE table(alphabet, cols=3, pad=0) %] [% FOREACH group = table.col %] [ [% group.first %] - [% group.last %] ([% group.size %] letters) ] [% END %] This produces the following output: [ a - i (9 letters) ] [ j - r (9 letters) ] [ s - z (8 letters) ] We can also use the general purpose "join" virtual method which joins the items of the list using the connecting string specified. [% USE table(alphabet, cols=5) %] [% FOREACH row = table.rows %] [% row.join(' - ') %] [% END %] Data in the table is ordered downwards rather than across but can easily be transformed on output. For example, to format our data in 5 columns with data ordered across rather than down, we specify "rows=5" to order the data as such: a f . . b g . c h d i e j and then iterate down through each column (a-e, f-j, etc.) printing the data across. a b c d e f g h i j . . . Example code to do so would be much like the following: [% USE table(alphabet, rows=3) %] [% FOREACH cols = table.cols %] [% FOREACH item = cols %] [% item %] [% END %] [% END %] Output: a b c d e f g h i j . . . In addition to a list reference, the "Table" plugin constructor may be passed a reference to a Template::Iterator object or subclass thereof. The Template::Iterator get_all() method is first called on the iterator to return all remaining items. These are then available via the usual Table interface. [% USE DBI(dsn,user,pass) -%] # query() returns an iterator [% results = DBI.query('SELECT * FROM alphabet ORDER BY letter') %] # pass into Table plugin [% USE table(results, rows=8 overlap=1 pad=0) -%] [% FOREACH row = table.cols -%] [% row.first.letter %] - [% row.last.letter %]: [% row.join(', ') %] [% END %] AUTHOR
Andy Wardley <abw@wardley.org> <http://wardley.org/> COPYRIGHT
Copyright (C) 1996-2007 Andy Wardley. All Rights Reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Template::Plugin perl v5.14.2 2011-12-20 Template::Plugin::Table(3pm)
All times are GMT -4. The time now is 01:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy