03-13-2009
Awk+Grep Input file needs to match a column and print the entire line
I'm having problems since few days ago, and i'm not able to make it works with a simple awk+grep script (or other way to do this).
For example, i have a input file1.txt:
cat inputfile1.txt
218299910417
1172051195
1172070231
1172073514
1183135117
1183135118
1183135119
1281440202
And i need to match these numbers, on another file on two specific columns, for example the $3 and $4 column, using the pipe delimiter
cat inputfile2.txt
AAAAA|DISTHOR1_U2|6981258207|218299910417|END
BBBBB|DISTHOR1_U2|6981118022|6981259131|END
FARFAR|DISTHOR1_U2|6981119404|1172070231|END
CCCCC|DISTHOR1_U2|1172073514|6981258793|END
BBBBB|DISTHOR1_U2|698515487|489498131|END
The expected result, is a output file that matches the elements from the first file, with the third and forth column from the second file, in this case, the output file will be:
cat outputfile1.txt
AAAAA|DISTHOR1_U2|6981258207|218299910417|END
FARFAR|DISTHOR1_U2|6981119404|1172070231|END
CCCCC|DISTHOR1_U2|1172073514|6981258793|END
I was able to do this, with this command, but he is looking for the whole file, not a specific column:
grep -f inputfile1.txt inputfile2.txt > outputfile1.txt
But this command is taking over an hour, because my input file1.txt has over 1600 records and the inputfile2.txt has over one million of records with 190 characters on each line, divided in 43 columns
Can someone help me with this?
Thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Friends,
File1.txt
abc|0|xyz
123|129|opq
def|0|678
890|pqw|sdf
How do I print the entire line where second column has value is 0?
Expected Result:
abc|0|xyz
def|0|678
Thanks,
Prashant
---------- Post updated at 02:14 PM ---------- Previous update was at 02:06 PM ----------
... (1 Reply)
Discussion started by: ppat7046
1 Replies
2. Shell Programming and Scripting
Hello
I need some help with this job.
file.txt
----- cut ----
TARGET
13/11/08
20:43:21
POINT 1
MOVE 8
772102y64312417771
TARGET
13/11/08
21:10:01
POINT 2
MOVE 5
731623jjd12njhd
----- cut ----
this is the example.
i need to grep for the word TARGET and print next 4 lines like... (1 Reply)
Discussion started by: alekkz
1 Replies
3. UNIX for Dummies Questions & Answers
Hello
I have a silly question. I need to grep a match in text file
and then print 5 lines after it.
grep -A 5 .... do it.
OK
The next thing I can not handle is I need each output to be on 1 line
match line2 line3 line4 line5
match line2 line3 line4 line5
etc..
I will really... (10 Replies)
Discussion started by: alekkz
10 Replies
4. Shell Programming and Scripting
Hi Experts,
I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me.
i have 2 files
file1.txt
9647810043118
9647810043126
9647810043155
9647810043161
9647810043166
9647810043185
9647810043200
9647810043203
9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies
5. Shell Programming and Scripting
Hi,
My input files is like this
axis1 0 1 10
axis2 0 1 5
axis1 1 2 -4
axis2 2 3 -3
axis1 3 4 5
axis2 3 4 -1
axis1 4 5 -6
axis2 4 5 1
Now, these are my following tasks
1. Print a first column for every two rows that has the same value followed by a string.
2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies
6. Shell Programming and Scripting
Hello Everyone,
I have a file with 5 fields in each line just like mentioned below. Also the 4th field is time elapsed(hh:mm:ss) since the process is running
xyz abc status 23:00:00 idle
abc def status 24:00:00 idle
def gji status 27:00:02 idle
fgh gty status 00:00:00 idle
Here I... (8 Replies)
Discussion started by: rahul2662
8 Replies
7. Shell Programming and Scripting
I have a file with class c IP addresses that I need to match to a column and print the matching lines of another file.
I started playing with grep -if file01.out file02.out but I am stuck as to how to match it to a column and print the matching lines;
cat file01.out
10.150.140... (5 Replies)
Discussion started by: lewk
5 Replies
8. Shell Programming and Scripting
Hi,
I have some data like below:
John 254
Chris 254
Matt 123
Abe 123
Raj 487
Moh 487
How can i print it using awk to have:
254 John,Chris
123 Matt,Abe
487 Raj,Moh
Thanks. (4 Replies)
Discussion started by: james2009
4 Replies
9. UNIX for Beginners Questions & Answers
I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file.
I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies
10. UNIX for Beginners Questions & Answers
hi
I have 2 file with more than 10 columns for both
1st file
apple,0,0,0......
orange,1,2,3.....
mango,2,4,5.....
2nd file
apple,2,3,4,5,6,7...
orange,2,3,4,5,6,8...
watermerlon,2,3,4,5,6,abc...
mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies
LEARN ABOUT DEBIAN
x2sys_solve
X2SYS_SOLVE(1gmt) Generic Mapping Tools X2SYS_SOLVE(1gmt)
NAME
x2sys_solve - Determine systematic corrections from crossovers
SYNOPSIS
x2sys_solve -Ccolumn -TTAG -Emode [ COE_list.d ] [ -V ] [ -W ] [ -Z ] [ -bi[s|S|d|D[ncol]|c[var1/...]] ]
DESCRIPTION
x2sys_solve will use the supplied crossover information to solve for systematic corrections that can then be applied per track to improve
data quality. Several systematic corrections can be solved for using a least-squares approach. Note: Only one data column can be processed
at the time.
-T Specify the x2sys TAG which tracks the attributes of this data type.
-C Specify which data column you want to process. Needed for proper formatting of the output correction table and must match the same
option used in x2sys_list when preparing the input data.
-E The correction type you wish to model. Choose among the following functions f(p), where p are the m parameters per track that we
will fit simultaneously using a least squares approach:
c will fit f(p) = a (a constant offset); records must contain cruise ID1, ID2, COE.
d will fit f(p) = a + b * d (linear drift; d is distance; records must contain cruise ID1, ID2, d1, d2, COE.
g will fit f(p) = a + b sin(y)^2 (1980-1930 gravity correction); records must contain cruise ID1, ID2, latitude y, COE.
h will fit f(p) = a + b cos(H) + c cos(2H) + d sin(H) + e sin(2H) (magnetic heading correction); records must contain cruise ID1,
ID2, heading H, COE.
s will fit f(p) = a * z (a unit scale correction); records must contain cruise ID1, ID2, z1, z2.
t will fit f(p) = a + b * (t - t0) (linear drift; t0 is the start time of the track); records must contain cruise ID1, ID2, t1-t0,
t2-t0, COE.
OPTIONS
No space between the option flag and the associated arguments.
COE_list.d
Name of file with the required crossover columns as produced by x2sys_list. NOTE: If -bi is used then the first two columns are
expected to hold the integer track IDs; otherwise we expect those columns to hold the text string names of the two tracks.
-V Selects verbose mode, which will send progress reports to stderr [Default runs "silently"].
-W Means that each input records has an extra column with the composite weight for each crossover record. These are used to obtain a
weighted least squares solution [no weights].
-Z For -Ed and -Et, determine the earliest time or shortest distance for each track, then use these values as the local origin for time
duration or distance calculations. The local origin is then included in the correction table [Default uses 0].
-bi Selects binary input. Append s for single precision [Default is d (double)]. Uppercase S or D will force byte-swapping. Option-
ally, append ncol, the number of columns in your binary input file if it exceeds the columns needed by the program. Or append c if
the input file is netCDF. Optionally, append var1/var2/... to specify the variables to be read.
EXAMPLES
To fit a simple bias offset to faa for all tracks under the MGD77 tag, try
x2sys_list COE_data.txt -V -TMGD77 -Cfaa -Fnc > faa_coe.txt
x2sys_solve faa_coe.txt -V -TMGD77 -Cfaa -Ec > coe_table.txt
To fit a faa linear drift with time instead, try
x2sys_list COE_data.txt -V -TMGD77 -Cfaa -FnTc > faa_coe.txt
x2sys_solve faa_coe.txt -V -TMGD77 -Cfaa -Et > coe_table.txt
To estimate heading corrections based on magnetic crossovers associated with the tag MGD77 from the file COE_data.txt, try
x2sys_list COE_data.txt -V -TMGD77 -Cmag -Fnhc > mag_coe.txt
x2sys_solve mag_coe.txt -V -TMGD77 -Cmag -Eh > coe_table.txt
To estimate unit scale corrections based on bathymetry crossovers, try
x2sys_list COE_data.txt -V -TMGD77 -Cdepth -Fnz > depth_coe.txt
x2sys_solve depth_coe.txt -V -TMGD77 -Cdepth -Es > coe_table.txt
SEE ALSO
x2sys_binlist(1), x2sys_cross(1), x2sys_datalist(1), x2sys_get(1), x2sys_init(1), x2sys_list(1), x2sys_put(1), x2sys_report(1)
GMT 4.5.7 15 Jul 2011 X2SYS_SOLVE(1gmt)