Ubuntu

Hello,

I have a 1.6 GB file that I would like to modify by matching some ids in col1 with the ids in col 1 of file2.txt and save the results into a 3rd file.

For example:

File 1 has 1411 rows, I ignore how many columns it has (thousands)
File 2 has 311 rows, 1 column

Would like to create

File 3 with 311 rows (thousands of columns)

What is the fastest way to do this without consuming too much memory?

Thank you!

Fastest way is syncsort but i dont know if you would have that....
then try grep. dont use awk.

I used this:

grep -A1 -A1 -f file1.txt file2 > file3

but it is taking forever and I don't know if it is going to be correct at the end
I don't know what -A1 -A1 mean (I'm assuming that is col1 File1 col1 File2)

Help please!

give some sample input of both the files
and desired output, and
conditions how the two files will be joined.
PS:- Use code tags

Both files have no headings

input of file 1 (has one 1 column, as shown below):

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364



input of file 2 (this file has 2,498,588 columns with single digit numbers, starting with column 1 as shown below, each column is separated by a space)

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364
.
.
.
.
.
.
.
.
.
.
.
MXY9423 <--- row #1411


desired output file 3 (with only #364 rows with the ids matched between file1 and file2 and 2,498,588 columns)

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364

Thank you for any help!

---------- Post updated at 11:10 PM ---------- Previous update was at 11:03 PM ----------

I just checked the results I obtained with grep -A1 -A1 -f file1.txt file2 > file3

and they are wrong. Instead of getting only 364 rows, I get 367 and some of the ids of file 1 are missing in the output file 3. I want to match the ids from file1 (my "golden" list) in file2 and output that in file 3

Both files have no headings

input of file1.txt (has one 1 column, as shown below):

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364



input of file2.ped (this file has more than 2 million columns with single digit numbers, starting with column 1 as shown below, each column is separated by a space)

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364
.
.
.
.
.
.
.
.
.
.
.
MXY9423 <--- row #1411


desired output file 3 (with only #364 rows with the ids matched between file1 and file2 and 2,498,588 columns)

MXY2344
MXY2455
.
.
.
.
.
.
.
MXY9150 <--- row #364

Thank you for any help!

---------- Post updated at 11:10 PM ---------- Previous update was at 11:03 PM ----------

I used grep -A1 -A1 -f file1.txt file2 > file3 but that did not work.

I only got one reply for this thread yesterday saying to use grep, so that's why I'm posting this again in hopes somebody would help.

Thank you!

If you want to grep the data from file2 which are present in file1

Code:

grep -f file1 file2 > file3
or
awk 'FILENAME=="file1"{A[$0]=$0}
FILENAME=="file2"{if(A[$1]==$1){print}}' file1 file2 > file3