I have 2 files. file1 contains unique files, the 1st field being the FileID and the 2nd is the FileName. File2 contains the timestamp, operation type, FileName, and FileSize respectively.
Basically, what I need to do is to match the filenames of each file. If they match, I need to create a new column in file2 that stores the FileID (taken from 1st col from file1).
Basically, the resulting file2 should be this (new column is in the front):
I will be running this on very large files (upwards of 900,000-1,000,000 lines in file2) and (around 5000 lines in file1). So I need it to run as fast as possible. I've been struggling with this one, so I hope someone can help.
Thank you in advance!
I was thinking that maybe I could sort file2 by the 3rd column (cat file2 | sort -k 3) first. Then compare col2 in file1 with col3 in file2. Here is the pseudocode that I haven't been able to implement in awk.
Sort col3 in file2 in ascending order (cat file2 | sort -k 3)
Code:
while (i <= lastline_file2) {
if (file2.col3 == file1.col2)
file2.newcol = file1.col1
i++ //increment line in file2
else
j++ //increment line in file1
}
Sort file2 back by timestamp
//Assuming the new column created in file2 was at the beginning
Code:
cat file2 | sort -k 2
Maybe there is a better way, but I'm not that great at awk yet.
Thanks!
Last edited by Scott; 06-20-2010 at 08:07 AM..
Reason: Code tags, please...
I've two files with data like below:
file1.txt:
AAA,Apples,123
BBB,Bananas,124
CCC,Carrot,125
file2.txt:
Store1|AAA|123|11
Store2|BBB|124|23
Store3|CCC|125|57
Store4|DDD|126|38
So,the field separator in file1.txt is a comma and in file2.txt,it is |
Now,the output should be... (2 Replies)
Hi friends,
My file is like:
Second file is :
I need to print the rows present in file one, but in order present in second file....I used
while read gh;do
awk ' $1=="' $gh'" {print >> FILENAME"output"} ' cat listoffirstfile
done < secondfile
but the output I am... (14 Replies)
Hi,
I'm a new user in awk and i'm trying to compare two files to create a third one if some values match in both files.
The first file has this content:
s 45.960746365 _21_ AGT 2490 [21:0 22:0
s 45.980418496 _21_ AGT 2491 [21:0 22:0
s 46.000090627 _21_ AGT 2492 [21:0 22:0
s 47.906552206... (2 Replies)
Dear Gurus,
I am very new to UNIX. I appreciate your help to manage my files.
I have 16 files with equal number of columns in it. Each file has 9 columns separated by space. I need to compare the values in the second column of first file and obtain the corresponding value in the 9th column... (12 Replies)
i have one file say file1 having many records.Each record contains 2000 characters.i have to compare 192-200 (stored as name)characters in this file from other file say file2 having name stored in 1-9 characters.
after comparing i have to print the record from file1 in another file say file3 ... (3 Replies)
Hi guys,
I'm rather new at using UNIX based systems, and when it comes to scripting etc I'm even newer.
I have two files which i need to compare.
file1: (some random ID's)
451245
451288
136588
784522
file2: (random ID's + e-mail assigned to ID)
123888 xc@xc.com
451245 ... (21 Replies)
I've been trying to use awk to compare two files that have pretty much the same data in apart from certain lines where in one file a fields value has changed. I want to print the line from the first file and the changed line from the second file.
At the moment, all I can get it to do is print the... (6 Replies)
Hello,
I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^
Here is what I'm trying to achieve :
file1:
test filea 3495;
test fileb 4578;
test filec 7689;
test filey 9978;
test filez 12300;
file2:
test filea... (11 Replies)
Hi,
I have two text files containing records in following format:
file1 format is:
name1 age1 nickname1 path1
name2 age2 nickname2 path2
file 1 example is:
abcd 13 abcd.13 /home/temp/abcd.13
efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
I have a log file which is continuously added to, called log.file. I'd like to
monitor this file, and when certain lines are found, update some totals in
another file. I've played around with tail -f, grep, and awk, but can't seem
to hit the right note, so to speak.
The lines I'm... (0 Replies)
06-19-2010
