Faster Line by Line String/Date Comparison of 2 Files
Hello,
I was wondering if anyone knows a faster way to search and compare strings and dates from 2 files?
I'm currently using "for loop" but seems sluggish as i have to cycle through 10 directories with 10 files each containing thousands of lines.
Given:
Heres what i'm trying to achieve:
Here is my straightforward solution so far, though its working fine, its not that elegant and fast:
I was trying to replace the if statement with the following (as i have learned through research, awk is much faster in line by line processing):
But cant seem to get it to work properly. Plus, i was thinking if i can use awk as well on the parent while loop to be more efficient, but i have no idea anymore if its possible to use another awk, within an awk statement.
My goal is just to make this script work faster. Any suggestions or alternate approaches is well appreciated.
Thank you very much guys.
Last edited by agentgrecko; 07-09-2012 at 07:07 AM..
I have sifted through many past posts but have not found a good solution.
I have a 2 files that hold a filename and its cksum value like this:
FILE1 contents:
45435347589 filename1
FILE1 is from the directory structure of one host and FILE2 is generated from a rsh: host2 cksum command.
I... (1 Reply)
1. The problem statement, all variables and given/known data:
I have standard web server log file. It contains different columns (like IP address, request result code, request type etc) including a date column with the format .
I have developed a log analysis command line utility that displays... (1 Reply)
Hi folks,
I'm trying to do a bit of extra monitoring on who is sending mail through our server and as I'm sure you'll understand, the log files aren't exactly small.
As a note, I'm working to sort out a solution to this myself, so I'll keep editing the post when I get a chance.
Here an example... (2 Replies)
I get a file which has all its content in a single row.
The file contains xml data containing 3000 records, but all in a single row, making it difficult for Unix to Process the file.
I decided to insert a new line character at all occurrences of a particular string in this file (say replacing... (4 Replies)
Hi All,
I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Hi,
I wanted to check whether the x,y,z coordinates of two files are equal or not. At times, when one file is converted to another suitable file extension , there are some chances that the data mismatch would happen during the conversion. In order to avoid the data misfit, i would like to... (6 Replies)
Hi,
i have a scenario were i should compare a few fields from each line then increment a variable based on that.
Example file
989878|8999|Y|0|Y|N|V
989878|8999|Y|0|N|N|V
989878|8999|Y|2344|Y|N|V
i have 3 conditions to check and increment a variable on every line
condition 1 if ( $3... (4 Replies)
Greetings Experts,
I need to handle the views created over monthly retention tables
for which every new table in YYYYMMDD format, there is
equivalent view created and the older table which might be
dropped, the view over it has to be re-created over a dummy
table so that it doesn't fail.... (2 Replies)