Searching for similar row(s) across multiple files
Hello Esteemed Members,
I need to write a script to search for files that have one or more than one rows similar.
Please note that there is no specific pattern that I am searching for. The rows can be different, I just need to find out two or more similar records in two or more files.
There are around 5000 such files that I need to search amongst.
The files are scattered in same directory but different sub-directories.
Yes, I understand Chirel, so what is the workaround?
What if I append the contents of all the files into one file, sort the records and remove the unique records from the resulting file.
Then I need to back track all the rows that have multiple instances and find out what are the original files to which they belong.
Will this work, if yes, then how do I write the same?
prefix each line of all file with "filename : ", then sort from field 3 to end
then uniq -d on the result ignoring the x chars at the start of lines.
The output will be the something like
filename1 : pattern
filename2 : pattern
etc
But i'm having issue with the uniq -d part.
Actually i have
Edit: well no issue, it's working like a charm
Last edited by Chirel; 07-25-2012 at 06:51 AM..
Reason: Working
I am new to unix and I would like to search multiple log files to find earliest occurrence of a text.
Ex:
Say I have 10 logs file each ending with .log and I want to find the text “CustomeError” .
I want to find the which log file “CustomeError” comes first and lines which surround’s ... (4 Replies)
Hi all,
I am sure some gurus will find a better way of doing this. FYI, I've manually modified some of the data 'coz they are somewhat confidential, so there may be some typo errors.
At the moment, I have 3 files that I am trying to search for. Sometime in the future, it may go beyond 3... (2 Replies)
I'm attempting to use grep in Perl with very little success.
What I would like to do in Perl is get the output of the following grep code:
grep -l 'pattern' *
This gives me a list of all the files in a directory that contain the pattern that was searched.
My attempts to do this in Perl... (4 Replies)
Hi all,
I am pretty new at this so be gentle. Also, if there is any chance you could explain what the code you use is actually doing, that would really help me out, Im learning after all :)
So I am trying to convert a selected column of numbers from input file1 into a row in output file2
... (3 Replies)
I have a list of pattern in a file, I want each of these pattern been searched from 4 files. I was wondering this can be done in SED / AWK.
say my 4 files to be searched are
> cat f1
abc/x(12) 1
abc/x 3
cde 2
zzz 3
fdf 4
> cat f2
fdf 4
cde 3
abc 2... (6 Replies)
Hi All,
I have a issue in pulling some heavy records , I have my input file has 10,000 records which i need to compare with daily appended log files from (sep 1st 2009 to till date) . I tried to use grep fgrep and even sed , but the as time is factor for me , i cannot wait for 5 days to get the... (3 Replies)
I have a situation where I need to search for multiple strings (error messages) such as 'aborted' 'file not found' etc in directory having logs. I have put all the error messages in a text file and using the command.
grep -f <textfile> <filetobegrepped>
I'm doing this thru a script where I... (5 Replies)
I would like a simple shell script that will allow me to display to screen all unsuccessful su attempts in my sulog file, for the present date.
I have been trying several different combinations of commands, but I can't quite get the syntax correct.
The mess I have right now (don't laugh) is... (4 Replies)
I am using a DEC ALPHA running Digital UNIX (formly DEC OSF/1) and ksh. I have a directory with hundreds of files that only share the extension .rpt. I would like to search that directory based on serial number and operation number and only files that meet both requirements to be printed out. I... (6 Replies)