Sponsored Content
Top Forums Shell Programming and Scripting find duplicate string in many different files Post 302713399 by vgersh99 on Wednesday 10th of October 2012 03:17:47 PM
Old 10-10-2012
something along these lines to search for a 'string' - not tested - should get you started.
Code:
awk -v str2find='SVELGQGYEQY'  '{
  fileA[$1]=($1 in fileA)?fileA[$1] FS FILENAME:FILENAME
  freq[$1,FILENAME]=$2
}
END {
  if ( str2find in fileA) {
     print "string", "file", "frequency"
     n=split(fileA[str2find], tmp, FS)
     for (i=1;i<=n;i++)
        print str2find, tmp[i], freq[str2find,tmp[i]]
  }
}' my100filesGohere


Last edited by vgersh99; 10-10-2012 at 04:28 PM..
This User Gave Thanks to vgersh99 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to find duplicate files with find ?

hello all I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies

2. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

3. Shell Programming and Scripting

Remove duplicate files based on text string?

Hi I have been struggling with a script for removing duplicate messages from a shared mailbox. I would like to search for duplicate messages based on the “Message-ID” string within the messages files. I have managed to find the duplicate “Message-ID” strings and (if I would like) delete... (1 Reply)
Discussion started by: spangberg
1 Replies

4. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

5. Shell Programming and Scripting

Find duplicate files by file size

Hi! I want to find duplicate files (criteria: file size) in my download folder. I try it like this: find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt; cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies

6. Shell Programming and Scripting

Find duplicate files but with different extensions

Hi ! I wonder if anyone can help on this : I have a directory: /xyz that has the following files: chsLog.107.20130603.gz chsLog.115.20130603 chsLog.111.20130603.gz chsLog.107.20130603 chsLog.115.20130603.gz As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies

7. Shell Programming and Scripting

Find duplicate rows between files

Hi champs, I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order. I have tried using the comm command, but its not working for my scenario. Input file1 srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies

8. Shell Programming and Scripting

Find help in shell - that clears away duplicate files

I am so frustrated!!! I want a nice command that clears away duplicate files: find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##' should work in my opinion. But it finds nothing even though I have files that have the file name: Scooby-Doo-1.txt Himalaya-2.jpg Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies

9. Shell Programming and Scripting

To Find Duplicate files using latest in Linux

I have tried the following code and with that i couldnt achieve what i want. #!/usr/bin/bash find ./ -type f \( -iname "*.xml" \) | sort -n > fileList sed -i '/\.\/fileList/d' fileList NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g') sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies

10. UNIX for Advanced & Expert Users

AIX find duplicate backup files

I would like find and delete old backup files in aix. How would I go about doing this? For example: server1_1-20-2020 server1_1-21-2020 server1_1-22-2020 server1_1-23-2020 server2_1-20-2020 server2_1-21-2020 server2_1-22-2020 server2_1-23-2020 How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies
aha(1)								 Ansi HTML Adapter							    aha(1)

NAME
aha - Ansi HTML Adapter SYNOPSIS
aha [options] [-f datei] DESCRIPTION
aha takes SGR-colored Input and prints W3C conform HTML-Code. aha reads the Input from a file or stdin and writes HTML-Code to stdout. OPTIONS
--help , -h , -? A help like this --black , -b Black Background and white "standard color" --pink , -p Pink Background --iso X , -i X Uses ISO 8859-X instead of utf-8. X must be 1..16 --title X , -t X Gives the html output the title --line-fix , -l Uses a fix for inputs using control sequences to change the cursor position like htop. It's a hot fix, it may not work with any pro- gram like htop. (See EXAMPLE) EXAMPLE
aha --help | aha --black --title "the awesome aha help"> aha-help.htm Creates an HTML file with the help of aha with black background colordiff oldfile.c newfile.c | aha > colordiff.htm Creates an HTML file with a colorful diff-output of two files "oldfile.c" and "newfile.c" with white background ls --color=always | aha --pink > ls.htm Creates an HTML file with a colorful ls-output with pink background. echo a | htop | aha --black --line-fix > htop.htm Creates an HTML file with the output of htop. You have to use --line-fix due the other new-line-commands htop uses. AUTHOR
Copyleft Alexander Matthes aka Ziz 2011 zizsdl@googlemail.com SEE ALSO
http://ziz.delphigl.com/tool_aha.php August 31, 2011 aha(1)
All times are GMT -4. The time now is 04:13 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy