10-10-2012
find duplicate string in many different files
I have more than 100 files like this:
HTML Code:
SVEAVLTGPYGYT 2
SVEGNFEETQY 10
SVELGQGYEQY 28
SVERTGTGYT 6
SVGLADYNEQF 21
SVGQGYEQY 32
SVKTVLGYEQF 2
SVNNEQF 12
SVRDGLTNSPLH 3
SVRRDREGLEQF 11
SVRTSGSYEQY 17
SVSVSGSPLQETQY 78
SVVHSTSPEAF 59
SVVPGNGYT 75
There is a string in $1 and its frequency in $2.
I have two questions. How can I merge these file into one file, which include all the string in $1 and each frequency in different fields?
How can I find the same string included in the 100 files, and output its each frequency?
I can do this using awk between two files, but failed to deal with so many.
Thank you!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hello all
I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies
2. Shell Programming and Scripting
I have a directory with images:
-rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg
-rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg
-rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg
-rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg
-rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies
3. Shell Programming and Scripting
Hi
I have been struggling with a script for removing duplicate messages from a shared mailbox.
I would like to search for duplicate messages based on the “Message-ID” string within the messages files.
I have managed to find the duplicate “Message-ID” strings and (if I would like) delete... (1 Reply)
Discussion started by: spangberg
1 Replies
4. Shell Programming and Scripting
What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies
5. Shell Programming and Scripting
Hi!
I want to find duplicate files (criteria: file size) in my download folder.
I try it like this:
find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt;
cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies
6. Shell Programming and Scripting
Hi !
I wonder if anyone can help on this : I have a directory: /xyz that has the following files:
chsLog.107.20130603.gz
chsLog.115.20130603
chsLog.111.20130603.gz
chsLog.107.20130603
chsLog.115.20130603.gz
As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies
7. Shell Programming and Scripting
Hi champs,
I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order.
I have tried using the comm command, but its not working for my scenario.
Input file1
srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies
8. Shell Programming and Scripting
I am so frustrated!!!
I want a nice command that clears away duplicate files:
find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##'
should work in my opinion. But it finds nothing even though I have files that have the file name:
Scooby-Doo-1.txt
Himalaya-2.jpg
Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies
9. Shell Programming and Scripting
I have tried the following code and with that i couldnt achieve what i want.
#!/usr/bin/bash
find ./ -type f \( -iname "*.xml" \) | sort -n > fileList
sed -i '/\.\/fileList/d' fileList
NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g')
sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies
10. UNIX for Advanced & Expert Users
I would like find and delete old backup files in aix. How would I go about doing this? For example:
server1_1-20-2020
server1_1-21-2020
server1_1-22-2020
server1_1-23-2020
server2_1-20-2020
server2_1-21-2020
server2_1-22-2020
server2_1-23-2020
How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies
YAGI(5) File Formats Manual YAGI(5)
NAME
yagi - yagi binary output format
DESCRIPTION
This manual page describes the output format of the file created by the Yagi-Uda project's yagi program. The file is a binary file with a
header of 100 bytes. The header consists of:
offset(bytes) information data-type
0 elements int
4 driven int
8 parasitic int
12 min_frequency double
20 max_frequency double
28 design frequency double
36 step_frequency double
44 angular_step double
52 0 double
92 0 double
Following the header, there starts the coordinates(x,y,l) of each
element and the voltage applied to each element. Then we write
the element currents one after the other, at each frequency.
Hence the element currents will be written 12,000 times with a
3 element yagi analysed at 4,000 different frequencies. All complex
data is stored in a structure of type fcomplex, defined as:
struct fcomplex {double real, double imaginary};
offset(bytes) information data-type
100 x1 double /* coordinates */
108 y1 double
116 l1 double
100+(n-1)*24 x_n double
100+(n-1)*24+8 y_n double
100+(n-1)*24+16 l_n double
-------------- Repeated for each element.
100+24*elements Voltage1 fcomplex /* voltage */
100+24*elements+(n-1)*16 Voltage_n fcomplex
-------------- Repeated for each element n at each frequency step f.
100+40*elements Current1@f1 fcomplex /* current */
100+40*elements+(n-1)*16 Current_n@f1 fcomplex
100+56*elements Current@f2 fcomplex
100+40*elements+(n-1)*16 Current_n@f2 fcomplex
FILES
filename.out binary file
SEE ALSO
yagi(1), output(1), input(1), optimise(1), first(1) and input(5).
AUTHORS
Dr. David Kirkby G8WRB (david.kirkby@onetel.net), with help with converting to DOS from Dr. Joe Mack NA3T (mack@fcrfv2.ncifcrf.gov).
Yagi version 1.16 24th October 2000 YAGI(5)