09-04-2006
pipe output from find -type f through basename (in a script only) and then through uniq -d
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I need a perl script which will create an output file after comparing two diff file in a directory path:
/export/home/abc/file1
/export/home/abc/file2
File Format: <IP>TAB<DeviceName><TAB>DESCRIPTIONS
file1:
10.1.2.1.3<tab>abc123def<tab>xyz.mm1.ppp.... (2 Replies)
Discussion started by: ricky007
2 Replies
2. Shell Programming and Scripting
I have a directory with images:
-rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg
-rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg
-rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg
-rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg
-rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies
3. Shell Programming and Scripting
What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies
4. Shell Programming and Scripting
Hi!
I want to find duplicate files (criteria: file size) in my download folder.
I try it like this:
find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt;
cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies
5. Shell Programming and Scripting
I have more than 100 files like this:
SVEAVLTGPYGYT 2
SVEGNFEETQY 10
SVELGQGYEQY 28
SVERTGTGYT 6
SVGLADYNEQF 21
SVGQGYEQY 32
SVKTVLGYEQF 2
SVNNEQF 12
SVRDGLTNSPLH 3
SVRRDREGLEQF 11
SVRTSGSYEQY 17
SVSVSGSPLQETQY 78
SVVHSTSPEAF 59
SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies
6. Shell Programming and Scripting
Hi !
I wonder if anyone can help on this : I have a directory: /xyz that has the following files:
chsLog.107.20130603.gz
chsLog.115.20130603
chsLog.111.20130603.gz
chsLog.107.20130603
chsLog.115.20130603.gz
As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies
7. Shell Programming and Scripting
Hi champs,
I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order.
I have tried using the comm command, but its not working for my scenario.
Input file1
srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies
8. Shell Programming and Scripting
I am so frustrated!!!
I want a nice command that clears away duplicate files:
find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##'
should work in my opinion. But it finds nothing even though I have files that have the file name:
Scooby-Doo-1.txt
Himalaya-2.jpg
Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies
9. Shell Programming and Scripting
I have tried the following code and with that i couldnt achieve what i want.
#!/usr/bin/bash
find ./ -type f \( -iname "*.xml" \) | sort -n > fileList
sed -i '/\.\/fileList/d' fileList
NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g')
sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies
10. UNIX for Advanced & Expert Users
I would like find and delete old backup files in aix. How would I go about doing this? For example:
server1_1-20-2020
server1_1-21-2020
server1_1-22-2020
server1_1-23-2020
server2_1-20-2020
server2_1-21-2020
server2_1-22-2020
server2_1-23-2020
How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies
uniq(1) User Commands uniq(1)
NAME
uniq - report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-f fields] [-s char] [ input_file [output_file]]
uniq [-c | -d | -u] [-n] [ + m] [ input_file [output_file]]
DESCRIPTION
The uniq utility will read an input file comparing adjacent lines, and write one copy of each input line on the output. The second and suc-
ceeding copies of repeated adjacent input lines will not be written.
Repeated lines in the input will not be detected if they are not adjacent.
OPTIONS
The following options are supported:
-c Precedes each output line with a count of the number of times the line occurred in the input.
-d Suppresses the writing of lines that are not repeated in the input.
-f fields Ignores the first fields fields on each input line when doing comparisons, where fields is a positive decimal integer. A
field is the maximal string matched by the basic regular expression:
[[:blank:]]*[^[:blank:]]*
If fields specifies more fields than appear on an input line, a null string will be used for comparison.
-s chars Ignores the first chars characters when doing comparisons, where chars is a positive decimal integer. If specified in con-
junction with the -f option, the first chars characters after the first fields fields will be ignored. If chars specifies
more characters than remain on an input line, a null string will be used for comparison.
-u Suppresses the writing of lines that are repeated in the input.
-n Equivalent to -f fields with fields set to n.
+m Equivalent to -s chars with chars set to m.
OPERANDS
The following operands are supported:
input_file A path name of the input file. If input_file is not specified, or if the input_file is -, the standard input will be used.
output_file A path name of the output file. If output_file is not specified, the standard output will be used. The results are unspeci-
fied if the file named by output_file is the file named by input_file.
EXAMPLES
Example 1: Using the uniq command
The following example lists the contents of the uniq.test file and outputs a copy of the repeated lines.
example% cat uniq.test
This is a test.
This is a test.
TEST.
Computer.
TEST.
TEST.
Software.
example% uniq -d uniq.test
This is a test.
TEST.
example%
The next example outputs just those lines that are not repeated in the uniq.test file.
example% uniq -u uniq.test
TEST.
Computer.
Software.
example%
The last example outputs a report with each line preceded by a count of the number of times each line occurred in the file:
example% uniq -c uniq.test
2 This is a test.
1 TEST.
1 Computer.
2 TEST.
1 Software.
example%
ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of uniq: LANG, LC_ALL, LC_CTYPE, LC_MES-
SAGES, and NLSPATH.
EXIT STATUS
The following exit values are returned:
0 Successful completion.
>0 An error occurred.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWesu |
+-----------------------------+-----------------------------+
|CSI |Enabled |
+-----------------------------+-----------------------------+
|Interface Stability |Standard |
+-----------------------------+-----------------------------+
SEE ALSO
comm(1), pack(1), pcat(1), sort(1), uncompress(1), attributes(5), environ(5), standards(5)
SunOS 5.10 20 Dec 1996 uniq(1)