UNIQ(1) General Commands Manual UNIQ(1)NAME
uniq - report repeated lines in a file
SYNOPSIS
uniq [ -udc [ +n ] [ -n ] ] [ input [ output ] ]
DESCRIPTION
Uniq reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are removed;
the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found; see sort(1). If the -u flag
is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of just the repeated
lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
The n arguments specify skipping an initial portion of each line in the comparison:
-n The first n fields together with any blanks before each are ignored. A field is defined as a string of non-space, non-tab charac-
ters separated by tabs and spaces from its neighbors.
+n The first n characters are ignored. Fields are skipped before characters.
SEE ALSO sort(1), comm(1)7th Edition April 29, 1985 UNIQ(1)
Check Out this Related Man Page
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e. the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e. the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
DIAGNOSTICS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD June 6, 1993 BSD
Hi
IŽd like to get a few explanations about how the sort command works when cascading the options.
cscyabl@comet:(develop)> more file
2:A2
2:A1
5:A2
5:A2
10:A1
cscyabl@comet:(develop)> sort -n -u file
2:A1
5:A2
10:A1
cscyabl@comet:(develop)> sort -u -n file
2:A1
5:A2
10:A1... (8 Replies)
infile:
z y x
c b a
desired output:
x y z
a b c
I don't want to sort the lines into this:
a b c
x y z
nor this:
c b a
z y x
The number of fields per line and number of lines is indeterminate. The field separator is always a space.
Thanks for the use of your collective brains.... (11 Replies)
Hi everybody:
Could anybody tell me how I can delete repeated rows from a file?, this is, for exemple I have a file like this:
0.490 958.73 281.85 6.67985 0.002481
0.490 954.833 283.991 8.73019 0.002471
0.590 950.504 286.241 6.61451 0.002461
0.690 939.323 286.112 6.16451 0.00246
0.790... (8 Replies)
Hello all,
I've got a strange behaviour of sort and uniq commands: they do not recognise apparently duplicated lines in a file (already sorted). The lines are identical by eye, but they must differ in smth, because when they are put in two files, those have slightly different size.
What can make... (8 Replies)
AWK help:
I have a file with following format. I need to remove any entries which are repeated based on first 3 characters. So from the following files I need to remove any entries start with "mas".
mas01bct
mas02bct
mas03bct
mas01bct
mas01bct
mas01bct
mas11bct
mas01bct
mas01bct... (11 Replies)
I am new to Perl and in text file of around 1000 lines having around 500 repeated line which I felt is no use and want to remove these line.so can somebody help in same for providing sample code how can i remove these repeated line in a file. (11 Replies)
Hello,
I need to insert varying lines (i.e. these lines are an output of another script) between lines starting with certain fields.
An example to make it more clear.
This is the file where I wanna insert lines:
(save it as "input.txt")
ContrInMi_c_mir 2 10066 181014 200750... (12 Replies)
I've got scripts trawling the network and dumping parsed text into files with an Epoch timestamp in column 1. I append the old data to the new data then just want to keep the top entry if there is an identical duplicate below (column 1 needs to be ignored).
sort -ur +1 works a treat on a Solaris... (16 Replies)
Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it
Input
fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Hi !
I am trying to remove doubbled entrys in a textfile only between delimiters.
Like that example but i dont know how to do that with sort or similar.
input:
{
aaa
aaa
}
{
aaa
aaa
}
output:
{
aaa
}
{ (8 Replies)
Gents.
Please a help.
I have this input
I will like to count how many times is repeated the record which have always the values 14 and 98 only.
1000 1
1000 1
1001 1
1001 1
1002 98
1002 98 ... (9 Replies)
I have a requirement where I have to read the lines between a repeated string
FileName: abc.txt
ls /data/abc.txt
1
2
#ZENCO
3
4
5
6
#ZENCO
11
213
454
7
#ZENCO (8 Replies)
I have a file where every line includes four expressions with a caret in the middle (plus some other "words" or fields, always separated by spaces). I would like to extract from this file, all those lines such that each of the four expressions containing a caret appears in at least four different... (9 Replies)
Remove duplicate lines which has been repeated 4 times attached test.txt
below command tried and not getting expect output.
for i in `cat test.txt | uniq`
do
num=`cat test.txt | grep $i | wc -l`
echo $i $num
done
test.txt
... (17 Replies)