What is your input like? Is order important? uniq requires lines in sorted order.
If your input data is small enough to be held in main memory, this may do:
But if your input data is larger than a few handfuls of megs, this is inefficient, you may have to sort it with
which will sort it and remove duplicates.
Hello, I have a file with a list of car makes and specific information for each make. An example is:
@Audi:Warranty
@Audi:Pricing
@Audi:Colors
@Acura:Warranty
@Acura:Pricing
@Acura:Colors
and so on through a bunch of makes. I need to make a list in a selection box of just one name of... (12 Replies)
Hi ,
Can You Please let Know How use unix uniq command on a single column for deleting records from file
with Below Structure.Pipe Delimter File .
Source
Name | Account_Id
A | 101
B... (2 Replies)
Dear Friends,
I want to know if it is possible to perform addition as part of uniq command.
Input:
213.64.56.208 1
213.64.56.208 2
213.64.56.208 3
213.46.27.204 10
213.46.27.204 20
213.46.27.204 30
The above input should be converted as follows
213.64.56.208 6
213.46.27.204 60
... (4 Replies)
Hello
i have tried to retrieve a distinct list of data from a file with a command combination but the uniq command line is not filtering the fields repeated.
Can someone help me ?
here is the file input
3837734|LAUNCH TEST01
3837735|LAUNCH TEST01
3837736|LAUNCH TEST01
3837737|LAUNCH... (4 Replies)
Platform :Oracle Linux 6.4
Shell : bash
The below file has 7 lines , some of them are duplicates. There are only 3 distinct lines. But why is the uniq command still showing 7 ?
I just want the distinct lines to be returned.
$ cat test.txt
SELECT FC.COORD_SET_ID FROM OM_ORDER_FLOW F, -... (2 Replies)
Discussion started by: kraljic
2 Replies
LEARN ABOUT OPENDARWIN
uniq
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e. the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e. the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
DIAGNOSTICS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD June 6, 1993 BSD