If original data is not sorted, the uniq command will give strange results. You may need to sort the data prior to a uniq command.
I manually placed a few of the last entries at the beginning of the file, and see the results:
hi
i have a very simple problem
iam moving files from download to archive folder
but before such a transfer want to make sure no two file of same
are present in my download directory
how to check for redundant file names
i thought of using WC but it counts inside the file (lines and... (5 Replies)
I am really really new to Unix. I'm lost with so many books around for different shell. I'm thinking of taking a course on Operating Systems but it contains a lot of Unix programming I think. For example, someone was talking about a "which" command. But I wasn't able to figure out what it does...... (10 Replies)
Hi,
I am stuck on writing a script that reads a file, retains wanted data but discards unwanted data from a CSV file.
This is a sample of my table {not the actual data)
ID Color tel_num Name color2 color3
abcdef green 5551212 jj88 red blue
acbdfe yellow... (5 Replies)
My ZFS on debian media server just died in a power outage, the zpool status shows this:
NAME STATE READ WRITE CKSUM
tank UNAVAIL 0 0 0 insufficient replicas
raidz1 UNAVAIL 0 0 0 corrupted data
sda ONLINE 0 0 0
sdb ONLINE 0 0 0
sdf ONLINE 0 0 0
sdh ONLINE 0 0 0
sdi ONLINE 0 0 0
sdk ONLINE 0... (2 Replies)
I want to find which pattern or strings have occurred more than one time so that I can remove unnecessary redundancy.
For example:
If I have the sentence:
A quick brown brown fox jumps jumps jumps over the lazy dog
in a file, then I want to know that
1. the word "brown" has... (7 Replies)
Hi All,
We have a redundant server system and every time we need any change in any particular script then we need to do it individually in each of the server. I want to create a shell script such that if i make changes in primary server and run the script then it should update the changes to ... (0 Replies)
Dear All,
I have to reduce the redundancy of a file that is like this:
a b 0
a c 0
a f 1
b a 1
b a 0
b c 1
d f 0
g h 1
f d 1
Basically, this file describe a network with relative nodes and edges.
The nodes are the different letters and the edges are represented by the numbers (in... (7 Replies)
In a huge log file (43MB, 43k lines) I am trying to extract data between two tag pairs on same line and export it to a file so I can pull it into Excel for a report.
One Pair is <Text>data I need</Text>
Other pair follows on same line and is <TimeStamp>more data I need</TimeStamp>
I would need... (2 Replies)
I have data file that has this in it:
data.txt
.........
.........
PPJ97**2017PPJ97**2017-03-21-13.35.15.887208********************START ERROR LOGGING******************
PPJ97**2017-03-21-13.35.15.887208** PROMPT APPLICATION ERROR **
PPJ97**2017-03-21-13.35.15.887208** IN TIMESTAMP |... (1 Reply)
Discussion started by: SkySmart
1 Replies
LEARN ABOUT FREEBSD
uniq
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e., the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e., the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of uniq as described in environ(7).
EXIT STATUS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'') as amended by Cor. 1-2002.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD December 17, 2009 BSD