I know uniq exists, but am not sure how to remove repeating lines when they are groups of two different lines repeating themselves, without using sort. I need them to be sorted in the original order, just to remove repeats.
Code:
cd /media/AUDIO/WAVE/9780743518673/mp3
~/Desktop/mp3-to-m4b 9780743518673-GOOD GERMAN "GOOD GERMAN" "Joseph Kanon" 2004
n
cd /media/AUDIO/WAVE/9780743518673/mp3
~/Desktop/mp3-to-m4b 9780743518673-GOOD GERMAN "GOOD GERMAN" "Joseph Kanon" 2004
n
cd /media/AUDIO/WAVE/9780743518673/mp3
~/Desktop/mp3-to-m4b 9780743518673-GOOD GERMAN "GOOD GERMAN" "Joseph Kanon" 2004
n
cd /media/AUDIO/WAVE/9780743518673/mp3
~/Desktop/mp3-to-m4b 9780743518673-GOOD GERMAN "GOOD GERMAN" "Joseph Kanon" 2004
n
cd /media/AUDIO/WAVE/9780743518802/mp3
~/Desktop/mp3-to-m4b 9780743518802-NEVER CHANGE "NEVER CHANGE" "Elizabeth Berg" 2004
n
cd /media/AUDIO/WAVE/9780743518802/mp3
~/Desktop/mp3-to-m4b 9780743518802-NEVER CHANGE "NEVER CHANGE" "Elizabeth Berg" 2004
n
cd /media/AUDIO/WAVE/9780743518802/mp3
~/Desktop/mp3-to-m4b 9780743518802-NEVER CHANGE "NEVER CHANGE" "Elizabeth Berg" 2004
n
cd /media/AUDIO/WAVE/9780743518802/mp3
~/Desktop/mp3-to-m4b 9780743518802-NEVER CHANGE "NEVER CHANGE" "Elizabeth Berg" 2004
n
cd /media/AUDIO/WAVE/9780743518802/mp3
~/Desktop/mp3-to-m4b 9780743518802-NEVER CHANGE "NEVER CHANGE" "Elizabeth Berg" 2004
n
cd /media/AUDIO/WAVE/9780743518840/mp3
~/Desktop/mp3-to-m4b 9780743518840-SQUANDERING AIMLESSLY "SQUANDERING AIMLESSLY" "David Brancaccio" 2004
n
cd /media/AUDIO/WAVE/9780743518840/mp3
~/Desktop/mp3-to-m4b 9780743518840-SQUANDERING AIMLESSLY "SQUANDERING AIMLESSLY" "David Brancaccio" 2004
n
cd /media/AUDIO/WAVE/9780743518840/mp3
~/Desktop/mp3-to-m4b 9780743518840-SQUANDERING AIMLESSLY "SQUANDERING AIMLESSLY" "David Brancaccio" 2004
n
cd /media/AUDIO/WAVE/9780743518840/mp3
~/Desktop/mp3-to-m4b 9780743518840-SQUANDERING AIMLESSLY "SQUANDERING AIMLESSLY" "David Brancaccio" 2004
n
Please help to remove. I colored the repeats to show what I want deleted.
Can someone help me with the following 2 objectives?
1) The following command is just an example. It gets a list of all print jobs. From there I am trying to extract the printer name. It works with the following command:
lpstat -W "completed" -o | awk -F- '{ print $1}'
Problem is, I want... (6 Replies)
I need to add the content of file1 to file2 - all lines but not those existing in file2 already, so the "cat file1 >> file2" doesn't work.
For example,
file1:
100 xxxxxx str1
102 xxxxxx str2
File2:
50 xxxxxxx xxx
30 xxxxxxxxxxx
102 xxxxxx str2 xxxx
......
the result:
50 xxxxxxx... (9 Replies)
Hi
I am trying to locate all strings seperated by the string -CR-, the string will have an unknown number of these -CR- strings, I've used the regex:
^(?:(.*?)-CR-)+$
As shown in the test code:
my ($myString)="This is the first line-CR-second line-CR-third line-CR-fourth... (3 Replies)
I want a shell script that will traverse a file system starting at specific path.
And look at all file names for repeating sequences of and remove them from the file name.
The portion of the name that gets removed has to be a repeating sequence of the same characters.
So the script would... (3 Replies)
Hello,
I have file to work with. It has 5 columns. The first three, altogether, constitutes the position. The 4th column contains some values for downstream analysis and the fifth column contains some values that I want to add to 4th column (only if they happen to be in the same position).
My... (5 Replies)
Hi,
I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
My file is in a good column format but several lines in the file begin with a zero. I'm in KSH and looking for a command to remove this zero and keep the text next to it. I don't want any of the zeros in the other columns removed.
Below is a snip from the file... all I need to do is remove that... (2 Replies)
Hi,
I need to find the lines which are repeating in a file
cat file1
abcdef 23-1
abcdef 24-1
bcdeff 25-0
ttdcfg 26-0
ttdcfg 20-0
bcdef1 25-0
bcdef2 25-0
bcdef3 25-0
bcdef4 25-0
bcdef4 00-0any help is greatly appreciated. Thanks in advance.
In need to find which one are... (3 Replies)
The bash below executes and seems to work fine on those files in which . However on those files where there is no additional CNV detected that line repeats multiple times
instead of only once. I tried adding an END as all lines are printed but that doesn't help. I can not seem to solve this... (5 Replies)
Discussion started by: cmccabe
5 Replies
LEARN ABOUT MOJAVE
uniq
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e., the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e., the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of uniq as described in environ(7).
EXIT STATUS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'') as amended by Cor. 1-2002.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD December 17, 2009 BSD