12-09-2009
Quote:
Originally Posted by
frans
I'v seen in another post that you wanted to remove duplicate lines
pipe the output or file file through the 'uniq' command and first through a 'sort' command if it's not already sorted.
... or use directly
sort -u (if loosing the original order is not an issue).
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am trying to write a script that kills old sessions, I've posted here over the past few days and the script is just about perfect except I want to be given the option to exclude specified PIDs from being killed. this is the entire script:
if
then
rm /tmp/idlepids
fi
if
then
rm... (2 Replies)
Discussion started by: raidzero
2 Replies
2. Shell Programming and Scripting
All,
So, I have an ldif file that contains about 6500 users worth of data. Some users have a block of text I'd like to remove, while some don't.
Example (block of text in question is the block starting with "authAuthority: ;Kerberosv5"):
User with text block:
# username, users,... (7 Replies)
Discussion started by: staze
7 Replies
3. Shell Programming and Scripting
Quick question...I'm trying to grab the .tif file name from this output from our fax server. What is the best way i can do this in a bash script? I have been looking at regular expressions with bash or using awk but having some trouble. thanks! The only output i want is... (5 Replies)
Discussion started by: kuliksco
5 Replies
4. Shell Programming and Scripting
Hello Friends,
I am stuck with the below problem.Any help will be appreciated.
I have a file which has say 100 lines.
On the second last line I have a line from which i want to remove certain characters..
e.g
CAST(CAST( A as varchar(50)) || ',' ||
CAST(CAST( B as varchar(50)) || ',' ||... (8 Replies)
Discussion started by: vital_parsley
8 Replies
5. Shell Programming and Scripting
how can i remove numbers of characters from the last name of file with respect to not remove the files extension
example
VFX_Official_Trailer_(HD)__Shhh__-_by_Freddy_Chavez_Olmos_&_Shervin_Shoghian-.mp4
i want to rename this to
VFX-Official-Trailer-(HD)-Shhh... (13 Replies)
Discussion started by: ateya
13 Replies
6. Shell Programming and Scripting
I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies
7. Shell Programming and Scripting
I am trying to use awk to place the contens of a filename in $1 and $2 followed by the data in the text file. Basically, put the filename within the text file. There are over 1000 files in the directory and as of now each file is saved with a unique name but it is not within the file. Thank you... (10 Replies)
Discussion started by: cmccabe
10 Replies
8. Shell Programming and Scripting
Hello experts,
I have a text file with lot of curly brackets (both opening { & closing } ). I need to delete them alongwith the text between opening & closing brackets' pair.
For ex: Input:-
59. Rh1 Qe4 {(Qf5-e4 Qd8-g8+ Kg6-f5
Qg8-h7+ Kf5-e5 Qh7-e7+ Ke5-f5 Qe7-d7+ Qe4-e6 Qd7-h7+ Qe6-g6... (6 Replies)
Discussion started by: prvnrk
6 Replies
9. Shell Programming and Scripting
Hello,
I am trying to print searched multiple keywords in multiple files.
It is almost okay with the code but the code puts filename in front of each line.
How may I get rid of it?
-grep -A1 'word1' *.txt | grep -A1 'word2' | grep -A1 'word3'
I expect:
Real outcome:
How may I... (3 Replies)
Discussion started by: baris35
3 Replies
10. UNIX for Beginners Questions & Answers
Hi everyone :)
I have a file "words.txt" containing hundreds of lines of text. Each line contains a slogan.
Using the code below i am able to generate an image with the slogan text from each line.
The image filename is saved matching the last word on each line.
Example:
Line 1: We do... (2 Replies)
Discussion started by: martinsmith
2 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines, one for each group
-D print all duplicate lines
--all-repeated[=METHOD]
like -D, but allow separating groups with an empty line; METHOD={none(default),prepend,separate}
-f, --skip-fields=N
avoid comparing the first N fields
--group[=METHOD]
show all items, separating groups with an empty line; METHOD={separate(default),prepend,append,both}
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
line delimiter is NUL, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without
'uniq'. Also, comparisons honor the rules specified by 'LC_COLLATE'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report uniq translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1), sort(1)
Full documentation at: <http://www.gnu.org/software/coreutils/uniq>
or available locally via: info '(coreutils) uniq invocation'
GNU coreutils 8.28 January 2018 UNIQ(1)