12-22-2008
to get rid of duplicate lines, use sort -u
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all,
I have the following data in a file x.csv:
> ,this is some text here
> ,,,,,,,,,,,,,,,,2006/11/16,0.23
> ,,,,,,,,,,,,,,,,2006/12/16,0.88
< ,,,,,,,,,,,,,,,,this shouldnt be deleted
I need to use SED to match anything with a > in the line and delete that line, can someone help... (7 Replies)
Discussion started by: not4google
7 Replies
2. Shell Programming and Scripting
Hi,
Consider following file with input:
`YFLG:NC^Byad_insert constraint {id=600104470} {profile=GENDER == 2} {profile=BEHAVIOR == 17} {profile=SITEATTR_MULT == siteid:211051} {profile=AGE in }
yad_insert ad {id=1718286093336959379} {type=R}
^AYFLG:YOO^Byad_insert constraint {id=600104471}... (1 Reply)
Discussion started by: bvids
1 Replies
3. Shell Programming and Scripting
I have a text file, a sample of which is as follows:
r/- * 0: WINDOWS/Microsoft.NET/Framework/v2.0.50727/ASP.NETWebAdminFiles/Images/headerGRADIENT_Tall.gif
r/- * 0: WINDOWS/SoftwareDistribution/Download/cf8ec753e88561d2ddb53e183dc05c3e/backoff.jpg
r/- * 0: ... (2 Replies)
Discussion started by: stumpyuk
2 Replies
4. Shell Programming and Scripting
I need to delete those lines from a file, which starts with 45.
How to do it? (3 Replies)
Discussion started by: mady135
3 Replies
5. Shell Programming and Scripting
I need to copy lines to a new file from files with sed using a pattern in char postions 1-3.
Then after the copy, I need to delete those same lines from the input files.
For example, string "ABC" in pos 1-3 (6 Replies)
Discussion started by: laksjfhoius9123
6 Replies
6. Shell Programming and Scripting
Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows.
header for file
1111
2222
3333
header for file
1111
2222
3333
header for file... (8 Replies)
Discussion started by: gary_w
8 Replies
7. Shell Programming and Scripting
Hi,
I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this :
for HOST in $(cat stations.lst |uniq)
do
# echo -n "$HOST"
if ping -c 1 $HOST > /dev/null 2>&1
then
HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies
8. Shell Programming and Scripting
Hi
Im trying to do the following in sed. I want to delete any blank line at the start of a file until it matches a pattern and then stops. for example:
Input
output:
I have got it to work within a range of two patterns with the following:
sed '/1/,/pattern/{/^]*$/d}'
The... (2 Replies)
Discussion started by: duonut
2 Replies
9. Shell Programming and Scripting
Hello,
i have a question.
My problem is that i have a file like:
TEST
JOHN
ADAM
MICHAEL
SEBASTIAN
ANDY
i want find for MICHAEL and want delete lines like this:
TEST (4 Replies)
Discussion started by: eightball
4 Replies
10. Shell Programming and Scripting
Hi Everyone,
I have an unusual requirement. Here is where i am stuck for sometime now...
I have this text file.. lets say .. output.sql...
it has lot of entries... here below is part of the entry...
.. . . . . . . . . . . . . . . . .... (3 Replies)
Discussion started by: vivek d r
3 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines, one for each group
-D, --all-repeated[=METHOD]
print all duplicate lines groups can be delimited with an empty line METHOD={none(default),prepend,separate}
-f, --skip-fields=N
avoid comparing the first N fields
--group[=METHOD]
show all items, separating groups with an empty line METHOD={separate(default),prepend,append,both}
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without
'uniq'. Also, comparisons honor the rules specified by 'LC_COLLATE'.
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report uniq translation bugs to <http://translationproject.org/team/>
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
COPYRIGHT
Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1), sort(1)
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 8.22 June 2014 UNIQ(1)