12-04-2008
Hi vidhyadhar,
--> uniq infile outfile
the above command removes the repeated lines from the file 'infile' only if the lines are repeated consequtively..
for example:
if infile contains the following contents
one
two
three
five
four
five
one
the uniq cmd wont remove the repeated lines one and five!!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I wish to delete lines from a log file. Should take a date string as a variable. Find the date in the file, get the line number for the string. Delete lines from 1 to that (line number - 1), all from within a shell script. Is that possible ?.
Thnks,
:confused: (6 Replies)
Discussion started by: umal
6 Replies
2. UNIX for Dummies Questions & Answers
I've got a file like this:
Grid-ref= 443, 229
167 169 204 233 290 309 308 326 300 251 194 161
148 189 228 251 296 329 331 338 308 263 219 179
178 203 215 252 277 319 327 335 312 264 196 149
120 172 226 253 297 329 323 322 305 242 203 136
... (20 Replies)
Discussion started by: su_in99
20 Replies
3. Shell Programming and Scripting
Hi,
Just consider there are around 10 lines in a file. Now is it possible to delete the first 2 lines in the file without opening the file. No matter whatever the content of the file is, I just wanna delete the first 2 lines without opening the file. Is that possible? If so, please help me out.... (3 Replies)
Discussion started by: toms
3 Replies
4. Shell Programming and Scripting
Hi,
Im doing a script to find the number of lines included inside a file newly. These lines are in between #ifdef FLAG1 and #else or #endif or #else and #endif.
I tried like this,
awk '/#ifdef Flag1/,/#e/{print}' aa.c | wc -l
awk '/#ifndef Flag1/,/#endif/{print}' aa.c | awk... (6 Replies)
Discussion started by: priyadarshini
6 Replies
5. Shell Programming and Scripting
Input:
a
b
b
c
d
d
I need:
a
c
I know how to get this (the lines that have duplicates) :
b
d
sort file | uniq -d
But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies
6. UNIX for Dummies Questions & Answers
Hiiii
I have a file which contains huge data as
a.dat:
PDE 1990 1 9 18 51 28.90 24.7500 95.2800 118.0 6.1 0.0 BURMA
event name: 010990D
time shift: 7.3000
half duration: 5.0000
latitude: 24.4200
longitude: 94.9500
depth: 129.6000
Mrr: ... (7 Replies)
Discussion started by: reva
7 Replies
7. Shell Programming and Scripting
Hi Gurus,
I am working with a korn shell script to simplify some operations of calculation number of lines inside compressed file.
The called function (inside a cycle) is the following:
#########################################
# F.ne: CheckCount
#########################################... (3 Replies)
Discussion started by: GERMANICO
3 Replies
8. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
9. Shell Programming and Scripting
I have a text file say g1.txt and content of this file is ...as below
1|HideCDrive, | : REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer" /v NoDrives /t REG_DWORD /d 4 /f,
2|HideRunButton, | : REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer"... (7 Replies)
Discussion started by: jalpasoni
7 Replies
10. Shell Programming and Scripting
Hello,
I have file of more than 10000 lines.
I want to delete 40 lines after every 20 lines.
e.g from a huge file, i want to delete line no from 34 - 74, then 94 - 134 etc and so on.
Please let me know how i can do it.
Best regards, (11 Replies)
Discussion started by: nehashine
11 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines
-D, --all-repeated[=delimit-method]
print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines
-f, --skip-fields=N
avoid comparing the first N fields
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use `sort -u' without
`uniq'. Also, comparisons honor the rules specified by `LC_COLLATE'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
Report uniq bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report uniq translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1)
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 8.12.197-032bb September 2011 UNIQ(1)