01-04-2007
uniq
first u need to sort the file and use the uniq command to get the correct lines
without any duplicates
sort the file and use the uniq command
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a flat file and need to count no of records in the file less the header and the trailer record.
I would appreciate any and all asistance
Thanks
Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
2. Shell Programming and Scripting
Hi All,
I have two files say file1 and file2.
I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions .
Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies
3. Shell Programming and Scripting
Hi,
I have a huge file say with 2000000 records. The file has 42 fields. I would like to pick randomly 1000 records from this huge file. Can anyone help me how to do this? (1 Reply)
Discussion started by: ajithshankar@ho
1 Replies
4. UNIX for Dummies Questions & Answers
Hi everyone.
I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this:
ID: 20
Name: X
Age: 19
ID: 21
Name: Z
ID: 22
Email: xxx@yahoo.com
Name: Y
Age: 19
I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies
5. Web Development
Hi all,
I have an interesting and I am sure simple question for yau'll.
Basically this is what I am after:
The table:
CREATE TABLE places (id INT, city VARCHAR(24), name VARCHAR(24));
The data:
id = 1, city = canberra, name = aaron
id = 2, city = canberra, name = andrew
id = 3, city... (4 Replies)
Discussion started by: STOIE
4 Replies
6. UNIX for Dummies Questions & Answers
Hello,
I have a file which looks like:
%_SPE_RDP_NUM_ECH(7)*************%
%_SPE_RDP_NUM_ECH(70)************%
%_SPE_RDP_NUM_ECH(71)************%
%_SPE_RDP_NUM_ECH(72)************%
%_SPE_RDP_NUM_ECH(8)*************%
%_SPE_RDP_NUM_ECH(9)*************%
%_SPE_FLUXPREV_PRES1_MT_HT(2)****%... (5 Replies)
Discussion started by: mvalonso
5 Replies
7. Shell Programming and Scripting
From the below ps output , I want the distinct values of the third field (ie. I need the distinct PPIDs)
$ ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Nov 10 - 48:49 /etc/init
root 1769576 1 0 Nov 10 - 0:07... (1 Reply)
Discussion started by: polavan
1 Replies
8. Shell Programming and Scripting
Hi Freinds ,
I have 2 files .
File 1
|nag|HYd|1|Che
|esw|Gun|2|hyd
|pra|bhe|3|hyd
|omu|hei|4|bnsj
|uer|oeri|5|uery
File 2
|nag|HYd|1|Che
|esw|Gun|2|hyd
|uer|oi|3|uery
output : (9 Replies)
Discussion started by: i150371485
9 Replies
9. Shell Programming and Scripting
Red Hat Enterprise Linux 5.4
I have a text file (error log file) , which has occurences of an error message like
ORA-01652: unable to extend temp segment by 8 in tablespace xxxxxThere are around 3000 error messages like this in the error log file. But there are only 7 or 8 distinct... (3 Replies)
Discussion started by: John K
3 Replies
10. Shell Programming and Scripting
Hi I am new to shell programming in unix
Please if I can provide help.
I have a file structure of a header record and "N" detail records.
The header record will be the total number of detail records
I need to split the file in 2:
One for the header
Another for all detail records
Could... (1 Reply)
Discussion started by: jamcogar
1 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines
-D, --all-repeated[=delimit-method]
print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines
-f, --skip-fields=N
avoid comparing the first N fields
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use `sort -u' without
`uniq'. Also, comparisons honor the rules specified by `LC_COLLATE'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
Report uniq bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report uniq translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1)
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 8.12.197-032bb September 2011 UNIQ(1)