01-21-2008
hmmmm, maybe
uniq only picks up repeats when they are consecutive lines (I think). I need the script to check the whole file for a repeat....
<---scratching head
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a file which consists of 1000 entries. Out of 1000 entries i have 500 Duplicate Entires. I want to remove the first Duplicate Entry (i,e entire Line) in the File.
The example of the File is shown below:
8244100010143276|MARISOL CARO||MORALES|HSD768|CARR 430 KM 1.7 ... (1 Reply)
Discussion started by: ravi_rn
1 Replies
2. Shell Programming and Scripting
Hi All,
I have one problem related to /.rhosts file.
According to my understanding, /.rhosts file is used for "rsh". What will happen if I have duplicate entries in this file?
e.g> my .rhosts file looks like
Code:
wcars42g
wcars89j
wcars42g
wcars42b
wcars42b
Will duplicate entries... (1 Reply)
Discussion started by: akash_mahakode
1 Replies
3. Shell Programming and Scripting
Hi,
I forgot how to start a new thread. :(
Can somebody please guide me?
I have one problem related to /.rhosts file.
According to my understanding, /.rhosts file is used for "rsh". What will happen if I have duplicate entries in this file?
e.g> my .rhosts file looks like
wcars42g... (2 Replies)
Discussion started by: akash_mahakode
2 Replies
4. Shell Programming and Scripting
Hi,
how to check when was the linux upgrade has been done to this host.
below command show the RHEL version.
cat /etc/redhat-release
Red Hat Enterprise Linux AS release 3 (Taroon Update 6)
i want to know when it was upgraded..
is there any command available for that ? or anywhere we... (3 Replies)
Discussion started by: mail2sant
3 Replies
5. Shell Programming and Scripting
Hi,
My script needs to check whether the remote host is up or not.
If it is up i need to start few servers in that host or else just a notification should be sent that the remote host is down?
Could someone help me out in this?
Regards
Arun (4 Replies)
Discussion started by: arunkumarmc
4 Replies
6. Shell Programming and Scripting
Hi,
I have a very big (with around 1 million entries) txt file with IPv4 addresses in the standard format, i.e. a.b.c.d
The file looks like
10.1.1.1
10.1.1.1
10.1.1.1
10.1.2.4
10.1.2.4
12.1.5.6
.
.
.
.
and so on....
There are duplicate/multiple entries for some IP... (3 Replies)
Discussion started by: sajal.bhatia
3 Replies
7. Shell Programming and Scripting
Hi
I have a file
1 xyz 456
1 xyz 456
1 xyz 456
2 abc 8459
3 gfd 657
4 ghf 658
4 ghf 658
I want the output
1 xyz 456
2 abc 8459
3 gfd 657
4 ghf 658 (3 Replies)
Discussion started by: manigrover
3 Replies
8. Shell Programming and Scripting
Hello experts,
I have a requirement where I have to implement two checks on a csv file:
1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit.
2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
Discussion started by: avikaljain
4 Replies
9. Shell Programming and Scripting
Hi,
There are duplicate entries in file, but uniq will not see because first field is different. How will I catch all lines, which are having duplicate IPs ?
bash-2.05# cat db.file | grep 172.30.133.11
dsrq-ctrl1-prod A 172.30.133.11
e911q-db1-nxge0 A 172.30.133.11... (4 Replies)
Discussion started by: solaris_1977
4 Replies
10. UNIX for Beginners Questions & Answers
Hi all, posting my first time, hope not breaking posting rules with it, if yes, let me know.
I'm trying to build a script to check a file in an sftp server through a remote server.
The intention is to check the file in a sftp host, and if the file is found or not, it should send an email.... (4 Replies)
Discussion started by: MrShinyPants
4 Replies
UNIQ(1) User Commands UNIQ(1)
NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines
-D, --all-repeated[=delimit-method]
print all duplicate lines delimit-method={none(default),prepend,separate} Delimiting is done with blank lines
-f, --skip-fields=N
avoid comparing the first N fields
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
end lines with 0 byte, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use `sort -u' without
`uniq'. Also, comparisons honor the rules specified by `LC_COLLATE'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
Report uniq bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report uniq translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
comm(1), join(1)
The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the
command
info coreutils 'uniq invocation'
should give you access to the complete manual.
GNU coreutils 8.5 February 2011 UNIQ(1)