Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Find unique IP address in a list Post 302512863 by methyl on Monday 11th of April 2011 06:24:47 PM
Old 04-11-2011
Your question needs example input and example output.

First impression is to sort the data so that "uniq" is presented with data it can process.

"remove the duplicates" is wonderfully ambiguous.

My interpretation is:
Code:
cat filename| sort | uniq -u

Or possibly just:
Code:
cat filename| sort | uniq

Hard to tell without an unambigous example.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

send email from address list and subject list

Hello, Here is my problem. there are two files. first.txt <<< contains email address ====== abc@mail.com abd@mail.com abe@mail.com second.txt <<< contains webpage links ======== http//www.test.com/abc/index.html http://www.test.com/abd/index.html http://www.test.com/abe/index.html... (2 Replies)
Discussion started by: paulds
2 Replies

2. Shell Programming and Scripting

Need to find Unique not used Number

Wrote a script to create a hidden account in OS X. It works perfect but I need to check if the UID is already in use before I tried to create the account. dscl . list /Users UniqueID | awk '{print $2}' | while read UIDS do if ; then echo "UID Is Already in USE" i=`expr "$2" - 1` echo... (4 Replies)
Discussion started by: elbombillo
4 Replies

3. UNIX for Dummies Questions & Answers

Selecting Unique Values from many List

I have a question I have like 19 different list which contains the name of the server but what I need is just unique ones. First thing I need to do is just make a unique list within the list itself i.e. delete anything that is repeated inside the list like for example in list1 i... (1 Reply)
Discussion started by: pareshan
1 Replies

4. Shell Programming and Scripting

ksh - how to list all ip address between 2 ip address

Trying to do a ksh script that needs to list all ip address between ip address a and b .. ie. Ip address A=192.168.1.200 Ip address B=192.168.2.15 So the subnet changes from 1 to 2 but I want to list all possible ip addresses between the 2.. Which would be: 192.168.1.200... (4 Replies)
Discussion started by: frustrated1
4 Replies

5. UNIX for Dummies Questions & Answers

Getting unique list of numbers using grep

Hi, I am going to fetch a list of numbers that starts with "0032" from a file with a format like the given below: " 0032459999 0032458888 0032457777 0032451111 0032452222 0032453333 0032459999 0032458888 0032457777 0032451111 0032452222 0032453333 " I want to get a unique... (6 Replies)
Discussion started by: tinku
6 Replies

6. UNIX for Dummies Questions & Answers

Find all the unique file extensions

Hi How can i find the unique list of file extensions in a folder/subfolders e.g. MAIN/ a.txt b.txt a.clas a.java b.class a.txt.112 c.12.ram.jar i just need to get the below out irrespective of file being present in folder or subfolders txt clas java (5 Replies)
Discussion started by: reldb
5 Replies

7. UNIX for Dummies Questions & Answers

Get List of Unique File Names

I have a large directory of web pages. I am doing a search through the web pages using grep and would like to get a list of unique file names of search results. The following command works fine to give me a list of file names where term appears: grep -l term *.html However, since these are... (3 Replies)
Discussion started by: rjulich
3 Replies

8. Shell Programming and Scripting

Generate unique mac address

Hi, I want to generate 2000 mac address. Please let me know how to do so. Perl script or there is some tool availlable wherein i can give the count and it will generate that many mac-address Thanks, Kriti (4 Replies)
Discussion started by: kriti
4 Replies

9. Shell Programming and Scripting

Script to find min for each unique value

I need a script that will search through multiple files and when the first 2 columns match, print out Columns 1 and 2 and the minimum value. File 1 24.01 -81.01 1.0 24.02 -81.02 1.0 24.03 -81.03 3.0 File 2 24.01 -81.01 5.0 24.02 -81.02 3.0 24.03 -81.03 ... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

10. UNIX for Beginners Questions & Answers

Find unique values but only in column 1

Hi All, Does anyone have any suggestions/examples of how i could show only lines where the first field is not duplicated. If the first field is listed more than once it shouldnt be shown even if the other columns make it unique. Example file : 876,RIBDA,EC2 876,RIBDH,EX7 877,RIBDF,E28... (4 Replies)
Discussion started by: mutley2202
4 Replies
UNIQ(1) 						    BSD General Commands Manual 						   UNIQ(1)

NAME
uniq -- report or filter out repeated lines in a file SYNOPSIS
uniq [-cdu] [-f fields] [-s chars] [input_file [output_file]] DESCRIPTION
The uniq utility reads the standard input comparing adjacent lines, and writes a copy of each unique input line to the standard output. The second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are not adjacent, so it may be necessary to sort the files first. The following options are available: -c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space. -d Don't output lines that are not repeated in the input. -f fields Ignore the first fields in each input line when doing comparisons. A field is a string of non-blank characters separated from adja- cent fields by blanks. Field numbers are one based, i.e. the first field is field one. -s chars Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the first chars characters after the first fields fields will be ignored. Character numbers are one based, i.e. the first character is character one. -u Don't output lines that are repeated in the input. If additional arguments are specified on the command line, the first such argument is used as the name of an input file, the second is used as the name of an output file. The uniq utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation. SEE ALSO
sort(1) STANDARDS
The uniq utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
January 6, 2007 BSD
All times are GMT -4. The time now is 10:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy