Getting unique list of numbers using grep

Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Getting unique list of numbers using grep
# 1  
Old 08-07-2009
Getting unique list of numbers using grep


I am going to fetch a list of numbers that starts with "0032" from a file with a format like the given below:

0032459999 0032458888 0032457777
0032451111 0032452222 0032453333

0032459999 0032458888 0032457777
0032451111 0032452222 0032453333
I want to get a unique list of number that starts with "0032".

How can it be achived using grep command. (also we may require sed, sort and uniq along with grep)

Just a note, the file will be a large in size with lot of repetation.
But just i need a unique list of numbers that starts with the pattern "0032".

I am planning to use fopen, reading the file line by line and then adding it to a array list...
But I guess, it can be done more effeciently using grep.

Any help is appreciated.

# 2  
Old 08-07-2009
To keep the forums high quality for all users, please take the time to format your posts correctly.

First of all, use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.)

Second, avoid adding color or different fonts and font size to your posts. Selective use of color to highlight a single word or phrase can be useful at times, but using color, in general, makes the forums harder to read, especially bright colors like red.

Third, be careful when you cut-and-paste, edit any odd characters and make sure all links are working property.

Thank You.

The UNIX and Linux Forums


grep ^0032 infile| sort| uniq

# 3  
Old 08-07-2009
Thanks for the help.

i have missed some information in my previous communication.

The number starts with the pattern "0032" and it is 10 character length. And I will
get multiple numbers on the same lines. Some blank spaces, blank lines will also come in between the data.

A sample file, as follows:

<Sample data>
0032459999 0032458888 0032457777
0032451111 0032452222 0032453333

0032459999 0032458888 0032457777
0032451111 0032452222 0032453333

</Sample data>

And numbers is duplicated all over the file.

Each number is seprated by two spaces.

Sometimes, a large chunk of white space prefixed before the numbers.
But always the length of line is fixed to 151 characters.

Actually I want to create a report and in the report I want to list the number one below another like the below one.

<Sample output>
</Sample output>

Any help on this is appreciated.
# 4  
Old 08-07-2009
For CODE-tags that "Sample output" inside <> should be changed to the word "code".
# 5  
Old 08-11-2009
getting uniq values


Now, using the below code:

grep ^0032 F*.txt | tr -s " " | tr -d "\n" | tr -s "\r" | sort -n | uniq

I am getting the following output
0032459999 0032458888 0032457777 0032451111 0032452222 0032453333 0032451111 0032458888 00324522220032451111 003245222200324511110032451111 0032452222 0032453333

sort and uniq commands are not working.

What should be done to get a uniq value from this.

Any help on this is appreciated.

# 6  
Old 08-11-2009
tr ' ' '\n' < file.txt | sort | uniq | grep -v "^$"

or better if you have some other numbers that are not starting with 0032
tr ' ' '\n' < file.txt | sort | uniq | grep "^0032"

grep -o "0032\([0-9]\)\+" file.txt | sort | uniq

Last edited by necroman08; 08-11-2009 at 12:43 PM..
# 7  
Old 08-12-2009
The last option is PERFECT. It gives exactly what i needed.

Tons of thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Printing unique numbers from each file

I have some files named file1, file2, fille3......etc. These files are in a folder f1. The content of files are shown below. I would like to count the unique pairs of third column in each file. some files have no data. It should be printed as zero. Your help would be appreciated. file1 ARG... (1 Reply)
Discussion started by: samra
1 Replies

2. Shell Programming and Scripting

Unique entries based on a range of numbers.

Hi, I have a matrix like this: Algorithm predicted_gene start_point end_point A x 65 85 B x 70 80 C x 75 85 D x 10 20 B y 125 130 C y 120 140 D y 200 210 Here there are four tab-separated columns. The first column is the used algorithm for prediction, and there are 4 of them A-D.... (8 Replies)
Discussion started by: flyfisherman
8 Replies

3. Shell Programming and Scripting

unique random numbers awk

Hi, I have a small piece of awk code (see below) that generates random numbers. gawk -F"," 'BEGIN { srand(); for (i = 1; i <= 30; i++) printf("%s AM329_%04d\n",$0,int(36 * rand())+1) }' OFS=, AM329_hole_names.csv The code works fine and generates alphanumeric numbers like AM329_0001,... (2 Replies)
Discussion started by: theflamingmoe
2 Replies

4. Shell Programming and Scripting

Replace string ids with unique numbers

Hello, I have a file with a 1000 ids in the form of strings. I want to replace each id with a unique numbers in the whole file. each id is repeating in all the columns. I know I can use sed command but there are many ids in file which are need to be converted example of input file B752... (4 Replies)
Discussion started by: ryan9011
4 Replies

5. UNIX for Dummies Questions & Answers

grep specific number from a list of numbers

Hello. I have 9060 files labelled File1 to File9060. They are in numerical order. When I grep a file eg. File90 it will show me all files that contain the pattern "File90", eg File901 or File9001. I can only get specific files for File1000 or higher. How can I resolve this problem? Is there a... (5 Replies)
Discussion started by: godzilla07
5 Replies

6. UNIX for Dummies Questions & Answers

Grep Unique

Hello, I have a file with a list of car makes and specific information for each make. An example is: @Audi:Warranty @Audi:Pricing @Audi:Colors @Acura:Warranty @Acura:Pricing @Acura:Colors and so on through a bunch of makes. I need to make a list in a selection box of just one name of... (4 Replies)
Discussion started by: macbb1117
4 Replies

7. Shell Programming and Scripting

How to generate 10.000 unique numbers?

hello, does anybody can give me a hint on how to generate a lot of numbers which are not identically via scripting etc? (7 Replies)
Discussion started by: xrays
7 Replies

8. Shell Programming and Scripting

grep for unique value

If i want to grep for a value in a file but display only unique value then which option can i use. ex: Values in the file IP <> value <2> IP <> value <2> IP <> value <3> IP <> value <1> i should get only the unique value (3 Replies)
Discussion started by: vls1210
3 Replies

9. UNIX for Dummies Questions & Answers

To get unique numbers from two files

here i have two files: file 1 1 2 3 4 5 5 6 7 8 9 file 2 4 5 6 6 8 8 (6 Replies)
Discussion started by: i.scientist
6 Replies

10. Shell Programming and Scripting

Shell script to check the unique numbers in huge data

Friends, I have to write a shell script,the description is---- i Have to check the uniqueness of the numbers in a file. A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I... (7 Replies)
Discussion started by: namishtiwari
7 Replies
Login or Register to Ask a Question