how to pick distinct records..........


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers how to pick distinct records..........
# 1  
Old 01-04-2007
how to pick distinct records..........

How can i pick distinct records which consists of duplicate data from a ASCII file using UNIX commands
# 2  
Old 01-04-2007
Try uniq command.
# 3  
Old 01-04-2007
uniq

first u need to sort the file and use the uniq command to get the correct lines
without any duplicates
sort the file and use the uniq command
# 4  
Old 01-04-2007
Use 'sort -u' if your file isn't already sorted.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Separate records of a file on 2 types of records

Hi I am new to shell programming in unix Please if I can provide help. I have a file structure of a header record and "N" detail records. The header record will be the total number of detail records I need to split the file in 2: One for the header Another for all detail records Could... (1 Reply)
Discussion started by: jamcogar
1 Replies

2. Shell Programming and Scripting

Need the distinct of these lines

Red Hat Enterprise Linux 5.4 I have a text file (error log file) , which has occurences of an error message like ORA-01652: unable to extend temp segment by 8 in tablespace xxxxxThere are around 3000 error messages like this in the error log file. But there are only 7 or 8 distinct... (3 Replies)
Discussion started by: John K
3 Replies

3. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

4. Shell Programming and Scripting

Get the distinct of a particular field

From the below ps output , I want the distinct values of the third field (ie. I need the distinct PPIDs) $ ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 Nov 10 - 48:49 /etc/init root 1769576 1 0 Nov 10 - 0:07... (1 Reply)
Discussion started by: polavan
1 Replies

5. UNIX for Dummies Questions & Answers

Grep distinct

Hello, I have a file which looks like: %_SPE_RDP_NUM_ECH(7)*************% %_SPE_RDP_NUM_ECH(70)************% %_SPE_RDP_NUM_ECH(71)************% %_SPE_RDP_NUM_ECH(72)************% %_SPE_RDP_NUM_ECH(8)*************% %_SPE_RDP_NUM_ECH(9)*************% %_SPE_FLUXPREV_PRES1_MT_HT(2)****%... (5 Replies)
Discussion started by: mvalonso
5 Replies

6. Web Development

SQL - Distinct plus more

Hi all, I have an interesting and I am sure simple question for yau'll. Basically this is what I am after: The table: CREATE TABLE places (id INT, city VARCHAR(24), name VARCHAR(24)); The data: id = 1, city = canberra, name = aaron id = 2, city = canberra, name = andrew id = 3, city... (4 Replies)
Discussion started by: STOIE
4 Replies

7. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

8. Shell Programming and Scripting

How to Pick Random records from a large file

Hi, I have a huge file say with 2000000 records. The file has 42 fields. I would like to pick randomly 1000 records from this huge file. Can anyone help me how to do this? (1 Reply)
Discussion started by: ajithshankar@ho
1 Replies

9. Shell Programming and Scripting

Based on num of records in file1 need to check records in file2 to set some condns

Hi All, I have two files say file1 and file2. I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions . Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question