Sponsored Content
Full Discussion: Deleting Duplicate Records
Top Forums Shell Programming and Scripting Deleting Duplicate Records Post 302391425 by Franklin52 on Monday 1st of February 2010 09:42:12 AM
Old 02-01-2010
Please search the forums using our internal Google search engine or our advanced internal search engine next time. Thank you.

This should work:
Code:
awk '!a[$1]++' file

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Records Duplicate

Hi Everyone, I have a flat file of 1000 unique records like following : For eg Andy,Flower,201-987-0000,12/23/01 Andrew,Smith,101-387-3400,11/12/01 Ani,Ross,401-757-8640,10/4/01 Rich,Finny,245-308-0000,2/27/06 Craig,Ford,842-094-8740,1/3/04 . . . . . . Now I want to duplicate... (9 Replies)
Discussion started by: ganesh123
9 Replies

2. Shell Programming and Scripting

combine duplicate records

I have a .DAT file like below 23666483030000653-B94030001OLFXXX000000120081227 23797049900000654-E71060001OLFXXX000000220081227 23699281320000655 E71060002OLFXXX000000320081227 22885068900000652 B86860003OLFXXX592123320081227 22885068900000652 B86860003ODL-SP592123420081227... (8 Replies)
Discussion started by: kshuser
8 Replies

3. Shell Programming and Scripting

Remove duplicate records

I want to remove the records based on duplicate. I want to remove if two or more records exists with combination fields. Those records should not come once also file abc.txt ABC;123;XYB;HELLO; ABC;123;HKL;HELLO; CDE;123;LLKJ;HELLO; ABC;123;LSDK;HELLO; CDF;344;SLK;TEST key fields are... (7 Replies)
Discussion started by: svenkatareddy
7 Replies

4. Shell Programming and Scripting

Remove Duplicate Records

Hi frinds, Need your help. item , color ,desc ==== ======= ==== 1,red ,abc 1,red , a b c 2,blue,x 3,black,y 4,brown,xv 4,brown,x v 4,brown, x v I have to elemnet the duplicate rows on the basis of item. the final out put will be 1,red ,abc (6 Replies)
Discussion started by: imipsita.rath
6 Replies

5. UNIX for Dummies Questions & Answers

Getting non-duplicate records

Hi, I have a file with these records abc xyz xyz pqr uvw cde cde In my o/p file , I want all the non duplicate rows to be shown. o/p abc pqr uvw Any suggestions how to do this? Thanks for the help. rs (2 Replies)
Discussion started by: rs123
2 Replies

6. UNIX for Dummies Questions & Answers

Need to keep duplicate records

Consider my input is 10 10 20 then, uniq -u will give 20 and uniq -dwill return 10. But i need the output as , 10 10 How we can achieve this? Thanks (4 Replies)
Discussion started by: pandeesh
4 Replies

7. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

8. Shell Programming and Scripting

How to keep the last 2 records from duplicate entries?

Gents, Please how I can get only the last 2 records from repetead values, from column 2 input 1 1011 1 1011 1 1012 1 1012 1 5001 1 5001 1 5002 1 5002 1 5003 1 5003 1 7001 1 7001 1 7002 1 7002 (2 Replies)
Discussion started by: jiam912
2 Replies

9. Shell Programming and Scripting

Duplicate records

Gents, I have a file which contends duplicate records in column 1, but the values in column 2 are different. 3099753489 3 3099753489 5 3101954341 12 3101954341 14 3102153285 3 3102153285 5 3102153297 3 3102153297 5 I will like to get something like this: output desired... (16 Replies)
Discussion started by: jiam912
16 Replies

10. Shell Programming and Scripting

Duplicate records

Gents, Please give a help file --BAD STATUS NOT RESHOOTED-- *** VP 41255/51341 in sw 2973 *** VP 41679/51521 in sw 2973 *** VP 41687/51653 in sw 2973 *** VP 41719/51629 in sw 2976 --BAD COG NOT RESHOOTED-- *** VP 41689/51497 in sw 2974 *** VP 41699/51677 in sw 2974 *** VP... (18 Replies)
Discussion started by: jiam912
18 Replies
WWW::OpenSearch(3pm)					User Contributed Perl Documentation				      WWW::OpenSearch(3pm)

NAME
WWW::OpenSearch - Search A9 OpenSearch compatible engines SYNOPSIS
use WWW::OpenSearch; my $url = "http://bulkfeeds.net/opensearch.xml"; my $engine = WWW::OpenSearch->new($url); my $name = $engine->description->ShortName; my $tags = $engine->description->Tags; # Perform search for "iPod" my $response = $engine->search("iPod"); for my $item (@{$response->feed->items}) { print $item->{description}; } # Retrieve the next page of results my $next_page = $response->next_page; for my $item (@{$next_page->feed->items}) { print $item->{description}; } DESCRIPTION
WWW::OpenSearch is a module to search A9's OpenSearch compatible search engines. See http://opensearch.a9.com/ for details. CONSTRUCTOR
new( $url ) Constructs a new instance of WWW::OpenSearch using the given URL as the location of the engine's OpenSearch Description document (retrievable via the description_url accessor). METHODS
fetch_description( [ $url ] ) Fetches the OpenSearch Descsription found either at the given URL or at the URL specified by the description_url accessor. Fetched description may be accessed via the description accessor. search( $query [, \%params] ) Searches the engine for the given query using the given search parameters. Valid search parameters include: o startPage o totalResults o startIndex o itemsPerPage See http://opensearch.a9.com/spec/1.1/response/#elements for details. do_search( $url [, $method] ) Performs a request for the given URL and returns a WWW::OpenSearch::Response object. Method defaults to 'GET'. ACCESSORS
description_url( [$description_url] ) agent( [$agent] ) description( [$description] ) AUTHOR
Brian Cassidy <bricas@cpan.org> Tatsuhiko Miyagawa <miyagawa@bulknews.net> COPYRIGHT AND LICENSE
Copyright 2005-2010 by Tatsuhiko Miyagawa and Brian Cassidy This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2010-05-25 WWW::OpenSearch(3pm)
All times are GMT -4. The time now is 03:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy