Sponsored Content
Full Discussion: awk remove first duplicates
Top Forums Shell Programming and Scripting awk remove first duplicates Post 302885760 by sybadm on Tuesday 28th of January 2014 09:28:34 AM
Old 01-28-2014
Quote:
Originally Posted by pamu
Code:
awk 'NR==FNR{A[$3]++;next}{if(A[$3] > 1 && !B[$3]){B[$3]++;next} }1' file file
 
12  NIL ABD LON
12  NIL ABC AMR
13  NIL ABC AMR
11  NIL ABK AMR


Works really well. bit slow.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to remove duplicates without sorting

Hello, I can remove duplicate entries in a file by: sort File1 | uniq > File2 but how can I remove duplicates without sorting the file? I tried cat File1 | uniq > File2 but it doesn't work thanks (4 Replies)
Discussion started by: orahi001
4 Replies

2. Shell Programming and Scripting

Remove duplicates

Hello Experts, I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
Discussion started by: forumthreads
4 Replies

3. Shell Programming and Scripting

remove duplicates and sort

Hi, I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another. Thanks (6 Replies)
Discussion started by: dvah
6 Replies

4. Shell Programming and Scripting

bash - remove duplicates

I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different. I need to go from this: http://***/fae78fe/file1.wmv http://***/39du7si/file1.wmv http://***/d8el2hd/file2.wmv http://***/h893js3/file2.wmv to this: ... (2 Replies)
Discussion started by: locoroco
2 Replies

5. Shell Programming and Scripting

Awk: Remove Duplicates

I have the following code for removing duplicate records based on fields in inputfile file & moves the duplicate records in duplicates file(1st Awk) & in 2nd awk i fetch the non duplicate entries in inputfile to tmp file and use move to update the original file. Requirement: Can both the awk... (4 Replies)
Discussion started by: siramitsharma
4 Replies

6. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

7. Shell Programming and Scripting

Remove top 3 duplicates

hello , I have a requirement with input in below format abc 123 xyz bcd 365 kii abc 987 876 cdf 987 uii abc 456 yuu bcd 654 rrr Expecting Output abc 456 yuu bcd 654 rrr cdf 987 uii (1 Reply)
Discussion started by: Tomlight
1 Replies

8. Shell Programming and Scripting

Sort and Remove duplicates

Here is my task : I need to sort two input files and remove duplicates in the output files : Sort by 13 characters from 97 Ascending Sort by 1 characters from 96 Ascending If duplicates are found retain the first value in the file the input files are variable length, convert... (4 Replies)
Discussion started by: ysvsr1
4 Replies

9. Shell Programming and Scripting

Remove duplicates

Hi I have a below file structure. 200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,, 200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,, 300,20140223,0.001,0.001,0.001,0.001,0.001 300,20140224,0.001,0.001,0.001,0.001,0.001 300,20140225,0.001,0.001,0.001,0.001,0.001 300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Discussion started by: tejashavele
1 Replies

10. Shell Programming and Scripting

awk - Remove duplicates during array build

Greetings Experts, Issue: Within awk script, remove the duplicate occurrences that are space (1 single space character) separated Description: I am processing 2 files using awk and during processing, I am building an array and there are duplicates on this; how can I delete the duplicates... (3 Replies)
Discussion started by: chill3chee
3 Replies
CosNaming_NamingContext(3erl)				     Erlang Module Definition				     CosNaming_NamingContext(3erl)

NAME
CosNaming_NamingContext - This interface supports different bind and access functions for names in a context. DESCRIPTION
This is the object that defines name scopes, names must be unique within a naming context. Objects may have multiple names and may exist in multiple naming contexts. Name context may be named in other contexts and cycles are permitted. The type NameComponent used below is defined as: -record('CosNaming_NameComponent', {id, kind=""}). where id and kind are strings. The type Binding used below is defined as: -record('CosNaming_Binding', {binding_name, binding_type}). where binding_name is a Name and binding_type is an enum which has the values nobject and ncontext . Both these records are defined in the file CosNaming.hrl and it is included with: -include_lib("orber/COSS/CosNaming/CosNaming.hrl"). There are a number of exceptions that can be returned from functions in this interface. * NotFound is defined as -record('CosNaming_NamingContext_NotFound', {rest_of_name, why}). * CannotProceed is defined as -record('CosNaming_NamingContext_CannotProceed', {rest_of_name, cxt}). * InvalidName is defined as -record('CosNaming_NamingContext_InvalidName', {}). * NotFound is defined as -record('CosNaming_NamingContext_NotFound', {}). * AlreadyBound is defined as -record('CosNaming_NamingContext_AlreadyBound', {}). * NotEmpty is defined as -record('CosNaming_NamingContext_NotEmpty', {). These exceptions are defined in the file CosNaming_NamingContext.hrl and it is included with: -include_lib("orber/COSS/CosNaming/CosNaming_NamingContext.hrl"). EXPORTS
bind(NamingContext, Name, Object) -> Return Types NameContext = #objref Name = [NameComponent] Object = #objref Return = ok Creates a binding of a name and an object in the naming context. Naming contexts that are bound using bind() do not participate in name resolution. rebind(NamingContext, Name, Object) -> Return Types NamingContext = #objref Name = [NameComponent] Object = #objref Return = ok Creates a binding of a name and an object in the naming context even if the name is already bound. Naming contexts that are bound using rebind() do not participate in name resolution. bind_context(NamingContext1, Name, NamingContex2) -> Return Types NamingContext1 = NamingContext2 =#objref Name = [NameComponent] Return = ok The bind_context function creates a binding of a name and a naming context in the current context. Naming contexts that are bound using bind_context() participate in name resolution. rebind_context(NamingContext1, Name, NamingContex2) -> Return Types NamingContext1 = NamingContext2 =#objref Name = [NameComponent] Return = ok The rebind_context function creates a binding of a name and a naming context in the current context even if the name already is bound. Naming contexts that are bound using rebind_context() participate in name resolution. resolve(NamingContext, Name) -> Return Types NamingContext = #objref Name = [NameComponent] Return = Object Object = #objref The resolve function is the way to retrieve an object bound to a name in the naming context. The given name must match exactly the bound name. The type of the object is not returned, clients are responsible for narrowing the object to the correct type. unbind(NamingContext, Name) -> Return Types NamingContext = #objref Name = [NameComponent] Return = ok The unbind operation removes a name binding from the naming context. new_context(NamingContext) -> Return Types NamingContext = #objref Return = #objref The new_context operation creates a new naming context. bind_new_context(NamingContext, Name) -> Return Types NamingContext = #objref Name = [NameComponent] Return = #objref The new_context operation creates a new naming context and binds it to Name in the current context. destroy(NamingContext) -> Return Types NamingContext = #objref Return = ok The destroy operation disposes the NamingContext object and removes it from the name server. The context must be empty e.g. not con- tain any bindings to be removed. list(NamingContext, HowMany) -> Return Types NamingContext = #objref HowMany = int() Return = {ok, BindingList, BindingIterator} BindingList = [Binding] BindingIterator = #objref The list operation returns a BindingList with a number of bindings up-to HowMany from the context. It also returns a BindinIterator which can be used to step through the list. If the total number of existing bindings are less than, or equal to, the HowMany parame- ter a NIL object reference is returned. Note: One must destroy the BindingIterator, unless it is a NIL object reference, by using 'BindingIterator':destroy(). Otherwise one can get dan- gling objects. Ericsson AB orber 3.6.20 CosNaming_NamingContext(3erl)
All times are GMT -4. The time now is 11:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy