Hello,
If order out of sections is not important, with (gnu) awk:
Code:
awk 'BEGIN{RS='\n\n'};{A[$0]=1};END{for (h in A) print h,"\n"}' file
Regards.
Thanks for your help. your code worked fine. I had already tried similar code but the difference was I didn't set RS, and instead of A[$0] =1 I assigned A[$0]=$0 and the array was getting jumbled up. Do you know the reason?
Rudic - I dont quite understand this code. can you please help me understand?
Hi,
I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g.
COL1 COL2 COL3
A 1234 1234
B 3k32 2322
C Xk32 TTT
A NEW XX22
B 3k32 ... (7 Replies)
My input file:
AVI.out <detail>named as the RRM .</detail>
AVI.out <detail>Contains 1 RRM .</detail>
AR0.out <detail>named as the tellurite-resistance.</detail>
AWG.out <detail>Contains 2 HTH .</detail>
ADV.out <detail>named as the DENR family.</detail>
ADV.out ... (10 Replies)
Hii Friends.. I have a huge set of data stored in a file.Which is as shown below
a.dat:
RAO 1869 12 19 0 0 0.00 17.9000 82.3000 10.0 0 0.00 0 3.70 0.00 0.00 0 0.00 3.70 4 NULL
LEE 1870 4 11 1 0 0.00 30.0000 99.0000 0.0 0 0.00 0 0.00 0.00 0.00 0 ... (3 Replies)
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Hi,
How can I remove duplicates from a file based on group on other column? for example:
Test1|Test2|Test3|Test4|Test5
Test1|Test6|Test7|Test8|Test5
Test1|Test9|Test10|Test11|Test12
Test1|Test13|Test14|Test15|Test16
Test17|Test18|Test19|Test20|Test21
Test17|Test22|Test23|Test24|Test5
... (2 Replies)
Hi All,
i have input file like below...
CA009156;20091003;M;AWBKCA72;123;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;321;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;231;;CANADIAN... (2 Replies)
Hi ,
Some time i got duplicated value in my files ,
bundle_identifier= B
Sometext=ABC
bundle_identifier= A
bundle_unit=500
Sometext123=ABCD
bundle_unit=400
i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed.
example of my file... (3 Replies)
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
LEARN ABOUT DEBIAN
locale::language
Locale::Language(3perl) Perl Programmers Reference Guide Locale::Language(3perl)NAME
Locale::Language - standard codes for language identification
SYNOPSIS
use Locale::Language;
$lang = code2language('en'); # $lang gets 'English'
$code = language2code('French'); # $code gets 'fr'
@codes = all_language_codes();
@names = all_language_names();
DESCRIPTION
The "Locale::Language" module provides access to standard codes used for identifying languages, such as those as defined in ISO 639.
Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639 two-
letter codes will be used.
SUPPORTED CODE SETS
There are several different code sets you can use for identifying languages. The ones currently supported are:
alpha-2
This is the set of two-letter (lowercase) codes from ISO 639, such as 'he' for Hebrew.
This code set is identified with the symbol "LOCALE_LANG_ALPHA_2".
This is the default code set.
alpha-3
This is the set of three-letter (lowercase) bibliographic codes from ISO 639, such as 'heb' for Hebrew.
This code set is identified with the symbol "LOCALE_LANG_ALPHA_3".
term
This is the set of three-letter (lowercase) terminologic codes from ISO 639.
This code set is identified with the symbol "LOCALE_LANG_TERM".
ROUTINES
code2language ( CODE [,CODESET] )
language2code ( NAME [,CODESET] )
language_code2code ( CODE ,CODESET ,CODESET2 )
all_language_codes ( [CODESET] )
all_language_names ( [CODESET] )
Locale::Language::rename_language ( CODE ,NEW_NAME [,CODESET] )
Locale::Language::add_language ( CODE ,NAME [,CODESET] )
Locale::Language::delete_language ( CODE [,CODESET] )
Locale::Language::add_language_alias ( NAME ,NEW_NAME )
Locale::Language::delete_language_alias ( NAME )
Locale::Language::rename_language_code ( CODE ,NEW_CODE [,CODESET] )
Locale::Language::add_language_code_alias ( CODE ,NEW_CODE [,CODESET] )
Locale::Language::delete_language_code_alias ( CODE [,CODESET] )
These routines are all documented in the Locale::Codes man page.
SEE ALSO
Locale::Codes
Locale::Constants
http://www.loc.gov/standards/iso639-2/
Source of the ISO 639 codes.
AUTHOR
See Locale::Codes for full author history.
Currently maintained by Sullivan Beck (sbeck@cpan.org).
COPYRIGHT
Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
Copyright (c) 2001-2010 Neil Bowers
Copyright (c) 2010-2011 Sullivan Beck
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2011-09-26 Locale::Language(3perl)