Filtering duplicates based on lookup table and rules
please help solving the following. I have access to redhat linux cluster having 32gigs of ram.
I have duplicate ids for variable names, in the file 1,2 are duplicates;3,4 and 5 are duplicates;6 and 7 are duplicates. My objective is to use only the first occurrence of these duplicates.
Lookup file
I need to use the following rules to filter the file below per category.
1) If all duplicates ids within a category have the same value, use the first occurrence and print the value.
example input
example output
2) If all duplicates within a category do not have the same value, print the first occurrence and print the value as ambiguous.
example input
example output
3) If only a single id (out of duplicate ids) is present in a category, then print the row as it is.
Data sample input
Filtered sample output
Last edited by ritakadm; 10-10-2014 at 12:51 AM..
Reason: added code tags for clarity
hi,
i am very much new in perl and have this very basic question in the same:(
the requirement is as below:
i have an input file (txt file) in which i have fields invoice number and customer number. Now i have to take input this combination of invoice n customer number and check in a... (2 Replies)
Using AIX 5.2, Bourne and Korn Shell.
I have two flat text files. One is a main file and one is a lookup table that contains a number of letter codes and membership numbers as follows:
316707965EGM01
315672908ANM92
Whenever one of these records from the lookup appears in the main file... (6 Replies)
Good Evening,
I started working on the 17x17 4-colouring challenge, and I ran into a bit of an I/O snag.
It was an enormous headache to detect the differences in very similar 289-char strings.
Eventually, it made more sense to associate a CRC-Digest with each colouring.
After learning... (0 Replies)
I have a file with the following format
--TABLEA_START--
field1=data1;field2=data2;field3=data3
--TABLEA_END--
--TABLEB_START--
field1=data1;field2=data2;field3=data3
--TABLEB_END--
--TABLEA_START--
field1=data1;field2=data2;field3=data3
... (0 Replies)
Dear all thanks for helping in advance.. Know this should be fairly simple but I failed in searching for an answer.
I have a file (replacement table) containing two columns, e.g.:
ACICJ ACIDIPHILIUM
ACIF2 ACIDITHIOBACILLUS
ACIF5 ACIDITHIOBACILLUS
ACIC5 ACIDOBACTERIUM
ACIC1 ACIDOTHERMUS... (10 Replies)
Hello,
I want to filter all the duplicates of a record to one place. Sample input and output will give you better idea.
I am new to unix. Can some one help me on this?
Input:
7488 7389 chr1.fa chr1.fa
3546 9887 chr5.fa chr9.fa
7387 7898 chrX.fa chr3.fa
7488 7389 chr1.fa chr1.fa... (2 Replies)
Hi,
I have a huge text file with filenames which which looks like the following ie uniquenumber_version_filename:
e.g.
1234_1_xxxx
1234_2_vfvfdbb
343333_1_vfvfdvd
2222222_1_ggggg
55555_1_xxxxxx
55555_2_vrbgbgg
55555_3_grgrbr
What I need to do is examine the file, look for... (4 Replies)
1. how to get the filter option on table so that user can enter the fields which ever they want to print only according to the need ?
2.how to print the full fledge table if there is no value in the rows of the table but it should print the whole rows and column in proper tabular form? (2 Replies)
Hi folks,
I have a log file in the below format and trying to get the output of the unique ones based on mnemonic IN PERL.
Could any one please let me know with the code and the logic ?
Severity Mnemonic Log Message
7 CLI_SCHEDULER Logfile for scheduled CLI... (3 Replies)
Hi All
I need to pass country code into a pipe delimited file for lookup.
It will search country code (column 3) in the file, if the country code matched, it will return value from other columns.
Here is my mapping file.
#CountryName|CountryRegion|CountryCode-3|CountryCode-2... (5 Replies)
Discussion started by: lafrance
5 Replies
LEARN ABOUT CENTOS
locale::codes::langext
Locale::Codes::LangExt(3) User Contributed Perl Documentation Locale::Codes::LangExt(3)NAME
Locale::Codes::LangExt - standard codes for language extension identification
SYNOPSIS
use Locale::Codes::LangExt;
$lext = code2langext('acm'); # $lext gets 'Mesopotamian Arabic'
$code = langext2code('Mesopotamian Arabic'); # $code gets 'acm'
@codes = all_langext_codes();
@names = all_langext_names();
DESCRIPTION
The "Locale::Codes::LangExt" module provides access to standard codes used for identifying language extensions, such as those as defined in
the IANA language registry.
Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default IANA language
registry codes will be used.
SUPPORTED CODE SETS
There are several different code sets you can use for identifying language extensions. A code set may be specified using either a name, or
a constant that is automatically exported by this module.
For example, the two are equivalent:
$lext = code2langext('acm','alpha');
$lext = code2langext('acm',LOCALE_LANGEXT_ALPHA);
The codesets currently supported are:
alpha
This is the set of three-letter (lowercase) codes from the IANA language registry, such as 'acm' for Mesopotamian Arabic.
This is the default code set.
ROUTINES
code2langext ( CODE [,CODESET] )
langext2code ( NAME [,CODESET] )
langext_code2code ( CODE ,CODESET ,CODESET2 )
all_langext_codes ( [CODESET] )
all_langext_names ( [CODESET] )
Locale::Codes::LangExt::rename_langext ( CODE ,NEW_NAME [,CODESET] )
Locale::Codes::LangExt::add_langext ( CODE ,NAME [,CODESET] )
Locale::Codes::LangExt::delete_langext ( CODE [,CODESET] )
Locale::Codes::LangExt::add_langext_alias ( NAME ,NEW_NAME )
Locale::Codes::LangExt::delete_langext_alias ( NAME )
Locale::Codes::LangExt::rename_langext_code ( CODE ,NEW_CODE [,CODESET] )
Locale::Codes::LangExt::add_langext_code_alias ( CODE ,NEW_CODE [,CODESET] )
Locale::Codes::LangExt::delete_langext_code_alias ( CODE [,CODESET] )
These routines are all documented in the Locale::Codes::API man page.
SEE ALSO
Locale::Codes
The Locale-Codes distribution.
Locale::Codes::API
The list of functions supported by this module.
http://www.iana.org/assignments/language-subtag-registry
The IANA language subtag registry.
AUTHOR
See Locale::Codes for full author history.
Currently maintained by Sullivan Beck (sbeck@cpan.org).
COPYRIGHT
Copyright (c) 2011-2013 Sullivan Beck
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.16.3 2013-02-27 Locale::Codes::LangExt(3)