Hey Guys,
I have file which looks like this,
Contig201#numbPA
Contig1452#nmdynD6PA
dm022p15.r#CG6461PA
dm005e16.f#SpatPA
IGU001_0015_A06.f#CG17593PA
I need to remove duplicates based on the chracter matching upto '#'.
for example if we consider this..
Contig201#numbPA... (4 Replies)
Input File is :
-------------
25060008,0040,03,
25136437,0030,03,
25069457,0040,02,
80303438,0014,03,1st
80321837,0009,03,1st
80321977,0009,03,1st
80341345,0007,03,1st
84176527,0047,03,1st
84176527,0047,03,
20000735,0018,03,1st
25060008,0040,03,
I am using the following in the script... (5 Replies)
Hello experts,
I am trying to remove all lines in a csv file where the 2nd columns is a duplicate. I am try to use sort with the key parameter
sort -u -k 2,2 File.csv > Output.csv
File.csv
File Name|Document Name|Document Title|Organization
Word Doc 1.doc|Word Document|Sample... (3 Replies)
example data
5666700842511TAfmoham03151008075205999900000001000001000++
5666700843130MAfmoham03151008142606056667008390315100005001
6666666663130MAfmoham03151008142606056667008390315100005001
I'd like to sort on position 10-14 where the characters are eq "130MA".
Then based on positions... (0 Replies)
Hi,
I'm using the below command to sort and remove duplicates in a file. But, i need to make this applied to the same file instead of directing it to another.
Thanks (6 Replies)
Hi ,
I have below data inside a file named ref.psv . I want to create a shell script which will do the below 2 points :
(1) sort the file content first based on the latest date which is the last column in the file (actual file its the 175th column)
(2)after sorting the file based on latest date... (3 Replies)
I need to use bash to remove duplicates without using sort first.
I can not use:
cat file | sort | uniq
But when I use only
cat file | uniq
some duplicates are not removed. (4 Replies)
I have a large database which has the following structure
a=b
where a is one language and b is the other and = is the delimiter
Since the data treats of language, homographs occur i.e. the same word on the left hand side can map in two different entries to two different glosses on the right... (3 Replies)
I have /tmp dir with filename as:
010020001_S-FOR-Sort-SYEXC_20160229_2212101.marker
010020001_S-FOR-Sort-SYEXC_20160229_2212102.marker
010020001-S-XOR-Sort-SYEXC_20160229_2212104.marker
010020001-S-XOR-Sort-SYEXC_20160229_2212105.marker
010020001_S-ZOR-Sort-SYEXC_20160229_2212106.marker... (4 Replies)
Following is the input. 1st and 3rd block are same(block starts here with '*' and ends before blank line) , 2nd and 4th blocks are also the same:
cat <file>
* Wed Feb 24 2016 Tariq Saeed <tariq.x.saeed@mail.com> 2.0.7-1.0.7
- add vmcore dump support for ocfs2
* Mon Jun 8 2015 Brian Maly... (4 Replies)
Discussion started by: Paras Pandey
4 Replies
LEARN ABOUT DEBIAN
xml::filter::sort::buffermgr
XML::Filter::Sort::BufferMgr(3pm) User Contributed Perl Documentation XML::Filter::Sort::BufferMgr(3pm)NAME
XML::Filter::Sort::BufferMgr - Implementation class used by XML::Filter::Sort
DESCRIPTION
The documentation is targetted at developers wishing to extend or replace this class. For user documentation, see XML::Filter::Sort.
Two classes are used to implement buffering records and spooling them back out in sorted order as SAX events. One instance of the
XML::Filter::Sort::Buffer class is used to buffer each record and one or more instances of the XML::Filter::Sort::BufferMgr class are used
to manage the buffers.
API METHODS
The API of this module as used by XML::Filter::Sort::Buffer consists of the following sequence of method calls:
1. When the first 'record' in a sequence is encountered, XML::Filter::Sort creates a XML::Filter::Sort::BufferMgr object using the "new()"
method.
2. XML::Filter::Sort calls the buffer manager's "new_buffer()" method to get a XML::Filter::Sort::Buffer object and all SAX events are
directed to this object until the end of the record is encountered. The following events are supported by the current buffer
implementation:
start_element()
characters()
comment()
processing_instruction()
end_element()
3. When the end of the record is detected, XML::Filter::Sort calls the buffer manager's "close_buffer()" method, which in turn calls the
buffer's "close()" method. The "close()" method returns a list of values for the sort keys and the buffer manager uses these to store
the buffer for later recall. Subsequent records are handled as per step 2.
4. When the last record has been buffered, XML::Filter::Sort calls the buffer manager's "to_sax()" method. The buffer manager retrieves
each of the buffers in sorted order and calls the buffer's "to_sax()" method.
Each buffer attempts to match the sort key paths as SAX events are received. Once a value has been found for a given key, that same path
match is not attempted against subsequent events. For efficiency, the code to match each key is compiled into a closure. For even more
efficiency, this compilation is done once when the XML::Filter::Sort object is created. The "compile_matches()" method in the buffer
manager class calls the "compile_matches()" method in the buffer class to achieve this.
DATA STRUCTURES
In the current implementation, the XML::Filter::Sort::BufferMgr class simply uses a hash to store the buffer objects. If only one sort key
was defined, only a single hash is required. The values in the hash are arrayrefs containing the list of buffers for records with
identical keys.
If two or more sort keys are defined, the hash values will be XML::Filter::Sort::BufferMgr objects which in turn will contain the buffers.
The following illustration may clarify the relationship (BM=buffer manager, B=buffer):
BM
+----------------+---------------+
| |
BM BM
+-----+--------+ +-----+----------+
| | | |
BM BM BM BM
+-----+----+ +----+------+ +----+----+ +------+------+
| | | | | | | | | | | |
[B,B,B] [B] [B,B] [B] [B,B] [B,B,B] [B] [B,B] [B] [B,B] [B,B,B] [B,B]
This layered storage structure is transparent to the XML::Filter::Sort object which instantiates and interacts with only one buffer manager
(the one at the top of the tree).
COPYRIGHT
Copyright 2002 Grant McLean <grantm@cpan.org>
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.12.4 2002-06-14 XML::Filter::Sort::BufferMgr(3pm)