I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records.
contents of file looks like
30002157,40342424,OTC,mart_rec,100, ,0
30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Trying to cut down the size of some log files. Now that I write this out it looks more dificult than i thought it would be.
Need a bash script or command that goes sequentially through all lines of a file, and does this:
if field1 (space separated) is the number 2012 print the entire line. Do... (7 Replies)
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Hi,
I have a file with fields like below:
A;XYZ;102345;222
B;XYZ;123243;333
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
desired output:
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
ie, if the 4rth field is a duplicate.. i need only those records where... (5 Replies)
Hi All,
i have input file like below...
CA009156;20091003;M;AWBKCA72;123;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;321;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;231;;CANADIAN... (2 Replies)
Hello,
I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Hi all,
I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following:
FILE:
1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100
2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY)
3. CDC, 54321, TEST3,... (4 Replies)
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT DEBIAN
devel::refcount
Devel::Refcount(3pm) User Contributed Perl Documentation Devel::Refcount(3pm)NAME
"Devel::Refcount" - obtain the REFCNT value of a referent
SYNOPSIS
use Devel::Refcount qw( refcount );
my $anon = [];
print "Anon ARRAY $anon has " . refcount($anon) . " reference
";
my $otherref = $anon;
print "Anon ARRAY $anon now has " . refcount($anon) . " references
";
DESCRIPTION
This module provides a single function which obtains the reference count of the object being pointed to by the passed reference value.
FUNCTIONS
$count = refcount($ref)
Returns the reference count of the object being pointed to by $ref.
COMPARISON WITH SvREFCNT
This function differs from "Devel::Peek::SvREFCNT" in that SvREFCNT() gives the reference count of the SV object itself that it is passed,
whereas refcount() gives the count of the object being pointed to. This allows it to give the count of any referent (i.e. ARRAY, HASH,
CODE, GLOB and Regexp types) as well.
Consider the following example program:
use Devel::Peek qw( SvREFCNT );
use Devel::Refcount qw( refcount );
sub printcount
{
my $name = shift;
printf "%30s has SvREFCNT=%d, refcount=%d
",
$name, SvREFCNT($_[0]), refcount($_[0]);
}
my $var = [];
printcount 'Initially, $var', $var;
my $othervar = $var;
printcount 'Before CODE ref, $var', $var;
printcount '$othervar', $othervar;
my $code = sub { undef $var };
printcount 'After CODE ref, $var', $var;
printcount '$othervar', $othervar;
This produces the output
Initially, $var has SvREFCNT=1, refcount=1
Before CODE ref, $var has SvREFCNT=1, refcount=2
$othervar has SvREFCNT=1, refcount=2
After CODE ref, $var has SvREFCNT=2, refcount=2
$othervar has SvREFCNT=1, refcount=2
Here, we see that SvREFCNT() counts the number of references to the SV object passed in as the scalar value - the $var or $othervar
respectively, whereas refcount() counts the number of reference values that point to the referent object - the anonymous ARRAY in this
case.
Before the CODE reference is constructed, both $var and $othervar have SvREFCNT() of 1, as they exist only in the current lexical pad. The
anonymous ARRAY has a refcount() of 2, because both $var and $othervar store a reference to it.
After the CODE reference is constructed, the $var variable now has an SvREFCNT() of 2, because it also appears in the lexical pad for the
new anonymous CODE block.
PURE-PERL FALLBACK
An XS implementation of this function is provided, and is used by default. If the XS library cannot be loaded, a fallback implementation in
pure perl using the "B" module is used instead. This will behave identically, but is much slower.
Rate pp xs
pp 225985/s -- -66%
xs 669570/s 196% --
SEE ALSO
o Test::Refcount - assert reference counts on objects
AUTHOR
Paul Evans <leonerd@leonerd.org.uk>
perl v5.14.2 2011-11-15 Devel::Refcount(3pm)