Hi,
is it possible to remove all duplicate lines from all txt files in a specific folder?
This is too hard for me maybe someone could help.
lets say we have an amount of textfiles 1 or 2 or 3 or... maximum 50
each textfile has lines with text.
I want all lines of all textfiles... (8 Replies)
I have a data set that has 4 columns, I want to know if I can delete duplicate lines while ignoring one of the columns, for example
10 chr1 ASF 30
15 chr1 ASF 20
5 chr1 ASF 30
6 chr2 EBC 15
4 chr2 EBC 30
...
I want to know if I can delete duplicate lines while ignoring column 1, so the... (5 Replies)
Dear All,
I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files.
Duplicate values may come in different files.... all files laying in single directory..
Need help to... (3 Replies)
Hi,
I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates.
Thanks in advance,
sudvishw :confused: (7 Replies)
Hi,
I am on a Solaris8 machine
If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error"
...or
Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great.
******************
... (3 Replies)
So, I have text files,
one "fail.txt"
And one
"color.txt"
I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file.
Afterwards there shall be no duplicate lines. (1 Reply)
Hi All
I have a list of files which will have duplicate list of blocks of text. Following is a sample of the file, I have removed the sensitive information from the file.
All the code samples starts from <TR BGCOLOR="white"> and Ends with IP address and two html tags like this.
10.14.22.22... (3 Replies)
I have two large files (~250GB) that I am trying to remove the where GT: 0/0 or 1/1 or 2/2 for both files. I was going to use a bash with the below awk, which I think will find each line but how do I remove that line is that condition is found? Thank you :).
Input
20 60055 . A ... (4 Replies)
I create a CGI in bash/html.
My awk script looks like :
echo "<table>"
for fn in /var/www/cgi-bin/LPAR_MAP/*;
do
echo "<td>"
echo "<PRE>"
awk -F',|;' -v test="$test" '
NR==1 {
split(FILENAME ,a,"");
}
$0 ~ test {
if(!header++){
... (12 Replies)
Discussion started by: Tim2424
12 Replies
LEARN ABOUT DEBIAN
devel::refcount
Devel::Refcount(3pm) User Contributed Perl Documentation Devel::Refcount(3pm)NAME
"Devel::Refcount" - obtain the REFCNT value of a referent
SYNOPSIS
use Devel::Refcount qw( refcount );
my $anon = [];
print "Anon ARRAY $anon has " . refcount($anon) . " reference
";
my $otherref = $anon;
print "Anon ARRAY $anon now has " . refcount($anon) . " references
";
DESCRIPTION
This module provides a single function which obtains the reference count of the object being pointed to by the passed reference value.
FUNCTIONS
$count = refcount($ref)
Returns the reference count of the object being pointed to by $ref.
COMPARISON WITH SvREFCNT
This function differs from "Devel::Peek::SvREFCNT" in that SvREFCNT() gives the reference count of the SV object itself that it is passed,
whereas refcount() gives the count of the object being pointed to. This allows it to give the count of any referent (i.e. ARRAY, HASH,
CODE, GLOB and Regexp types) as well.
Consider the following example program:
use Devel::Peek qw( SvREFCNT );
use Devel::Refcount qw( refcount );
sub printcount
{
my $name = shift;
printf "%30s has SvREFCNT=%d, refcount=%d
",
$name, SvREFCNT($_[0]), refcount($_[0]);
}
my $var = [];
printcount 'Initially, $var', $var;
my $othervar = $var;
printcount 'Before CODE ref, $var', $var;
printcount '$othervar', $othervar;
my $code = sub { undef $var };
printcount 'After CODE ref, $var', $var;
printcount '$othervar', $othervar;
This produces the output
Initially, $var has SvREFCNT=1, refcount=1
Before CODE ref, $var has SvREFCNT=1, refcount=2
$othervar has SvREFCNT=1, refcount=2
After CODE ref, $var has SvREFCNT=2, refcount=2
$othervar has SvREFCNT=1, refcount=2
Here, we see that SvREFCNT() counts the number of references to the SV object passed in as the scalar value - the $var or $othervar
respectively, whereas refcount() counts the number of reference values that point to the referent object - the anonymous ARRAY in this
case.
Before the CODE reference is constructed, both $var and $othervar have SvREFCNT() of 1, as they exist only in the current lexical pad. The
anonymous ARRAY has a refcount() of 2, because both $var and $othervar store a reference to it.
After the CODE reference is constructed, the $var variable now has an SvREFCNT() of 2, because it also appears in the lexical pad for the
new anonymous CODE block.
PURE-PERL FALLBACK
An XS implementation of this function is provided, and is used by default. If the XS library cannot be loaded, a fallback implementation in
pure perl using the "B" module is used instead. This will behave identically, but is much slower.
Rate pp xs
pp 225985/s -- -66%
xs 669570/s 196% --
SEE ALSO
o Test::Refcount - assert reference counts on objects
AUTHOR
Paul Evans <leonerd@leonerd.org.uk>
perl v5.14.2 2011-11-15 Devel::Refcount(3pm)