Sponsored Content
Top Forums Shell Programming and Scripting Help with Perl script for identifying dupes in column1 Post 302932822 by Skrynesaver on Sunday 25th of January 2015 05:35:21 AM
Old 01-25-2015
In Perl...
Code:
#! /usr/bin/perl
use strict;
use warnings;
open (my $words, '<', $ARGV[0]);
my (%seen,@unique,@dupe);
while($words){
        if (/^(\w+)=\w+$/){
                if ($seen{$1}<1){
                        $seen{$1}++;
                        push (@unique,$_);
                }
                else{
                        push @dupe, $_;
                }
        }
        else{
                print "Not a word definition:$_"
        }
}
print "UNIQUE WORDS\n\n", join ('',@unique),
        "DUPLICATES\n\n", join ('',@dupe);

This User Gave Thanks to Skrynesaver For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Identifying prior date in shell script

Hi guys, I'm a newbie to shell script. I have to write a shell script that is supposed to give me a date which should be older than the current date when I supply "no. of days" as a command line parameter to the script. (i.e)., if I am giving the no. of days value as 305, the script should... (5 Replies)
Discussion started by: royalibrahim
5 Replies

2. Red Hat

identifying SU mode by shell script

the following bash script is not working in fedora-11 Could anyone help me please? #!/bin/bash if -ne 0];then echo " you are in root" else echo " you must be in root -su mode??" fi exit (2 Replies)
Discussion started by: Turtel
2 Replies

3. Shell Programming and Scripting

Using an awk script to identify dupes in two files

Hello, I have two files. File1 or the master file contains two columns separated by a delimiter: a=b b=d e=f g=h File 2 which is the file to be processed has only a single column a h c b What I need is an awk script to identify unique names from file 2 which are not found in the... (6 Replies)
Discussion started by: gimley
6 Replies

4. Shell Programming and Scripting

Script for identifying and deleting dupes in a line

I am compiling a synonym dictionary which has the following structure Headword=Synonym1,Synonym2 and so on, with each synonym separated by a comma. As is usual in such cases manual preparation of synonyms results in repeating the synonym which results in dupes as in the example below:... (3 Replies)
Discussion started by: gimley
3 Replies

5. Shell Programming and Scripting

Removing Dupes from huge file- awk/perl/uniq

Hi, I have the following command in place nawk -F, '!a++' file > file.uniq It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error: bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies

6. Shell Programming and Scripting

Help in modifying existing Perl Script to produce report of dupes

Hello, I have a large amount of data with the following structure: Word=Transliterated word I have written a Perl Script (reproduced below) which goes through the full file and identifies all dupes on the right hand side. It creates successfully a new file with two headers: Singletons and Dupes.... (5 Replies)
Discussion started by: gimley
5 Replies

7. Shell Programming and Scripting

Awk - Script assistance on identifying non matching fields

Hoping for some assistance. my source file consists of: os, ip, username win7, 123.56.78, john win7, 123.56.78, paul win7, 10.1.1.1, john win7, 10.2.2.3, joe I've been trying to run a script that will only return ip and username where the IP address is the same and the username is... (3 Replies)
Discussion started by: tekvaio
3 Replies

8. Shell Programming and Scripting

Identifying dupes within a database and creating unique sub-sets

Hello, I have a database of name variants with the following structure: variant=variant=variant The number of variants can be as many as thirty to forty. Since the database is quite large (at present around 60,000 lines) duplicate sets of variants creep in. Thus John=Johann=Jon and... (2 Replies)
Discussion started by: gimley
2 Replies

9. Programming

Perl script to merge cells in column1 which has same strings, for all sheets in a excel workbook

Perl script to merge cells ---------- Post updated at 12:59 AM ---------- Previous update was at 12:54 AM ---------- I am using below code to read files from a dir and print to excel. open(my $in, '<', $file) or die "Could not open file: $!"; my $rowCount = 0; my $colCount = 0;... (11 Replies)
Discussion started by: Jack_Bruce
11 Replies

10. Shell Programming and Scripting

Modify script to remove dupes with two delimiters

Hello, I have a script which removes duplicates in a database with a single delimiter = The script is given below: # script to remove dupes from a row with structure word=word BEGIN{FS="="} {for(i=1;i<=NF;i++){a++;}for(i in a){b=b"="i}{sub("=","",b);$0=b;b="";delete a}}1 How do I modify... (6 Replies)
Discussion started by: gimley
6 Replies
TAP::Parser::SourceHandler::Perl(3pm)			User Contributed Perl Documentation		     TAP::Parser::SourceHandler::Perl(3pm)

NAME
TAP::Parser::SourceHandler::Perl - Stream TAP from a Perl executable VERSION
Version 3.25 SYNOPSIS
use TAP::Parser::Source; use TAP::Parser::SourceHandler::Perl; my $source = TAP::Parser::Source->new->raw( 'script.pl' ); $source->assemble_meta; my $class = 'TAP::Parser::SourceHandler::Perl'; my $vote = $class->can_handle( $source ); my $iter = $class->make_iterator( $source ); DESCRIPTION
This is a Perl TAP::Parser::SourceHandler - it has 2 jobs: 1. Figure out if the TAP::Parser::Source it's given is actually a Perl script ("can_handle"). 2. Creates an iterator for Perl sources ("make_iterator"). Unless you're writing a plugin or subclassing TAP::Parser, you probably won't need to use this module directly. METHODS
Class Methods "can_handle" my $vote = $class->can_handle( $source ); Only votes if $source looks like a file. Casts the following votes: 0.9 if it has a shebang ala "#!...perl" 0.75 if it has any shebang 0.8 if it's a .t file 0.9 if it's a .pl file 0.75 if it's in a 't' directory 0.25 by default (backwards compat) "make_iterator" my $iterator = $class->make_iterator( $source ); Constructs & returns a new TAP::Parser::Iterator::Process for the source. Assumes "$source->raw" contains a reference to the perl script. "croak"s if the file could not be found. The command to run is built as follows: $perl @switches $perl_script @test_args The perl command to use is determined by "get_perl". The command generated is guaranteed to preserve: PERL5LIB PERL5OPT Taint Mode, if set in the script's shebang Note: the command generated will not respect any shebang line defined in your Perl script. This is only a problem if you have compiled a custom version of Perl or if you want to use a specific version of Perl for one test and a different version for another, for example: #!/path/to/a/custom_perl --some --args #!/usr/local/perl-5.6/bin/perl -w Currently you need to write a plugin to get around this. "get_taint" Decode any taint switches from a Perl shebang line. # $taint will be 't' my $taint = TAP::Parser::SourceHandler::Perl->get_taint( '#!/usr/bin/perl -t' ); # $untaint will be undefined my $untaint = TAP::Parser::SourceHandler::Perl->get_taint( '#!/usr/bin/perl' ); "get_perl" Gets the version of Perl currently running the test suite. SUBCLASSING
Please see "SUBCLASSING" in TAP::Parser for a subclassing overview. Example package MyPerlSourceHandler; use strict; use vars '@ISA'; use TAP::Parser::SourceHandler::Perl; @ISA = qw( TAP::Parser::SourceHandler::Perl ); # use the version of perl from the shebang line in the test file sub get_perl { my $self = shift; if (my $shebang = $self->shebang( $self->{file} )) { $shebang =~ /^#!(.*perl.*?)(?:(?:s)|(?:$))/; return $1 if $1; } return $self->SUPER::get_perl(@_); } SEE ALSO
TAP::Object, TAP::Parser, TAP::Parser::IteratorFactory, TAP::Parser::SourceHandler, TAP::Parser::SourceHandler::Executable, TAP::Parser::SourceHandler::File, TAP::Parser::SourceHandler::Handle, TAP::Parser::SourceHandler::RawTAP perl v5.14.2 2012-06-05 TAP::Parser::SourceHandler::Perl(3pm)
All times are GMT -4. The time now is 02:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy