Sponsored Content
Top Forums Shell Programming and Scripting Perl- Finding average "frequency" of occurrence of duplicate lines Post 302545287 by acsg on Tuesday 9th of August 2011 12:49:37 AM
Old 08-09-2011
Perl- Finding average "frequency" of occurrence of duplicate lines

Hello,

I am working with a perl script that tries to find the average "frequency" in which lines are duplicated. So far I've only managed to find the way to count how many times the lines are repeated, the code is as follows:

Code:
perl -ae'
my $filename= $ENV{'i'};
open (FILE, "$filename") or die  $!;

my %seen= ();


while(my $line = <FILE>){
  my @fields = split(/\s+/, $line);
  my @fields2= @fields[3..16];
  my $niin= join("\t", @fields2);
  $seen{$niin}++;
  }

foreach my $keys (sort {$seen{$b} <=> $seen{$a}} keys %seen){
    print "$keys = $seen{$keys}\n";
}

close (FILE);


'

Which produces this type of output:

Code:
225    1    225    2    225    3    225    4    225    5    225    6    225    7 = 31789
225    10    225    11    225    12    225    13    225    14    225    15    225    0 = 31772
225    8    225    9    225    10    225    11    225    12    225    13    225    14 = 31714
225    3    225    4    225    5    225    6    225    7    225    8    225    9 = 31686

Now, what I want to do is find a way to find out the number of (in average) "every how many lines a certain line is repeated". So I was wondering if it's possible to have some sort of record and then in the end just calculate the average?

I actually have another way to calculate this frequency. In the original file being read, the first field is a unix timestamp (which i "cut out" for the counting of the duplicate lines). So I thought it would be possible as well to try to keep a record of the "time between repetitions" and then make an average in the end. Of course this would imply keeping a record for each duplicate line, which seems like a rather intricate operation. An example of the lines is :

Code:
1301892853.870    1316    efc0696e        225    1    225    2    225    3    225    4    225    5    225    6    225    7

The first field being the unix timestamp. The first, second and third field are ignored for the comparison of duplicate lines.

Any help is deeply appreciated.

---------- Post updated 08-09-11 at 07:49 AM ---------- Previous update was 08-08-11 at 08:40 AM ----------

Is this really not accomplishable the way I asked for in perl? Is there any other way to do it? Any ideas please? Smilie

Thanks again...

Last edited by pludi; 08-08-2011 at 03:24 AM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies

2. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

3. Solaris

"Load Average" vs "virtual processor"

Hi, I have one question regarding the understanding of “load average” in a platform with virtual processors. Suppose in this situation: Total number of physical processors: 1 Number of virtual processors: 32 Total number of cores: 4 Number of cores per physical... (1 Reply)
Discussion started by: MDING
1 Replies

4. Shell Programming and Scripting

finding the strings beween 2 characters "/" & "/" in .txt file

Hi all. I have a .txt file that I need to sort it My file is like: 1- 88 chain0 MASTER (FF-TE) FFFF 1962510 /TCK T FD2TQHVTT1 /jtagc/jtag_instreg/updateinstr_reg_1 dff1 (TI,SO) 2- ... (10 Replies)
Discussion started by: Behrouzx77
10 Replies

5. Shell Programming and Scripting

Cant get awk 1liner to remove duplicate lines from Delimited file, get "event not found" error..help

Hi, I am on a Solaris8 machine If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error" ...or Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great. ****************** ... (3 Replies)
Discussion started by: andy b
3 Replies

6. Shell Programming and Scripting

Find lines with "A" then change "E" to "X" same line

I have a bunch of random character lines like ABCEDFG. I want to find all lines with "A" and then change any "E" to "X" in the same line. ALL lines with "A" will have an "X" somewhere in it. I have tried sed awk and vi editor. I get close, not quite there. I know someone has already solved this... (10 Replies)
Discussion started by: nightwatchrenba
10 Replies

7. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

8. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

9. AIX

Apache 2.4 directory cannot display "Last modified" "Size" "Description"

Hi 2 all, i have had AIX 7.2 :/# /usr/IBMAHS/bin/apachectl -v Server version: Apache/2.4.12 (Unix) Server built: May 25 2015 04:58:27 :/#:/# /usr/IBMAHS/bin/apachectl -M Loaded Modules: core_module (static) so_module (static) http_module (static) mpm_worker_module (static) ... (3 Replies)
Discussion started by: penchev
3 Replies
BP_OLIGO_COUNT(1p)					User Contributed Perl Documentation					BP_OLIGO_COUNT(1p)

NAME
oligo_count - oligo count and frequency SYNOPSIS
Usage: oligo_count [-h/--help] [-l/--length OLIGOLENGTH] [-f/--format SEQFORMAT] [-i/--in/-s/--sequence SEQFILE] [-o/--out OUTFILE] DESCRIPTION
This scripts counts occurrence and frequency for all oligonucleotides of given length. It can be used to determine what primers are useful for frequent priming of nucleic acid for random labeling. Note that this script could be run by utilizing the compseq program which is part of EMBOSS. OPTIONS
The default sequence format is fasta. If no outfile is given, the results will be printed to standard out. All other options can entered interactively. FEEDBACK
Mailing Lists User feedback is an integral part of the evolution of this and other Bioperl modules. Send your comments and suggestions preferably to the Bioperl mailing list. Your participation is much appreciated. bioperl-l@bioperl.org - General discussion http://bioperl.org/wiki/Mailing_lists - About the mailing lists Reporting Bugs Report bugs to the Bioperl bug tracking system to help us keep track of the bugs and their resolution. Bug reports can be submitted via the web: https://redmine.open-bio.org/projects/bioperl/ AUTHOR - Charles C. Kim Email cckim@stanford.edu HISTORY
Written July 2, 2001 Submitted to bioperl scripts project 2001/08/06 >> 100 x speed optimization by Heikki Lehvaslaiho perl v5.14.2 2012-03-02 BP_OLIGO_COUNT(1p)
All times are GMT -4. The time now is 03:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy