Help getting a code in awk - Want to know how much of the data is covered by entries Post: 303037838

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Help getting a code in awk - Want to know how much of the data is covered by entries Post 303037838 by rdrtx1 on Wednesday 14th of August 2019 01:46:11 PM

08-14-2019

Read Only

Code:

awk '
NR > 1 {
   if (!id1[$2]++) {ids[idc++]=$2; len[$2]=$4;}
   for (i=$5; i<$6; i++) if (!value[$2,i]++) coverage[$2]++;
}
END {
   for (i=0; i<idc; i++)
      printf "%d --- %d / %d = %.3f\n", ids[i],
             coverage[ids[i]], len[ids[i]],
             (coverage[ids[i]] / len[ids[i]]);
}
' data

Note: for first line there is only one range of coverage. Check the range in output shown.

Last edited by rdrtx1; 08-14-2019 at 08:54 PM.. Reason: efficiency++

This User Gave Thanks to rdrtx1 For This Post:

rdrtx1

View Public Profile for rdrtx1

Find all posts by rdrtx1

We Also Found This Discussion For You

1. Shell Programming and Scripting

awk code to ignore the first occurence unknown number of rows in a data column

Hello experts, Shown below is the 2 column sample data(there are many data columns in actual input file), Key, Data A, 1 A, 2 A, 2 A, 3 A, 1 A, 1 A, 1 I need the below output. Key, Data A, 2 A, 2 A, 3 A, 1 A, 1 A, 1

LEARN ABOUT DEBIAN

ids2ngram

IDS2NGRAM(1)						User Contributed Perl Documentation					      IDS2NGRAM(1)

NAME

       ids2ngram - generate n-gram data file from ids file

SYNOPSIS

       ids2ngram [option]... ids_file...

DESCRIPTION

       ids2ngram generates idngram file, which is a sorted [id1,..,idN,freq] array, from binary id stream files. Here, the id stream files are
       always generated by mmseg or slmseg. Basically, it finds all occurrence of n-words tuples (i.e. the tuple of (id1,..,idN)), and sorts these
       tuples by the lexicographic order of the ids make up the tuples, then write them to specified output file.

INPUT

       The input file is presented as a binary id stream, which looks like:
	   [id0,...,idX]

OPTIONS

       All the following options are mandatory.

       -n,--NMax N
	   Generates N-gram result. ids2ngram does only support uni-gram, bi-gram, and trigram, so any number not in the range of 1..3 is not
	   valid.

       -s,--swap swap-file
	   Specify the temporary intermediate file.

       -o, --out output-file
	   Specify the result idngram file, e.g. the array of [id1, ..., idN, freq]

       -p, --para N
	   Specify the maximum n-gram items per paragraph. ids2ngram writes to the temporary file on a per-paragraph basis. Every time it writes a
	   paragraph out, it frees the corresponding memory allocated for it. When your computer system permits, a higher N is suggested. This can
	   speed up the processing speed because of less I/O.

EXAMPLE

       Following example will use three input idstream file idsfile[1,2,3] to generate the idngram file all.id3gram. Each para (internal map size
       or hash size) would be 1024000, using swap file for temp result. All temp para result would eventually be merged to got the final result.

       ids2ngram -n 3 -s /tmp/swap -o all.id3gram -p 1024000 idsfile1 idsfile2 idsfile3

AUTHOR

       Originally written by Phill.Zhang <phill.zhang@sun.com>.  Currently maintained by Kov.Chai <tchaikov@gmail.com>.

SEE ALSO

       mmseg(1), slmseg(1), slmbuild (1).

perl v5.14.2							    2012-06-09							      IDS2NGRAM(1)

We Also Found This Discussion For You

1. Shell Programming and Scripting

awk code to ignore the first occurence unknown number of rows in a data column

Discussion started by: ks_reddy

LEARN ABOUT DEBIAN

ids2ngram