Sponsored Content
Full Discussion: counting using awk
Top Forums Shell Programming and Scripting counting using awk Post 302525732 by mirni on Friday 27th of May 2011 09:28:13 PM
Old 05-27-2011
Hi and welcome to the forum.
Try to break your problem down into simpler sub-tasks. E.g: you want the top 10 counts, so it would make sense to sort your input files first:
Code:
sort -n -k2,2 -r file1

will do a numeric (-n) sort descending (reverse -r) on second field (-k2,2). Now to find the top 10, you just need to look at the first 10 lines.
So I'd approach this with feeding the sorted files into awk:
Code:
awk '#do the hard work' <(sort -nrk2,2 file1) <(sort -nrk2,2 file2)  <(sort -nrk2,2 file3)

Now to pull the maximum of the top 10 from each input, you could do something like:
Code:
awk 'FNR<=10{  #I only care about the first ten lines in each file
  if($2>cnt[$1])    #get the global max among the files
    cnt[$1]=$2
}
END{
  for(i in cnt)
    print i "  " cnt[i]
}' <(sort -nrk2,2 file1) <(sort -nrk2,2 file2)  <(sort -nrk2,2 file3) >output.txt

output.txt should now contain something like:
Code:
chr_1_1_50  100
chr3_101_150  65
chr1_1_50  120

(in random order, since 'for(i in cnt)' doesn't sort anything).

I don't quite understand what do you mean by
Quote:
they need to be picked from file2 and file3
or what is your desired output. But if you take it one small step at a time, you're gonna eventually get there.
E.g. you could read the lines from output.txt and grep for the name in the input files to get the other values:
Code:
while read name count ; do 
  grep $name file1 >> globalTop10inFile1.txt
done < output.txt

etc.

Approaching the problem in this step-by-step fashion, it's much easier to debug -- you can verify the intermediate results easily.
Give it a shot and let us know how it goes!

mirni
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Counting records with AWK

I've been working with an awk script and I'm wondeing id it's possible to count records in a file which DO NOT contain, in this instance fields 12 and 13. With the one script I am wanting to display the count for the records WITH fields 12 and 13 and a seperate count of records WITHOUT fields... (2 Replies)
Discussion started by: Glyn_Mo
2 Replies

2. Shell Programming and Scripting

Counting with Awk

I need "awk solution" for simple counting! File looks like: STUDENT GRADE student1 A student2 A student3 B student4 A student5 B Desired Output: GRADE No.of Students A 3 B 2 Thanks for awking! (4 Replies)
Discussion started by: saint2006
4 Replies

3. Shell Programming and Scripting

counting non integer number in awk

Hi, I am having the following number in the file tmp 31013.004 20675.336 43318.190 30512.926 48992.559 277893.111 41831.330 8749.113 415980.576 28273.054 I want to add these numbers, I am using following script awk 'END{print s}{s += $1}' tmp its giving answer 947239 which is correct,... (3 Replies)
Discussion started by: chaitubek
3 Replies

4. Shell Programming and Scripting

awk finding counting sequence

Can awk count numbers until it reaches the end of the sequence after the slash? input: serv1a, 32, 41/47, 53, 89/100, 108/11, 113. serv1b, 1/2, 114/18, 121/35, 139/40, 143/55, 159/64, serv2, 255/56, 274/77, 763, 774/75, 777, 1434/35, 1444/50, 1715, 2025/31, 2048. serv10b, 804, 808, 929/32,... (9 Replies)
Discussion started by: sdf
9 Replies

5. Shell Programming and Scripting

Counting Instances of a String with AWK

I have a list of URLs and I want to be able to count the number of instances of addresses ending in a certain TLD and output and sort it like so. 5 bdcc.com 48 zrtzr.com 49 rvo.com Input is as so ync.org sduzj.edu sduzj.edu sduzj.edu sduzj.edu sduzj.edu sduzj.edu sduzj.edu... (1 Reply)
Discussion started by: Pjstaab
1 Replies

6. Shell Programming and Scripting

Counting Fields with awk

ok, so a user can specify options as is shown below: ExA: cpu.pl!23!25!-allow or ExB: cpu.pl!23!25!-block!all options are delimited by the exclamation mark. now, in example A, there are 4 options provided by the user. in example B, there are 5 options provided by the user. ... (3 Replies)
Discussion started by: SkySmart
3 Replies

7. Shell Programming and Scripting

awk counting question

Probably a simple to this, but unsure how to do it. I would prefer an AWK solution. Below is the data set. 1 2 3 2 5 7 4 6 9 1 5 4 8 5 7 1 1 10 15 3 12 3 7 9 9 8 10 4 5 2 9 1 10 4 7 9 7 12 6 9 13 8 For the second... (11 Replies)
Discussion started by: mollydog11
11 Replies

8. UNIX for Dummies Questions & Answers

Awk: Counting occurrences between two files

Hi, I have two text files (1.txt and 2.txt). 2.txt contains two columns which are extracted from 1.txt using a simple if(condition) print. I want to: - count how many times the values contained in 2.txt appear in 1.txt -if they appear just one time, I have to delete the entire row in... (5 Replies)
Discussion started by: Pintug
5 Replies

9. Shell Programming and Scripting

Counting lines in a file using awk

I want to count lines of a file using AWK (only) and not in the END part like this awk 'END{print FNR}' because I want to use it. Does anyone know of a way? Thanks a lot. (7 Replies)
Discussion started by: guitarist684
7 Replies

10. UNIX for Dummies Questions & Answers

Word-counting and substitution with awk

Hi!! I am trying to write a program which allows me to count how many times I used the same word in a text: {$0 = tolower ($0) gsub (/_]/, "", $0) for (i = 1; i <= NF; i++) freq++ } END { for (word in freq) printf "%s\t%d\n", word, freq It seems work but... (3 Replies)
Discussion started by: ettore8888
3 Replies
JOIN(1) 						    BSD General Commands Manual 						   JOIN(1)

NAME
join -- relational database operator SYNOPSIS
join [-a file_number | -v file_number] [-e string] [-j file_number field] [-o list] [-t char] [-1 field] [-2 field] file1 file2 DESCRIPTION
The join utility performs an ``equality join'' on the specified files and writes the result to the standard output. The ``join field'' is the field in each file by which the files are compared. The first field in each line is used by default. There is one line in the output for each pair of lines in file1 and file2 which have identical join fields. Each output line consists of the join field, the remaining fields from file1 and then the remaining fields from file2. The default field separators are tab and space characters. In this case, multiple tabs and spaces count as a single field separator, and leading tabs and spaces are ignored. The default output field separator is a single space character. Many of the options use file and field numbers. Both file numbers and field numbers are 1 based, i.e. the first file on the command line is file number 1 and the first field is field number 1. The following options are available: -a file_number In addition to the default output, produce a line for each unpairable line in file file_number. (The argument to -a must not be preceded by a space; see the COMPATIBILITY section.) -e string Replace empty output fields with string. -o list The -o option specifies the fields that will be output from each file for each line with matching join fields. Each element of list has the form 'file_number.field', where file_number is a file number and field is a field number. The elements of list must be either comma (``,'') or whitespace separated. (The latter requires quoting to protect it from the shell, or, a simpler approach is to use multiple -o options.) -t char Use character char as a field delimiter for both input and output. Every occurrence of char in a line is significant. -v file_number Do not display the default output, but display a line for each unpairable line in file file_number. The options -v 1 and -v 2 may be specified at the same time. -1 field Join on the field'th field of file 1. -2 field Join on the field'th field of file 2. When the default field delimiter characters are used, the files to be joined should be ordered in the collating sequence of sort(1), using the -b option, on the fields on which they are to be joined, otherwise join may not report all field matches. When the field delimiter char- acters are specified by the -t option, the collating sequence should be the same as sort(1) without the -b option. If one of the arguments file1 or file2 is ``-'', the standard input is used. The join utility exits 0 on success, and >0 if an error occurs. COMPATIBILITY
For compatibility with historic versions of join, the following options are available: -a In addition to the default output, produce a line for each unpairable line in both file 1 and file 2. (To distinguish between this and -a file_number, join currently requires that the latter not include any white space.) -j1 field Join on the field'th field of file 1. -j2 field Join on the field'th field of file 2. -j field Join on the field'th field of both file 1 and file 2. -o list ... Historical implementations of join permitted multiple arguments to the -o option. These arguments were of the form ``file_num- ber.field_number'' as described for the current -o option. This has obvious difficulties in the presence of files named ``1.2''. These options are available only so historic shell scripts don't require modification and should not be used. SEE ALSO
awk(1), comm(1), paste(1), sort(1), uniq(1) STANDARDS
The join command is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. BSD
April 28, 1995 BSD
All times are GMT -4. The time now is 06:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy