Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk Grouping and Subgrouping with Counts
Top Forums UNIX for Dummies Questions & Answers awk Grouping and Subgrouping with Counts Post 302776019 by JoshCrosby on Tuesday 5th of March 2013 10:16:19 PM
Old 03-05-2013
Code awk Grouping and Subgrouping with Counts
So I have a ton of files, lines in excess of 3 MIL per file.

I need to find a solution to find the top 3 products, and then get the top 5 skews with a count of how many times that skew was viewed.

This is a sample file, shortened it for readability. Each ROW is counted as view.

Here's the sample file.
Code:
product|skew
p1|12345
p2|23456
p3|234
p4|98707
p1|12345
p2|23456
p3|2343
p4|98706
p1|12345
p2|23456
p3|234
p5|36748
p4|98708
p1|12345
p2|23456
p3|234
p4|98708
p1|12345
p6|23467
p2|23456
p3|234345
p4|98708
p1|12345
p2|23456
p3|234345
p4|98707

I can get the first top, but i'm having a tough time getting the second with count. I imagine I will have to create 2 arrays and loop through those to get the correct counts.

Can anybody provide any guidance?

for the first I can do this, piping sort and head, but stuck to get the rest.
Code:
awk -F"|" '{product[$1]++}END{for(n in product) print n, product[n]}' products.txt | sort -k2 -nr | head -2

which prints:
Code:
p4 6
p3 6

Expected result should be something like
Code:
product	skew count
p4	98708 	3
p4	98707	2
p4	98706	2
p3	234 	3
p3	234345	2
etc......

Last edited by JoshCrosby; 03-05-2013 at 11:23 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

counts

How can i do a simple record count in my shell script? i just want to count the number of records i receive from a specific file. (11 Replies)
Discussion started by: k@ssidy
11 Replies

2. UNIX for Dummies Questions & Answers

counts

To start I have a table that has ticketholders. Each ticket holder has a unique number and each ticket holder is associated to a so called household number. You can have multiple guests w/i a household. I would like to create 3 flags (form a, for a household that has 1-4 gst) form b 5-8 gsts... (3 Replies)
Discussion started by: sbr262
3 Replies

3. Shell Programming and Scripting

Grouping using sed/awk ?

I run awk cat $1|awk '{print $6}' and get a lot of results and I want results to group them. For example my result is (o/p is unknown to user) xyz xyz abc pqr xyz pqr etc I wanna group them as xyz=total found 7 abc=total .... pqr= Thank (3 Replies)
Discussion started by: pujansrt
3 Replies

4. Shell Programming and Scripting

awk grouping by name script

Hello I am trying to figure out a script which could group a log file by user names. I worked with awk command and I could trim the log file to: <USER: John Frisbie > /* Thu Aug 06 2009 15:11:45.7974 */ FLOAT GRANT WRITE John Frisbie (500 of 3005 write) <USER: Shawn Sanders > /* Thu Aug 06... (2 Replies)
Discussion started by: Avto
2 Replies

5. Shell Programming and Scripting

AWK script to create max value of 3rd column, grouping by first column

Hi, I need an awk script (or whatever shell-construct) that would take data like below and get the max value of 3 column, when grouping by the 1st column. clientname,day-of-month,max-users ----------------------------------- client1,20120610,5 client2,20120610,2 client3,20120610,7... (3 Replies)
Discussion started by: ckmehta
3 Replies

6. Shell Programming and Scripting

awk and perl grouping.

Hello folks. After awk, i have decided to start to learn perl, and i need some help. I have following output : 1 a 1 b 2 k 2 f 3 s 3 p Now with awk i get desired output by issuing : awk ' { a = a FS $2 } END { for ( i in a) print i,a }' input 1 a b 2 k f 3 s p Can... (1 Reply)
Discussion started by: Peasant
1 Replies

7. Shell Programming and Scripting

grouping using sed or awk

I have below inside a file. 11.22.33.44 user1 11.22.33.55 user2 I need this manipulated as alias server1.domain.com='ssh user1@11.22.33.44' alias server2.domain.com='ssh user2@11.22.33.55' (3 Replies)
Discussion started by: anil510
3 Replies

8. UNIX for Dummies Questions & Answers

awk adding counts together from column

Hello Im new treat me nicely, I have a headache :) I have a script that seemed to work now it doesnt anyway, the last part is adding counts of unique items in a csv file eg 05492U34 38 05492U34 47 two columns, (many different values like this in file) i want... (7 Replies)
Discussion started by: aniquebmx
7 Replies

9. Shell Programming and Scripting

Grouping and Subgrouping using awk

I have a data which looks like 1440993600|L|ABCDEF 1440993600|L|ABCD 1440993601|L|ABCDEF 1440993602|L|ABC 1440993603|L|ABCDE . . . 1441015200|L|AB 1441015200|L|ABC 1441015200|L|ABCDEF So basically, the $1 is epoch date, $2 and $3 is some application data From one if the... (5 Replies)
Discussion started by: hemanty4u
5 Replies

10. Shell Programming and Scripting

Output counts of all matching strings lessthan a number using awk

The awk below is supposed to count all the matching $5 strings and count how many $7 values is less than 20. I don't think I need the portion in bold as I do not need any decimal point or format, but can not seem to get the correct counts. Thank you :). file chr5 77316500 77316628 ... (6 Replies)
Discussion started by: cmccabe
6 Replies

LEARN ABOUT MOJAVE

locale::codes::langext5.18

Locale::Codes::LangExt(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangExt(3pm)

NAME

       Locale::Codes::LangExt - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangExt;

	  $lext = code2langext('acm');		       # $lext gets 'Mesopotamian Arabic'
	  $code = langext2code('Mesopotamian Arabic'); # $code gets 'acm'

	  @codes   = all_langext_codes();
	  @names   = all_langext_names();

DESCRIPTION

       The "Locale::Codes::LangExt" module provides access to standard codes used for identifying language extensions, such as those as defined in
       the IANA language registry.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default IANA language
       registry codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language extensions. A code set may be specified using either a name, or
       a constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langext('acm','alpha');
	  $lext = code2langext('acm',LOCALE_LANGEXT_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from the IANA language registry, such as 'acm' for Mesopotamian Arabic.

	   This is the default code set.

ROUTINES

       code2langext ( CODE [,CODESET] )
       langext2code ( NAME [,CODESET] )
       langext_code2code ( CODE ,CODESET ,CODESET2 )
       all_langext_codes ( [CODESET] )
       all_langext_names ( [CODESET] )
       Locale::Codes::LangExt::rename_langext  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangExt::add_langext  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangExt::delete_langext  ( CODE [,CODESET] )
       Locale::Codes::LangExt::add_langext_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangExt::delete_langext_alias  ( NAME )
       Locale::Codes::LangExt::rename_langext_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangExt::add_langext_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangExt::delete_langext_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.iana.org/assignments/language-subtag-registry
	   The IANA language subtag registry.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.18.2							    2013-11-04					       Locale::Codes::LangExt(3pm)
All times are GMT -4. The time now is 07:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy