Sponsored Content
Top Forums Shell Programming and Scripting Determining Word Frequency of Specific Terms Post 302295052 by radoulov on Friday 6th of March 2009 11:46:05 AM
Old 03-06-2009
Yes, try the following code:

Code:
awk 'END {
  print f ":"
    for (Z in z)
      printf "Total number of %s records = %d\n", \
      Z, z[Z]
    if (sc) {
      print "-----------------------------------"
      printf "Total number of Special records = %d\n", \
      sc  
      for (S in sa)
        printf "Total number of %s records = %d\n", \
        S, sa[S]
        }        
    print RS
    }
FNR == 1 {
  if (f) {
    print f ":"
    for (Z in z)
      printf "Total number of %s records = %d\n", \
      Z, z[Z]
    if (sc) {
      print "-----------------------------------"
      printf "Total number of Special records = %d\n", \
      sc
      for (S in sa)
        printf "Total number of %s records = %d\n", \
        S, sa[S]
        }        
    print RS
    split(x, z)
    split(x, sa)
    s = sc = 0
    }
    f = FILENAME
  }    
$3 ~ /^(PTR|MX|NS|CNAME|A)$/ && !s { z[$3]++ }
s && $2 == "IN" { sc++; sa[$3]++ }
/SPECIALS/ { s = 1 }' db*

Be aware that it assumes that the SPECIALS section is always after the main section!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Word frequency with additional information

Hello everyone, I am using a chunk of code to display the frequency of a file name in a list of directories. The code looks like this: find . -name "*.log" | cut -d/ -f4 | cut -d. -f1 | awk '{print $1}' | sort | uniq -c | sort -nr The file paths would look something like this:... (1 Reply)
Discussion started by: ToeLint
1 Replies

2. Shell Programming and Scripting

word frequency counter - awk solution?

Dear all, i need your help on this. There is a text file, i need to count word frequency for each word with frequency >40 in each line of file and output it into another file with columns like this: word1,word2,word3, ...wordn 0,0,1 1,2,0 3,2,0 etc -- each raw represents... (13 Replies)
Discussion started by: irrevocabile
13 Replies

3. Shell Programming and Scripting

Word Frequency Sort

hello, Here is a program for creating a word-frequency # wf.gk --- program to generate word frequencies from a file { # remove punctuation: This will remove all punctuations from the file gsub(/_]/, "", $0) #Start frequency analysis for (i = 1; i <= NF; i++) freq++ } END #Print output... (11 Replies)
Discussion started by: gimley
11 Replies

4. UNIX for Dummies Questions & Answers

How to print line starts with specific word and contains specific word using sed?

Hi, I have gone through may posts and dint find exact solution for my requirement. I have file which consists below data and same file have lot of other data. <MAPPING DESCRIPTION ='' ISVALID ='YES' NAME='m_TASK_UPDATE' OBJECTVERSION ='1'> <MAPPING DESCRIPTION ='' ISVALID ='NO'... (11 Replies)
Discussion started by: tmalik79
11 Replies

5. Shell Programming and Scripting

Help with calculating frequency of specific word in a string

Input file: #read_1 AWEAWQQRZZZQWQQWZ #read_2 ZZAQWRQTWQQQWADSADZZZ #read_3 POGZZZZZZADWRR . . Desired output file: #read_1 3 #read_1 1 #read_2 2 #read_2 3 #read_3 6 . . (3 Replies)
Discussion started by: perl_beginner
3 Replies

6. Shell Programming and Scripting

Fetch entries in front of specific word till next word

Hi all I have following file which I have to edit for research purpose file:///tmp/moz-screenshot.png body, div, table, thead, tbody, tfoot, tr, th, td, p { font-family: &quot;Liberation Sans&quot;; font-size: x-small; } Drug: KRP-104 QD Drug: Placebo Drug: Metformin|Drug:... (15 Replies)
Discussion started by: Priyanka Chopra
15 Replies

7. Shell Programming and Scripting

Convert a list of word/terms into their Regexp representation

Ok this might sound pretty weird but here is the request. Running on a linux system in bash or Perl (i really don't know perl but the end user has a few pearl script already) Start File looks something like this (4000 entries) TEST PLAN T//TF T-TF TEST (T) Hacker ... I am thinking about... (3 Replies)
Discussion started by: oly_r
3 Replies

8. Shell Programming and Scripting

Shell scripting: frequency of specific word in a string and statistics

Hello friends, I need a BIG help from UNIX collective intelligence: I have a CSV file like this: VALUE,TIMESTAMP,TEXT 1,Sun May 05 16:13:05 +0000 2013,"RT @gracecheree: Praying God sends me a really great man one day. Gotta trust in his timing. 0,Sun May 05 16:13:05 +0000 2013,@sendi__... (19 Replies)
Discussion started by: kraterions
19 Replies

9. Shell Programming and Scripting

Count frequency of unique values in specific column

Hi, I have tab-deliminated data similar to the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 I want to count the frequency of each individual "unique" value in the 1st column. Thus, the desired output would be as follows: dot 3 cat 1 hot 1 is... (5 Replies)
Discussion started by: owwow14
5 Replies

10. Shell Programming and Scripting

Search for a specific word and print only the word from the input file

Hi, I have a sample file as shown below, I am looking for sed or any command which prints the complete word only from the input file. Ex: $ cat "sample.log" I am searching for a word which is present in this file We can do a pattern search using grep but I need to cut only the word which... (1 Reply)
Discussion started by: mohan_kumarcs
1 Replies
MALLINFO(3)						     Linux Programmer's Manual						       MALLINFO(3)

NAME
mallinfo - obtain memory allocation information SYNOPSIS
#include <malloc.h> struct mallinfo mallinfo(void); DESCRIPTION
The mallinfo() function returns a copy of a structure containing information about memory allocations performed by malloc(3) and related functions. This structure is defined as follows: struct mallinfo { int arena; /* Non-mmapped space allocated (bytes) */ int ordblks; /* Number of free chunks */ int smblks; /* Number of free fastbin blocks */ int hblks; /* Number of mmapped regions */ int hblkhd; /* Space allocated in mmapped regions (bytes) */ int usmblks; /* Maximum total allocated space (bytes) */ int fsmblks; /* Space in freed fastbin blocks (bytes) */ int uordblks; /* Total allocated space (bytes) */ int fordblks; /* Total free space (bytes) */ int keepcost; /* Top-most, releasable space (bytes) */ }; The fields of the mallinfo structure contain the following information: arena The total amount of memory allocated by means other than mmap(2) (i.e., memory allocated on the heap). This figure includes both in-use blocks and blocks on the free list. ordblks The number of ordinary (i.e., non-fastbin) free blocks. smblks The number of fastbin free blocks (see mallopt(3)). hblks The number of blocks currently allocated using mmap(2). (See the discussion of M_MMAP_THRESHOLD in mallopt(3).) hblkhd The number of bytes in blocks currently allocated using mmap(2). usmblks The "highwater mark" for allocated space--that is, the maximum amount of space that was ever allocated. This field is maintained only in nonthreading environments. fsmblks The total number of bytes in fastbin free blocks. uordblks The total number of bytes used by in-use allocations. fordblks The total number of bytes in free blocks. keepcost The total amount of releasable free space at the top of the heap. This is the maximum number of bytes that could ideally (i.e., ignoring page alignment restrictions, and so on) be released by malloc_trim(3). ATTRIBUTES
For an explanation of the terms used in this section, see attributes(7). +-----------+---------------+------------------------------+ |Interface | Attribute | Value | +-----------+---------------+------------------------------+ |mallinfo() | Thread safety | MT-Unsafe init const:mallopt | +-----------+---------------+------------------------------+ mallinfo() would access some global internal objects. If modify them with non-atomically, may get inconsistent results. The identifier mallopt in const:mallopt mean that mallopt() would modify the global internal objects with atomics, that make sure mallinfo() is safe enough, others modify with non-atomically maybe not. CONFORMING TO
This function is not specified by POSIX or the C standards. A similar function exists on many System V derivatives, and was specified in the SVID. BUGS
Information is returned for only the main memory allocation area. Allocations in other arenas are excluded. See malloc_stats(3) and mal- loc_info(3) for alternatives that include information about other arenas. The fields of the mallinfo structure are typed as int. However, because some internal bookkeeping values may be of type long, the reported values may wrap around zero and thus be inaccurate. EXAMPLE
The program below employs mallinfo() to retrieve memory allocation statistics before and after allocating and freeing some blocks of mem- ory. The statistics are displayed on standard output. The first two command-line arguments specify the number and size of blocks to be allocated with malloc(3). The remaining three arguments specify which of the allocated blocks should be freed with free(3). These three arguments are optional, and specify (in order): the step size to be used in the loop that frees blocks (the default is 1, meaning free all blocks in the range); the ordinal position of the first block to be freed (default 0, meaning the first allocated block); and a number one greater than the ordinal position of the last block to be freed (default is one greater than the maximum block number). If these three arguments are omitted, then the defaults cause all allocated blocks to be freed. In the following example run of the program, 1000 allocations of 100 bytes are performed, and then every second allocated block is freed: $ ./a.out 1000 100 2 ============== Before allocating blocks ============== Total non-mmapped bytes (arena): 0 # of free chunks (ordblks): 1 # of free fastbin blocks (smblks): 0 # of mapped regions (hblks): 0 Bytes in mapped regions (hblkhd): 0 Max. total allocated space (usmblks): 0 Free bytes held in fastbins (fsmblks): 0 Total allocated space (uordblks): 0 Total free space (fordblks): 0 Topmost releasable block (keepcost): 0 ============== After allocating blocks ============== Total non-mmapped bytes (arena): 135168 # of free chunks (ordblks): 1 # of free fastbin blocks (smblks): 0 # of mapped regions (hblks): 0 Bytes in mapped regions (hblkhd): 0 Max. total allocated space (usmblks): 0 Free bytes held in fastbins (fsmblks): 0 Total allocated space (uordblks): 104000 Total free space (fordblks): 31168 Topmost releasable block (keepcost): 31168 ============== After freeing blocks ============== Total non-mmapped bytes (arena): 135168 # of free chunks (ordblks): 501 # of free fastbin blocks (smblks): 0 # of mapped regions (hblks): 0 Bytes in mapped regions (hblkhd): 0 Max. total allocated space (usmblks): 0 Free bytes held in fastbins (fsmblks): 0 Total allocated space (uordblks): 52000 Total free space (fordblks): 83168 Topmost releasable block (keepcost): 31168 Program source #include <malloc.h> #include <stdlib.h> #include <string.h> static void display_mallinfo(void) { struct mallinfo mi; mi = mallinfo(); printf("Total non-mmapped bytes (arena): %d ", mi.arena); printf("# of free chunks (ordblks): %d ", mi.ordblks); printf("# of free fastbin blocks (smblks): %d ", mi.smblks); printf("# of mapped regions (hblks): %d ", mi.hblks); printf("Bytes in mapped regions (hblkhd): %d ", mi.hblkhd); printf("Max. total allocated space (usmblks): %d ", mi.usmblks); printf("Free bytes held in fastbins (fsmblks): %d ", mi.fsmblks); printf("Total allocated space (uordblks): %d ", mi.uordblks); printf("Total free space (fordblks): %d ", mi.fordblks); printf("Topmost releasable block (keepcost): %d ", mi.keepcost); } int main(int argc, char *argv[]) { #define MAX_ALLOCS 2000000 char *alloc[MAX_ALLOCS]; int numBlocks, j, freeBegin, freeEnd, freeStep; size_t blockSize; if (argc < 3 || strcmp(argv[1], "--help") == 0) { fprintf(stderr, "%s num-blocks block-size [free-step " "[start-free [end-free]]] ", argv[0]); exit(EXIT_FAILURE); } numBlocks = atoi(argv[1]); blockSize = atoi(argv[2]); freeStep = (argc > 3) ? atoi(argv[3]) : 1; freeBegin = (argc > 4) ? atoi(argv[4]) : 0; freeEnd = (argc > 5) ? atoi(argv[5]) : numBlocks; printf("============== Before allocating blocks ============== "); display_mallinfo(); for (j = 0; j < numBlocks; j++) { if (numBlocks >= MAX_ALLOCS) { fprintf(stderr, "Too many allocations "); exit(EXIT_FAILURE); } alloc[j] = malloc(blockSize); if (alloc[j] == NULL) { perror("malloc"); exit(EXIT_FAILURE); } } printf(" ============== After allocating blocks ============== "); display_mallinfo(); for (j = freeBegin; j < freeEnd; j += freeStep) free(alloc[j]); printf(" ============== After freeing blocks ============== "); display_mallinfo(); exit(EXIT_SUCCESS); } SEE ALSO
mmap(2), malloc(3), malloc_info(3), malloc_stats(3), malloc_trim(3), mallopt(3) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. Linux 2017-09-15 MALLINFO(3)
All times are GMT -4. The time now is 05:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy