Sponsored Content
Top Forums Shell Programming and Scripting Performance of calculating total number of matching records in multiple files Post 302951974 by neutronscott on Wednesday 12th of August 2015 10:07:17 AM
Old 08-12-2015
Your suggested method would start a new nawk on every file and probably be very slow.

Using + terminator instead of ; with find's -exec groups many files together. It's still possible that it ends up being several invocations of nawk because of argument limits. Then we need to somehow keep track of the sum and pass the value around. One solution is to have find just cat all the files and pipe it to nawk.

Code:
find . -type f -name 'cdr_201507*txt' -exec cat {} + | nawk -F, '
  $1=="charging" && $9=="9647512971064" && $29~/<faultString>[Ss]ending [Ee]xception<\/faultString>/ {c++}
  END{print c}'

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep running total/ final total across multiple files

Ok, another fun hiccup in my UNIX learning curve. I am trying to count the number of occurrences of an IP address across multiple files named example.hits. I can extract the number of occurrences from the files individually but when you use grep -c with multiple files you get the output similar to... (5 Replies)
Discussion started by: MrAd
5 Replies

2. Shell Programming and Scripting

Calculate total space, total used space and total free space in filesystem names matching keyword

Good afternoon! Im new at scripting and Im trying to write a script to calculate total space, total used space and total free space in filesystem names matching a keyword (in this one we will use keyword virginia). Please dont be mean or harsh, like I said Im new and trying my best. Scripting... (4 Replies)
Discussion started by: bigben1220
4 Replies

3. Shell Programming and Scripting

Creating a file with matching records from two other files

Hi All, I have 2 files (file1 & file2). File1 and File2 have m and n columns respectively I have to compare value in column1 of file1 with file2 and find line(s) from file2 matching column1 value. The value can be in any column in the matching lines of file2. The output should be... (10 Replies)
Discussion started by: Swagi
10 Replies

4. Shell Programming and Scripting

Calculating number of records by field

Hi, I have CSV file which looks like below, i want to calulate number of records for each brand say SOLO_UNBEATABLE E and SOLO_UNBEATABLE F combined and record count is say 20 . i want to calculate for each brand, and here only first record will have all data and rest of record for the brand... (2 Replies)
Discussion started by: raghavendra.cse
2 Replies

5. Shell Programming and Scripting

Nawk, creating a variable total from multiple lines(records)

Good Morning/Afternoon All, I am having some trouble creating a variable called "total" to display the sum of the values in a specific field, $6 for example. The data I am working on is in the following form: John Doe:(555) 555-5555:1:2:3 Jane Doe:(544) 444-5556:4:5:6 Moe Doe:(654)... (2 Replies)
Discussion started by: SEinT
2 Replies

6. UNIX for Dummies Questions & Answers

Write the total number of rows in multiple files into another file

Hello Friends, I know you all are busy and inteligent too... I am stuck with one small issue if you can help me then it will be really great. My problem is I am having some files i.e. Input.txt1 Input.txt2 Input.txt3 Now my task is I need to check the total number of rows in... (4 Replies)
Discussion started by: malaya kumar
4 Replies

7. Emergency UNIX and Linux Support

Calculating total space in GB for all files with typical pattern

Hi Experts, In a particular dir, I have many files *AJAY*. How can I get total size of all such files. I tried du -hs *AJAY* but it gave me individual size of all files. All I require is summation of all. Thanks, Ajay (4 Replies)
Discussion started by: ajaypatil_am
4 Replies

8. HP-UX

Total number of files in a FS

Hello people, On HP-UX B.11.11 U 9000/800 How can I have in aprox. the total number of files in a specific FS? Is the number of used inodes a rough estimation of my total number of files? Server1 /Data:df -i . /Data (/dev/vg_Data/lvol1 ) : 18292960 total i-nodes 15800945 free... (3 Replies)
Discussion started by: drbiloukos
3 Replies

9. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

10. UNIX for Beginners Questions & Answers

Matching fields between two files, repeated records

In two previous posts (here) and (here), I received help from forum members comparing multiple fields across two files and selectively printing portions of each as output based upon would-be matches using awk. I had been fairly comfortable populating awk arrays with fields and using awk's special... (3 Replies)
Discussion started by: jvoot
3 Replies
acctcms(1M)						  System Administration Commands					       acctcms(1M)

NAME
acctcms - command summary from process accounting records SYNOPSIS
/usr/lib/acct/acctcms [ -a [-o] [-p]] [-c] [-j] [-n] [-s] [-t] filename... DESCRIPTION
acctcms reads one or more filenames, normally in the form described in acct.h(3HEAD). It adds all records for processes that executed iden- tically named commands, sorts them, and writes them to the standard output, normally using an internal summary format. OPTIONS
-a Print output in ASCII rather than in the internal summary format. The output includes command name, number of times executed, total kcore-minutes, total CPU minutes, total real minutes, mean size (in K), mean CPU minutes per invocation, "hog factor," char- acters transferred, and blocks read and written, as in acctcom(1). Output is normally sorted by total kcore-minutes. Use the following options only with the -a option: -o Output a (non-prime) offshift-time-only command summary. -p Output a prime-time-only command summary. When -o and -p are used together, a combination prime-time and non-prime-time report is produced. All the output summaries are total usage except number of times executed, CPU minutes, and real minutes, which are split into prime and non-prime. -c Sort by total CPU time, rather than total kcore-minutes. -j Combine all commands invoked only once under "***other". -n Sort by number of command invocations. -s Any file names encountered hereafter are already in internal summary format. -t Process all records as total accounting records. The default internal summary format splits each field into prime and non-prime- time parts. This option combines the prime and non-prime time parts into a single field that is the total of both, and provides upward compatibility with old style acctcms internal summary format records. EXAMPLES
Example 1: Using the acctcms command. A typical sequence for performing daily command accounting and for maintaining a running total is: example% acctcms filename ... > today example% cp total previoustotal example% acctcms -s today previoustotal > total example% acctcms -a -s today ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWaccu | +-----------------------------+-----------------------------+ SEE ALSO
acctcom(1), acct(1M), acctcon(1M), acctmerg(1M), acctprc(1M), acctsh(1M), fwtmp(1M), runacct(1M), acct(2), acct.h(3HEAD), utmpx(4), attributes(5) NOTES
Unpredictable output results if -t is used on new style internal summary format files, or if it is not used with old style internal summary format files. SunOS 5.10 22 Feb 1999 acctcms(1M)
All times are GMT -4. The time now is 12:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy