Sponsored Content
Top Forums Shell Programming and Scripting awk and sum with original data Post 303025167 by vgersh99 on Thursday 25th of October 2018 12:27:25 PM
Old 10-25-2018
Code:
sort -k4,4 myFile | awk '$4!=p {if(p) print "Total= " tot;p=$4;tot=0} {print;tot+=$NF} END {print "Total= " tot}'

This User Gave Thanks to vgersh99 For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies

2. Shell Programming and Scripting

Identify matching data in a file and output to original line, in perl

Hi, I haven't done this for awhile, and further, I've never done it in perl so I appreciate any help you can give me. I have a file of lines, each with 5 data points that look like this: AB,N,ALLIANCEBERNSTEIN HLDNG L.P,AB,N ALD,N,ALLIED CAPITAL CORPORATION,ALD,N AFC,N,ALLIED CAPITAL... (4 Replies)
Discussion started by: Pcushing
4 Replies

3. UNIX for Advanced & Expert Users

A variable and sum of its value in a huge data.

Hi Experts, I got a question.. In the following output of `ps -elf | grep DataFlow` I get:- 242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0 242001 A mqsiadm... (5 Replies)
Discussion started by: varungupta
5 Replies

4. UNIX for Dummies Questions & Answers

Sum of data in row format

Hi All, I have some numbers in two different files file1 4.21927E+00 4.68257E+00 5.56871E+00 3.59490E+01 7.65806E+01 1.39827E+02 and file2 5.61142E+00 6.21648E+00 7.40152E+00 4.41917E+01 8.31586E+01 1.42938E+02 I would like to get file3 which contains in each column the sum of the... (6 Replies)
Discussion started by: f_o_555
6 Replies

5. Shell Programming and Scripting

Help with calculate total sum of same data problem

Long list of input file: AGDRE1 0.1005449050 AGDRE1 2.1005443435 AGDRE1 1.2005449050 AGDRE1 5.1005487870 AASFV3 50.456304789 AASFV3 2.3659706549 AASFV3 6.3489807860 AASFV3 3.0089890148 RTRTRS 5.6546403546 . . Desired output file: AGDRE1 8.5021829410 AASFV3 62.180245240... (2 Replies)
Discussion started by: perl_beginner
2 Replies

6. Shell Programming and Scripting

AWK print and retain original format

I have a file with very specific column spacing formatting, I wish to do the following: awk '{print $1, $2, $3, $4, $5, $6, $19-$7, $20-$8, $21-$9, $10, $11, $12}' merge.pdb > vector.pdb but the format gets ruined. I have tried with print -f but to no avail.... (7 Replies)
Discussion started by: chrisjorg
7 Replies

7. Shell Programming and Scripting

Help with sum of data set

Input file 2 1159,310, 4 142,199,218,91, 3 91,273,349, Desired output result 2 1469 4 650 3 713 I have long list of input file as shown above. It has a "," delimited to separate between each record in column 2.... (1 Reply)
Discussion started by: perl_beginner
1 Replies

8. Shell Programming and Scripting

Help with sum range of data set together

Input File: 2000 3 1998 2 1997 2 1994 1 1991 1 1989 1 1987 2 1986 2 1985 1 1984 1 . . 10 277256 9 278274 8 282507 7 284837 6 287066 5 292967 (4 Replies)
Discussion started by: perl_beginner
4 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies
CORE(5) 						     Linux Programmer's Manual							   CORE(5)

NAME
core - core dump file DESCRIPTION
The default action of certain signals is to cause a process to terminate and produce a core dump file, a disk file containing an image of the process's memory at the time of termination. This image can be used in a debugger (e.g., gdb(1)) to inspect the state of the program at the time that it terminated. A list of the signals which cause a process to dump core can be found in signal(7). A process can set its soft RLIMIT_CORE resource limit to place an upper limit on the size of the core dump file that will be produced if it receives a "core dump" signal; see getrlimit(2) for details. There are various circumstances in which a core dump file is not produced: * The process does not have permission to write the core file. (By default the core file is called core, and is created in the current working directory. See below for details on naming.) Writing the core file will fail if the directory in which it is to be created is nonwritable, or if a file with the same name exists and is not writable or is not a regular file (e.g., it is a directory or a symbolic link). * A (writable, regular) file with the same name as would be used for the core dump already exists, but there is more than one hard link to that file. * The file system where the core dump file would be created is full; or has run out of inodes; or is mounted read-only; or the user has reached their quota for the file system. * The directory in which the core dump file is to be created does not exist. * The RLIMIT_CORE (core file size) or RLIMIT_FSIZE (file size) resource limits for the process are set to zero; see getrlimit(2) and the documentation of the shell's ulimit command (limit in csh(1)). * The binary being executed by the process does not have read permission enabled. * The process is executing a set-user-ID (set-group-ID) program that is owned by a user (group) other than the real user (group) ID of the process. (However, see the description of the prctl(2) PR_SET_DUMPABLE operation, and the description of the /proc/sys/fs/suid_dumpable file in proc(5).) Naming of core dump files By default, a core dump file is named core, but the /proc/sys/kernel/core_pattern file (since Linux 2.6 and 2.4.21) can be set to define a template that is used to name core dump files. The template can contain % specifiers which are substituted by the following values when a core file is created: %% a single % character %p PID of dumped process %u (numeric) real UID of dumped process %g (numeric) real GID of dumped process %s number of signal causing dump %t time of dump, expressed as seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC) %h hostname (same as nodename returned by uname(2)) %e executable filename (without path prefix) %c core file size soft resource limit of crashing process (since Linux 2.6.24) A single % at the end of the template is dropped from the core filename, as is the combination of a % followed by any character other than those listed above. All other characters in the template become a literal part of the core filename. The template may include '/' charac- ters, which are interpreted as delimiters for directory names. The maximum size of the resulting core filename is 128 bytes (64 bytes in kernels before 2.6.19). The default value in this file is "core". For backward compatibility, if /proc/sys/kernel/core_pattern does not include "%p" and /proc/sys/kernel/core_uses_pid (see below) is nonzero, then .PID will be appended to the core filename. Since version 2.4, Linux has also provided a more primitive method of controlling the name of the core dump file. If the /proc/sys/ker- nel/core_uses_pid file contains the value 0, then a core dump file is simply named core. If this file contains a nonzero value, then the core dump file includes the process ID in a name of the form core.PID. Piping core dumps to a program Since kernel 2.6.19, Linux supports an alternate syntax for the /proc/sys/kernel/core_pattern file. If the first character of this file is a pipe symbol (|), then the remainder of the line is interpreted as a program to be executed. Instead of being written to a disk file, the core dump is given as standard input to the program. Note the following points: * The program must be specified using an absolute pathname (or a pathname relative to the root directory, /), and must immediately follow the '|' character. * The process created to run the program runs as user and group root. * Command-line arguments can be supplied to the program (since kernel 2.6.24), delimited by white space (up to a total line length of 128 bytes). * The command-line arguments can include any of the % specifiers listed above. For example, to pass the PID of the process that is being dumped, specify %p in an argument. Controlling which mappings are written to the core dump Since kernel 2.6.23, the Linux-specific /proc/PID/coredump_filter file can be used to control which memory segments are written to the core dump file in the event that a core dump is performed for the process with the corresponding process ID. The value in the file is a bit mask of memory mapping types (see mmap(2)). If a bit is set in the mask, then memory mappings of the corre- sponding type are dumped; otherwise they are not dumped. The bits in this file have the following meanings: bit 0 Dump anonymous private mappings. bit 1 Dump anonymous shared mappings. bit 2 Dump file-backed private mappings. bit 3 Dump file-backed shared mappings. bit 4 (since Linux 2.6.24) Dump ELF headers. bit 5 (since Linux 2.6.28) Dump private huge pages. bit 6 (since Linux 2.6.28) Dump shared huge pages. By default, the following bits are set: 0, 1, 4 (if the CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS kernel configuration option is enabled), and 5. The value of this file is displayed in hexadecimal. (The default value is thus displayed as 33.) Memory-mapped I/O pages such as frame buffer are never dumped, and virtual DSO pages are always dumped, regardless of the coredump_filter value. A child process created via fork(2) inherits its parent's coredump_filter value; the coredump_filter value is preserved across an execve(2). It can be useful to set coredump_filter in the parent shell before running a program, for example: $ echo 0x7 > /proc/self/coredump_filter $ ./some_program This file is only provided if the kernel was built with the CONFIG_ELF_CORE configuration option. NOTES
The gdb(1) gcore command can be used to obtain a core dump of a running process. If a multithreaded process (or, more precisely, a process that shares its memory with another process by being created with the CLONE_VM flag of clone(2)) dumps core, then the process ID is always appended to the core filename, unless the process ID was already included else- where in the filename via a %p specification in /proc/sys/kernel/core_pattern. (This is primarily useful when employing the LinuxThreads implementation, where each thread of a process has a different PID.) EXAMPLE
The program below can be used to demonstrate the use of the pipe syntax in the /proc/sys/kernel/core_pattern file. The following shell session demonstrates the use of this program (compiled to create an executable named core_pattern_pipe_test): $ cc -o core_pattern_pipe_test core_pattern_pipe_test.c $ su Password: # echo '|$PWD/core_pattern_pipe_test %p UID=%u GID=%g sig=%s' > /proc/sys/kernel/core_pattern # exit $ sleep 100 ^ # type control-backslash Quit (core dumped) $ cat core.info argc=5 argc[0]=</home/mtk/core_pattern_pipe_test> argc[1]=<20575> argc[2]=<UID=1000> argc[3]=<GID=100> argc[4]=<sig=3> Total bytes in core dump: 282624 Program source /* core_pattern_pipe_test.c */ #define _GNU_SOURCE #include <sys/stat.h> #include <fcntl.h> #include <limits.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #define BUF_SIZE 1024 int main(int argc, char *argv[]) { int tot, j; ssize_t nread; char buf[BUF_SIZE]; FILE *fp; char cwd[PATH_MAX]; /* Change our current working directory to that of the crashing process */ snprintf(cwd, PATH_MAX, "/proc/%s/cwd", argv[1]); chdir(cwd); /* Write output to file "core.info" in that directory */ fp = fopen("core.info", "w+"); if (fp == NULL) exit(EXIT_FAILURE); /* Display command-line arguments given to core_pattern pipe program */ fprintf(fp, "argc=%d ", argc); for (j = 0; j < argc; j++) fprintf(fp, "argc[%d]=<%s> ", j, argv[j]); /* Count bytes in standard input (the core dump) */ tot = 0; while ((nread = read(STDIN_FILENO, buf, BUF_SIZE)) > 0) tot += nread; fprintf(fp, "Total bytes in core dump: %d ", tot); exit(EXIT_SUCCESS); } SEE ALSO
bash(1), gdb(1), getrlimit(2), mmap(2), prctl(2), sigaction(2), elf(5), proc(5), pthreads(7), signal(7) COLOPHON
This page is part of release 3.27 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2010-02-27 CORE(5)
All times are GMT -4. The time now is 07:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy