Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Split a file based on number sum at the second column and the third column. Post 303044707 by demo10 on Monday 2nd of March 2020 05:06:12 AM
Old 03-02-2020
Quote:
Originally Posted by nezabudka
Hi
Maybe just like that?
Code:
awk '
/^\S+\s+12109/,/^(\S+\s+){2}14678\s/ {print > "file1"}
/^\S+\s+15573/,/^(\S+\s+){2}15612\s/ {print > "file2"}
/^\S+\s+20498/,/^(\S+\s+){2}21668\s/ {print > "file3"}
' file


Thank you so much. This is what I need.


Quote:
Originally Posted by nezabudka
maybe so?
Code:
#!/bin/bash

step=2999
declare -i start=12109 end=start+step count=1
stop=$(awk '{if($3>max) max=$3} END {print max}' file)

while [ $end -le $stop ]; do
        awk -vA=$start -vZ=$end -vf="file$count" '
                $2>=A && $3<=Z {print > f}
        ' file
        start+=step
        end+=step
        count+=1
done

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How do I sum one column based on another column?

Hi, I am new to this forum and new to awk. I have a file that contains 2 columns. Heres an example of what it looks like: 10 + 20 + 40 + 50 - 70 - So the file is tab-delimited. What I want to do is add 10 to column 1 whenever column 2 is + and substract 10 from column 1... (1 Reply)
Discussion started by: phil_heath
1 Replies

2. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

3. Shell Programming and Scripting

Split the file based on column

Hi, I have a file sample_1.txt (300k rows) which has data like below: * Also each record is around 64k bytes 11|1|abc|102553|125589|64k bytes of data 10|2|def|123452|123356|...... 13|2|geh|144351|121123|... 25|4|fgh|165250|118890|.. 14|1|abc|186149|116657|......... (6 Replies)
Discussion started by: sol_nov
6 Replies

4. Shell Programming and Scripting

Sum Of Column Based On Column Condition

I have a following inputfile MT,AP,CDM,TTML,MUM,GS,SUCC,3 MT,AP,CDM,TTSL,AP,GS,FAIL,9 MT,AP,CDM,RCom,MAH,GS,SUCC,3 MT,AP,CDM,RTL,HP,GS,SUCC,1 MT,AP,CDM,Uni,UPE,GS,SUCC,2 MT,AP,CDM,Uni,MUM,GS,SUCC,2 TTSL,AP,GS,MT,MAH,CDM,SUCC,20 TTML,AP,GS,MT,MAH,CDM,FAIL,10... (2 Replies)
Discussion started by: siramitsharma
2 Replies

5. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

6. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

7. Shell Programming and Scripting

Split column data if the table has n number of column's

please write a shell script Table -------------------------- 1 2 3 a b c 3 4 5 c d e 7 8 9 f g h Output should be like this --------------- 1 2 3 3 4 5 7 8 9 a b c c d e f g h (1 Reply)
Discussion started by: Priti2277
1 Replies

8. Shell Programming and Scripting

Split column data if the table has n number of column's with some record

Split column data if the table has n number of column's with some record then how to split n number of colmn's line by line with records Table --------- Col1 col2 col3 col4 ....................col20 1 2 3 4 .................... 20 a b c d .................... v ... (11 Replies)
Discussion started by: Priti2277
11 Replies

9. Shell Programming and Scripting

Sum of a column as new column based on header in a script

Hello, I am trying to store sum of a column as a new column inside a file but have to find the column names dynamically I/p c1,c2,c3,c4,c5 10,20,30,40,50 20,30,40,50,60 If i want to find sum only column c1, c3 and output it as c6,c7 O/p c1,c2,c3,c4,c5,c6,c7 10,20,30,40,50,30,70... (6 Replies)
Discussion started by: mkathi
6 Replies

10. UNIX for Beginners Questions & Answers

Sum in file based column

Hi All, I have a file as below and want to sum based on the id in the first column Input 10264;ATE; 12 10265;SES;11 10266AUT;50 10264;ATE;10 10265;SES;13 10266AUT;89 10264;ATE;1 10265;SES;15 10266AUT;78 Output 10264;ATE; 23 10265;SES;39 10266AUT;139 (6 Replies)
Discussion started by: arunkumar_mca
6 Replies
RRDGRAPH_DATA(1)						      rrdtool							  RRDGRAPH_DATA(1)

NAME
rrdgraph_data - preparing data for graphing in rrdtool graph SYNOPSIS
DEF:<vname>=<rrdfile>:<ds-name>:<CF>[:step=<step>][:start=<time>][:end=<time>][:reduce=<CF>] VDEF:vname=RPN expression CDEF:vname=RPN expression DESCRIPTION
These three instructions extract data values out of the RRD files, optionally altering them (think, for example, of a bytes to bits conversion). If so desired, you can also define variables containing useful information such as maximum, minimum etcetera. Two of the instructions use a language called RPN which is described in its own manual page. Variable names (vname) must be made up strings of the following characters "A-Z, a-z, 0-9, -,_" and a maximum length of 255 characters. When picking variable names, make sure you do not choose a name that is already taken by an RPN operator. A safe bet it to use lowercase or mixed case names for variables since operators will always be in uppercase. DEF
DEF:<vname>=<rrdfile>:<ds-name>:<CF>[:step=<step>][:start=<time>][:end=<time>][:reduce=<CF>] This command fetches data from an RRD file. The virtual name vname can then be used throughout the rest of the script. By default, an RRA which contains the correct consolidated data at an appropriate resolution will be chosen. The resolution can be overridden with the --step option. The resolution can again be overridden by specifying the step size. The time span of this data is the same as for the graph by default, you can override this by specifying start and end. Remember to escape colons in the time specification! If the resolution of the data is higher than the resolution of the graph, the data will be further consolidated. This may result in a graph that spans slightly more time than requested. Ideally each point in the graph should correspond with one CDP from an RRA. For instance, if your RRD has an RRA with a resolution of 1800 seconds per CDP, you should create an image with width 400 and time span 400*1800 seconds (use appropriate start and end times, such as "--start end-8days8hours"). If consolidation needs to be done, the CF of the RRA specified in the DEF itself will be used to reduce the data density. This behavior can be changed using ":reduce=<CF>". This optional parameter specifies the CF to use during the data reduction phase. Example: DEF:ds0=router.rrd:ds0:AVERAGE DEF:ds0weekly=router.rrd:ds0:AVERAGE:step=7200 DEF:ds0weekly=router.rrd:ds0:AVERAGE:start=end-1h DEF:ds0weekly=router.rrd:ds0:AVERAGE:start=11:00:end=start+1h VDEF
VDEF:vname=RPN expression This command returns a value and/or a time according to the RPN statements used. The resulting vname will, depending on the functions used, have a value and a time component. When you use this vname in another RPN expression, you are effectively inserting its value just as if you had put a number at that place. The variable can also be used in the various graph and print elements. Example: "VDEF:avg=mydata,AVERAGE" Note that currently only aggregation functions work in VDEF rpn expressions. Patches to change this are welcome. CDEF
CDEF:vname=RPN expression This command creates a new set of data points (in memory only, not in the RRD file) out of one or more other data series. The RPN instructions are used to evaluate a mathematical function on each data point. The resulting vname can then be used further on in the script, just as if it were generated by a DEF instruction. Example: "CDEF:mydatabits=mydata,8,*" About CDEF versus VDEF At some point in processing, RRDtool has gathered an array of rates ready to display. CDEF works on such an array. For example, CDEF:new=ds0,8,* would multiply each of the array members by eight (probably transforming bytes into bits). The result is an array containing the new values. VDEF also works on such an array but in a different way. For example, VDEF:max=ds0,MAXIMUM would scan each of the array members and store the maximum value. When do you use VDEF versus CDEF? Use CDEF to transform your data prior to graphing. In the above example, we'd use a CDEF to transform bytes to bits before graphing the bits. You use a VDEF if you want max(1,5,3,2,4) to return five which would be displayed in the graph's legend (to answer, what was the maximum value during the graph period). If you want to apply 'complex' operations to the result of a VDEF you have to use a CDEF again since VDEFs only look like RPN expressions, they aren't really. SEE ALSO
rrdgraph gives an overview of how rrdtool graph works. rrdgraph_data describes DEF,CDEF and VDEF in detail. rrdgraph_rpn describes the RPN language used in the ?DEF statements. rrdgraph_graph page describes all of the graph and print functions. Make sure to read rrdgraph_examples for tips&tricks. AUTHOR
Program by Tobias Oetiker <tobi@oetiker.ch> This manual page by Alex van den Bogaerdt <alex@vandenbogaerdt.nl> with corrections and/or additions by several people 1.4.7 2009-10-14 RRDGRAPH_DATA(1)
All times are GMT -4. The time now is 06:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy