Sponsored Content
Top Forums Shell Programming and Scripting Compute value from more than three consecutive rows Post 302971123 by kathy wang on Friday 15th of April 2016 11:41:30 PM
Old 04-16-2016
Compute value from more than three consecutive rows

Hello all, I am working on a file like below:

Code:
site Date time value1 value2
0023 2014-01-01 00:00 32.0 23.7
0023 2014-01-01 01:00 38.0 29.9
0023 2014-01-01 02:00 85.0 26.6
0023 2014-01-01 03:00 34.0 25.3
0023 2014-01-01 04:00 37.0 23.8
0023 2014-01-01 05:00 80.0 20.3
0023 2014-01-01 06:00 90.0 20.0
0023 2014-01-01 07:00 180.0 20.0
0023 2014-01-01 08:00 30.0 20.0

The first column is site, second column is date (whole year of 2014), and third represent time (from 00:00 to 23:00 for each day), fourth and fifth columns are values. I need to compare column 4 and 5 based on the condition below:

For each site (column 1), if column 4 is more than 3 times of columns 5, and this pattern last for equal or more than 3 hours continually, plus the maximum of them must be higher than 100, print all the lines that meet the standard and count how many cases exist for each site. There are totally around 150 sites and each site has hourly data each day. Here is the output I want:

Code:
0023 2014-01-01 05:00 80.0 20.3 1
0023 2014-01-01 06:00 90.0 20.0 1
0023 2014-01-01 07:00 180.0 20.0 1
0023 2014-06-30 23:00 200.0 30.3 2
0023 2014-07-01 00:00 303.0 30.3 2
0023 2014-07-01 01:00 134.0 30.3 2
0025 2014-07-01 01:00 136.0 25.3 1           
0025 2014-07-01 02:00 116.0 25.3 1
0025 2014-07-01 03:00 106.0 25.3 1

Any help is highly appreciated!

Last edited by kathy wang; 04-18-2016 at 03:53 PM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

compute total from a text file

Hi, I've encountered a problem with a perl and ksh script that totals a certain field in a text file. The computed total of the script is 295540304 but the expected is 297959288, a 2 million difference. The KSH script reads from bottom to top, and the discrepancy started on line 47 (1279th MAN... (1 Reply)
Discussion started by: rsf01
1 Replies

2. UNIX for Advanced & Expert Users

CPU Usage at another Compute Node?

Hi, I am trying to get the cpu usage of a job/process. The machine that has these jobs is a cluster with 1 master node and 10 compute nodes. Now, the complexity is that not all jobs are on the master node. So, in order to get the cpu usage of a job at another node, I have to ssh into it and... (0 Replies)
Discussion started by: davidfrank
0 Replies

3. Shell Programming and Scripting

How to capture 2 consecutive rows when a condition is true ?

Hi All, i have an input below. As long as "x= 1" , i would want to capture 2 lines using sed or awk for eg : 0001 x= 1 $---------------------------------..-.--.. 0001 tt= 137 171 423 1682 2826 0 Pls help. Thanks in advance. Note that the number of lines in each block do... (37 Replies)
Discussion started by: Raynon
37 Replies

4. Shell Programming and Scripting

to compute diskspace

Guys, have any idea for the script like this? also to compute w/ decimal. thanks a=10 b=20 c=30 d=40 if a < b then ( a -b)*1024 = free space b + (c -d) = total space if a > b then (b / d)*1024 = cpu (3 Replies)
Discussion started by: kenshinhimura
3 Replies

5. Shell Programming and Scripting

compute compilation time using script

Hi, I use this script to compute compilation time several time to get system performance and compare different system: #!/bin/sh # measure the different between time before and # after the compilation of benchmark # Start at iteration 1 num=1 while do # Add one to the iteration... (3 Replies)
Discussion started by: zainab
3 Replies

6. Web Development

How to compute previous and next buttons?

I have a project to migrate my club's membership database from Access to web based using MySQL/PHP, but I have a problem I can't get my head around and would appreciate some help... Background... I want to be able to display each member's data on screen and add a Previous and Next button to move... (2 Replies)
Discussion started by: JerryHone
2 Replies

7. Shell Programming and Scripting

remove consecutive duplicate rows

I have some data that looks like, 1 3300665.mol 3300665 5177008 102.093 2 3300665.mol 3300665 5177008 102.093 3 3294015.mol 3294015 5131552 102.114 4 3294015.mol 3294015 5131552 102.114 5 3293734.mol 3293734 5129625 104.152 6 3293734.mol ... (13 Replies)
Discussion started by: LMHmedchem
13 Replies

8. Shell Programming and Scripting

Compute Difference and Edit second, third columns

Hi Friends, My input file is like this chr1 100 200 chr1 300 330 chr1 2000 2000 chr1 5000 5000 chr2 7790 7890 chr2 8000 8000 If the difference of third and second columns is zero, then subtract 500 from second column and add 500 to the third column. So, my output would be chr1... (1 Reply)
Discussion started by: jacobs.smith
1 Replies

9. Shell Programming and Scripting

Compute in milisecond by use of mktime

Hi, I want to calculate diff b/w these starttime and endtime with use of mktime. I need response time in milisecond. I am using mktime to get these times. last three digits are in milisecond Starttime 2013-04-03 08:54:19,989 End time 2013-04-03 08:54:39,389 (9 Replies)
Discussion started by: random_thoughts
9 Replies

10. UNIX for Beginners Questions & Answers

Forward Display from compute node

Hello, sorry if this is an easy thing, but I tried to make it run the whole morning, but it doesn't work so far... I am logged in on a visualization node of a high performance cluster. On this node I can run several gui-based software for the postprocessing of my cfd-data. As the... (1 Reply)
Discussion started by: Shlaeae
1 Replies
DP(8)								     [nmh-1.5]								     DP(8)

NAME
dp - parse dates 822-style SYNOPSIS
/usr/lib/mh/dp [-form formatfile] [-format string] [-width columns] [-version] [-help] dates ... DESCRIPTION
Dp is a program that parses dates according to the ARPA Internet standard. It also understands many non-standard formats, such as those produced by TOPS-20 sites and some UNIX sites using ctime(3). It is useful for seeing how nmh will interpret a date. The dp program treats each argument as a single date, and prints the date out in the official 822-format. Hence, it is usually best to enclose each argument in quotes for the shell. To override the output format used by dp, the -format string or -format file switches are used. This permits individual fields of the address to be extracted with ease. The string is simply a format string and the file is simply a format file. See mh-format(5) for the details. Here is the default format string used by dp: %<(nodate{text})error: %{text}%|%(putstr(pretty{text}))%> which says that if an error was detected, print the error, a `:', and the date in error. Otherwise, output the 822-proper format of the date. FILES
$HOME/.mh_profile The user profile PROFILE COMPONENTS
None SEE ALSO
ap(8), Standard for the Format of ARPA Internet Text Messages (RFC-822) DEFAULTS
`-format' default as described above `-width' default to the width of the terminal CONTEXT
None BUGS
The argument to the -format switch must be interpreted as a single token by the shell that invokes dp. Therefore, one must usually place the argument to this switch inside quotes. MH.6.8 11 June 2012 DP(8)
All times are GMT -4. The time now is 12:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy