Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Fill in missing hours and interpolate values using awk. Post 302984506 by Don Cragun on Wednesday 26th of October 2016 01:55:05 PM
Old 10-26-2016
Quote:
Originally Posted by emirzaei
it is work only for one day, if have two days data, it dos not work!
YYYYMMDDHHmm is date and the time and date range of each field is important.
Code:
40754,35.6931,51.3092,201612312100,24.3
40754,35.6931,51.3092,201612312200,24.7
40754,35.6931,51.3092,201612312300,25.0
40754,35.6931,51.3092,201701010100,23.0
40754,35.6931,51.3092,201701010200,22.0


Moderator's Comments:
Mod Comment
Please wrap code and data input/output in CODE tags.
It make it far easier to read and preserves multiple spaces for indenting or fixed width data.
Hi emirzaei,
RudiC did include this note with his suggestion:
Quote:
Note: it doesn't allow for crossing midnight.
Why don't you try to extend the awk script RudiC supplied to so it can handle missing lines that cross date boundaries

If you get it to work, post your script (in CODE tags) so others reading this thread can learn from what you have done.

If you can't get it to work, show us what you have tried (in CODE tags) and show us what it is doing (in CODE tags) and explain what still needs to be done. We are here to help you learn how to write your own code; not to act as your unpaid programming staff updating suggestions to meet changing requirements after suggestions are posted.

Can we assume that there will never be a string of missing entries that extends for more than 23 hours? (If we can make that assumption, we don't have to perform any arithmetic on dates; we just have to note that the date changed between two entries and the hours on adjacent entries aren't sequential.)

What operating system and shell are you using? (Tools available for performing arithmetic on dates varies considerably from system to system and shell to shell.) Telling us what operating system and shell you're using in the 1st post in a thread is always a good idea. Doing so will lead to fewer suggestions being made that will not work in your environment.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fill in missing numbers in range

I need to edit a list of numbers on the following form: 1 1.0 2 1.4 5 2.1 7 1.9 I want: 1 1.0 2 1.4 3 0.0 4 0.0 5 2.1 6 0.0 7 1.9 (i want to add the missing number in column 1 together with 0.0 in column 2). I guess it is rather trivial but i didn't even manage to read column... (5 Replies)
Discussion started by: bistru
5 Replies

2. Shell Programming and Scripting

fill in missing columns

It can't be that hard, but I just can't figure it out: I have file like: File Sub-brick M_1 S_1 M_2 S_2 M_4 S_4 ... xxx 1 214 731 228 621 132 578 ... and would like to get 0 0 where M_3 S_3 is missing xxx 1 214 731 228 621 0 0 132 578 ... I wrote following script, but can't figure out... (3 Replies)
Discussion started by: avvk
3 Replies

3. Shell Programming and Scripting

Fill the values between -500 to 500 -awk

input -200 2.4 0 2.6 30 2.8 output -500 0 -499 0 -488 0 .......... .......... .... -200 2.4 .... ... 0 2.6 (6 Replies)
Discussion started by: quincyjones
6 Replies

4. Shell Programming and Scripting

Fill missing numbers in second column with zeros

Hi All, I have 100 files with names like this: 1.dat, 2.dat, 3.dat until 100.dat. My dat files look like this: 42323 0 438939 1 434 0 0.9383 3434 120.23 3 234 As you can see in the second column, some numbers are missing. I want to fill those missing places with 0's in all... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

5. Shell Programming and Scripting

Fill missing values with 2

Hi All, I have 100 .txt files which look like this: 3 4 5 6 7 Now, some files have some numbers missing in them and they look like this: 4 5 6 (6 Replies)
Discussion started by: shoaibjameel123
6 Replies

6. Shell Programming and Scripting

Fill in missing Data

hello everyone, I have a task to input missing data into a file. example of my data below: Wed Feb 01 09:00:02 EST 2012,,,0.4,0.3,,0.3,,0.3,,0.5,,0.3,,,0.4,0.3, Wed Feb 01 09:00:11 EST 2012,,,,,,,0.2,,,,,,,,,, Wed Feb 01 09:00:22 EST... (23 Replies)
Discussion started by: Nolph
23 Replies

7. Shell Programming and Scripting

Need help looking for missing hours.

I have a file that should cover a days worth of stats, at the beginning of each 15 minute report I have a unique header that looks like the below example. The "0000" and "0015" will change in the header line to show which 15 Minute interval the report is covering and of course from day to day the... (7 Replies)
Discussion started by: fsanchez
7 Replies

8. Shell Programming and Scripting

Fill in missing rows with zero to have uniform table

Hello, I have two files of same structure except some rows are missing randomly in each file. How do I fill the missing rows to have the exact ID column (S01 ~ S96) and rest columns filled with "0" with awk? The purpose of this step is to join the two files side by side. The closest thread is... (17 Replies)
Discussion started by: yifangt
17 Replies

9. Shell Programming and Scripting

Fill in missing values

Hi, I have a data sample as shown below. I want to fill in the left column so that the line will be continuous. For example, between 1 and 5 should be 2,3,4. And corresponding values in the right column will be 0. Thus the expected data should look like that: 1 1 1 10 1 2 1 3 1 5 1 6 2 0... (6 Replies)
Discussion started by: theanh0508
6 Replies
RRDCGI(1)							      rrdtool								 RRDCGI(1)

NAME
rrdcgi - Create web pages containing RRD graphs based on templates SYNOPSIS
"#!/path/to/"rrdcgi [--filter] DESCRIPTION
rrdcgi is a sort of very limited script interpreter. Its purpose is to run as a cgi-program and parse a web page template containing special <RRD:: tags. rrdcgi will interpret and act according to these tags. In the end it will printout a web page including the necessary CGI headers. rrdcgi parses the contents of the template in 3 steps. In each step it looks only for a subset of tags. This allows nesting of tags. The argument parser uses the same semantics as you are used from your C-shell. --filter Assume that rrdcgi is run as a filter and not as a cgi. Keywords RRD::CV name Inserts the CGI variable of the given name. RRD::CV::QUOTE name Inserts the CGI variable of the given name but quotes it, ready for use as an argument in another RRD:: tag. So even when there are spaces in the value of the CGI variable it will still be considered to be one argument. RRD::CV::PATH name Inserts the CGI variable of the given name, quotes it and makes sure it starts neither with a '/' nor contains '..'. This is to make sure that no problematic pathnames can be introduced through the CGI interface. RRD::GETENV variable Get the value of an environment variable. <RRD::GETENV REMOTE_USER> might give you the name of the remote user given you are using some sort of access control on the directory. RRD::GOODFOR seconds Specify the number of seconds this page should remain valid. This will prompt the rrdcgi to output a Last-Modified, an Expire and if the number of seconds is negative a Refresh header. RRD::INCLUDE filename Include the contents of the specified file into the page returned from the cgi. RRD::SETENV variable value If you want to present your graphs in another time zone than your own, you could use <RRD::SETENV TZ UTC> to make sure everything is presented in Universal Time. Note that the values permitted to TZ depend on your OS. RRD::SETVAR variable value Analog to SETENV but for local variables. RRD::GETVAR variable Analog to GETENV but for local variables. RRD::TIME::LAST rrd-file strftime-format This gets replaced by the last modification time of the selected RRD. The time is strftime-formatted with the string specified in the second argument. RRD::TIME::NOW strftime-format This gets replaced by the current time of day. The time is strftime-formatted with the string specified in the argument. Note that if you return : (colons) from your strftime format you may have to escape them using if the time is to be used as an argument to a GRAPH command. RRD::TIME::STRFTIME START|END start-spec end-spec strftime-format This gets replaced by a strftime-formatted time using the format strftime-format on either start-spec or end-spec depending on whether START or END is specified. Both start-spec and end-spec must be supplied as either could be relative to the other. This is intended to allow pretty titles on graphs with times that are easier for non RRDtool folks to figure out than "-2weeks". Note that again, if you return : (colon) from your strftime format, you may have to escape them using if the time is to be used as an argument to a GRAPH command. RRD::GRAPH rrdgraph arguments This tag creates the RRD graph defined by its argument and then is replaced by an appropriate <IMG ... > tag referring to the graph. The --lazy option in RRD graph can be used to make sure that graphs are only regenerated when they are out of date. The arguments to the RRD::GRAPH tag work as described in the rrdgraph manual page. Use the --lazy option in your RRD::GRAPH tags, to reduce the load on your server. This option makes sure that graphs are only regenerated when the old ones are out of date. If you do not specify your own --imginfo format, the following will be used: <IMG SRC="%s" WIDTH="%lu" HEIGHT="%lu"> Note that %s stands for the filename part of the graph generated, all directories given in the PNG file argument will get dropped. RRD::PRINT number If the preceding RRD::GRAPH tag contained and PRINT arguments, then you can access their output with this tag. The number argument refers to the number of the PRINT argument. This first PRINT has number 0. RRD::INTERNAL <var> This tag gets replaced by an internal var. Currently these vars are known: VERSION, COMPILETIME. These vars represent the compiled-in values. EXAMPLE 1 The example below creates a web pages with a single RRD graph. #!/usr/local/bin/rrdcgi <HTML> <HEAD><TITLE>RRDCGI Demo</TITLE></HEAD> <BODY> <H1>RRDCGI Example Page</H1> <P> <RRD::GRAPH demo.png --lazy --title="Temperatures" DEF:cel=demo.rrd:exhaust:AVERAGE LINE2:cel#00a000:"D. Celsius"> </P> </BODY> </HTML> EXAMPLE 2 This script is slightly more elaborate, it allows you to run it from a form which sets RRD_NAME. RRD_NAME is then used to select which RRD you want to use as source for your graph. #!/usr/local/bin/rrdcgi <HTML> <HEAD><TITLE>RRDCGI Demo</TITLE></HEAD> <BODY> <H1>RRDCGI Example Page for <RRD::CV RRD_NAME></H1> <H2>Selection</H2> <FORM><INPUT NAME=RRD_NAME TYPE=RADIO VALUE=roomA> Room A, <INPUT NAME=RRD_NAME TYPE=RADIO VALUE=roomB> Room B. <INPUT TYPE=SUBMIT></FORM> <H2>Graph</H2> <P> <RRD::GRAPH <RRD::CV::PATH RRD_NAME>.png --lazy --title "Temperatures for "<RRD::CV::QUOTE RRD_NAME> DEF:cel=<RRD::CV::PATH RRD_NAME>.rrd:exhaust:AVERAGE LINE2:cel#00a000:"D. Celsius"> </P> </BODY> </HTML> EXAMPLE 3 This example shows how to handle the case where the RRD, graphs and cgi-bins are separate directories #!/.../bin/rrdcgi <HTML> <HEAD><TITLE>RRDCGI Demo</TITLE></HEAD> <BODY> <H1>RRDCGI test Page</H1> <RRD::GRAPH /.../web/pngs/testhvt.png --imginfo '<IMG SRC=/.../pngs/%s WIDTH=%lu HEIGHT=%lu >' --lazy --start -1d --end now DEF:http_src=/.../rrds/test.rrd:http_src:AVERAGE AREA:http_src#00ff00:http_src > </BODY> </HTML> Note 1: Replace /.../ with the relevant directories Note 2: The SRC=/.../pngs should be paths from the view of the webserver/browser AUTHOR
Tobias Oetiker <tobi@oetiker.ch> 1.4.3 2008-12-22 RRDCGI(1)
All times are GMT -4. The time now is 07:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy