Iterate a min/max awk script over time-series temperature data


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Iterate a min/max awk script over time-series temperature data
# 1  
Old 01-09-2009
Bug Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as returning the min/max for the given day I specify e.g. ($3==1)... that is the third column of data holds the day and I've designated it to read the first day in the code below.

I imagine I need to designate a variable (or array) and a for loop so that it will read each day in the month and return a min/max. When its all done correctly the final output file (01minmax.txt) should have the min/max for each day of the month.

awk -F, ' ($1==101 && $3==1) {print}' 01_weath.dat > foo
awk 'BEGIN {FS=","}
min==""{
min=max=$5
}
{
if ($5 > max){max = $5};
if ($5 < min){min = $5};
}
END {print min,max}
' foo > 01minmax.txt

Any help would be greatly appreciated!
# 2  
Old 01-09-2009
try something like this:
Code:
awk ' BEGIN {for (i=1; i<32; i++)
             {
             	min[i]=999
             	max[i]=-999
             } 
            }
      {
       	if( $5 < min[$3] ) { min[$3]=$5 }
       	if( $5 > max[$3] ) { max[$3]=$5 }
      }
      END {for(i in min) {if( min[i] < 999 ) {print i, min[i], max[i]} }
      }
     '  weather_01.dat  > 01minmax.dat

# 3  
Old 01-09-2009
The next logical q -- do it for 12 months - this is a primitive pass at a next step.
Code:
#!/bin/ksh
mkfile()
{
   awk ' BEGIN {for (i=1; i<32; i++)
             {
             	min[i]=999
             	max[i]=-999
             } 
            }
      {
       	if( $5 < min[$3] ) { min[$3]=$5 }
       	if( $5 > max[$3] ) { max[$3]=$5 }
      }
      END {for(i in min) {if( min[i] < 999 ) {print i, min[i], max[i]} }
      }
     '  "$1"  >  "$2"
     
}

for i in 01 02 03 04 05 06 07 08 09 10 11 12
do
    infile="weather_"$i".dat"
    outfile=$i"minmax.dat
    mkfile $infile $outfile
done

# 4  
Old 01-09-2009
Another approach:

Code:
awk -F, 'NR==1 {min=$5;max=$5;d=$3;next}
d != $3 {print d, min, max;d=$3;max=$5;min=$5}
$5 > max{max=$5}
$5 < min{min=$5}
END{print d, min, max;d=$3}
' 01_weath.dat > 01minmax.txt

Regards
# 5  
Old 01-09-2009
Thanks Jim! I tried the first post you sent and it appears to have calculated the max/min correctly.

I was however unsure why the output (for the first example you posted) was not in 1-31 order see below. There is a pattern to the output but I'm confused why. I have seen a sort script that could remedy this .....or perhaps it can be written in the min/max script?? Thanks again!

22 -7.82 2.929
2 -11.62 .601
23 -2.93 3.462
3 -15.32 -9.37
24 -5.804 -1.434
4 -13.62 .584
25 -8.23 .841
5 -3.418 4.143
26 -5.457 .822
6 1.98 8.57
27 -4.039 -.437
7 4.377 13.87
28 -4.2 4.802
8 7.33 17.04
29 -4.151 5.221
9 6.738 17.38
10 2.6 8.22
11 1.814 8.89
12 1.145 8.66
13 -2.05 7.77
14 -1.218 2.287
15 -2.003 3.015
16 -4.854 1.851
17 -7.27 -1.045
18 -3.29 8.93
19 -3.434 1.765
30 .024 7.98
31 -4.331 2.256
20 -9.03 .285
21 -11.01 -3.436
1 -4.005 .684
# 6  
Old 01-09-2009
Thanks Franklin52! Your code returns the data in 1-31 order.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to count respon time max min avg for nginx logs?

Hi All, need your help, i want count respon time max and average my nginx logs, based on hourly or minutes per api... my nginx.log sample : 10.1.1.1 - - "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 . 10.1.1.1 - - "POST... (4 Replies)
Discussion started by: fajar_3t3
4 Replies

2. Shell Programming and Scripting

Find min and max time taken from a log file

You have a log file as attached in sample input with various operations and time taken by each of them. Write a script to find the min and max time taken for each operation. Sample output is attached. Sample Input is given as below: operation1,83621 operation2,72321 operation3,13288... (1 Reply)
Discussion started by: Chandan_Bose
1 Replies

3. Programming

Php number array from max, min, step size mysql data

I want to create a form with data values in a dropdown list. The values in the dropdown list need to be generated on the fly from max, min and increment values contained in a mysql database. Hopefully this makes sense, I really have no idea where to start :confused: Thanks (6 Replies)
Discussion started by: barrydocks
6 Replies

4. Shell Programming and Scripting

How to get min and max values using awk?

Hi, I need your kind help to get min and max values from file based on value in $5 . File1 SP12.3 stc 2240806 2240808 + ID1_N003 ID2_N003T0 SP12.3 sto 2241682 2241684 + ID1_N003 ID2_N003T0 SP12.3 XE 2239943 2240011 + ID1_N003 ID2_N003T0 SP12.3 XE 2240077 2241254 + ID1_N003 ... (12 Replies)
Discussion started by: redse171
12 Replies

5. Shell Programming and Scripting

awk script to find min and max value

I need to find the max/min of columns 1 and 2 of a 2 column file what contains the special character ">". I know that this will find the max value of column 1. awk 'BEGIN {max = 0} {if ($1>max) max=$1} END {print max}' input.file But what if I needed to ignore special characters in the... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

6. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

7. Shell Programming and Scripting

AWK script - extracting min and max values from selected lines

Hi guys! I'm new to scripting and I need to write a script in awk. Here is example of file on which I'm working ATOM 4688 HG1 PRO A 322 18.080 59.680 137.020 1.00 0.00 ATOM 4689 HG2 PRO A 322 18.850 61.220 137.010 1.00 0.00 ATOM 4690 CD ... (18 Replies)
Discussion started by: grincz
18 Replies

8. Shell Programming and Scripting

Data stream between min and max

Hi, I have a text file containing numbers. There are up to 6 numbers per row and I need to read them, check if they are 0 and if they are not zero check if they are within a given interval (min,max). If they exceed the max or min they should be set to max or min respectively, if they are in the... (4 Replies)
Discussion started by: f_o_555
4 Replies

9. Shell Programming and Scripting

Count time min/max/average for ping

I am redirecting my ping output to a file. The sample output is like this: 64 bytes from xx.xx.xx.167: icmp_seq=4490 ttl=116 3.75 ms 2011Jul12- 15 40 16 64 bytes from xx.xx.xx.167: icmp_seq=4491 ttl=116 5.29 ms 2011Jul12- 15 40 17 64 bytes from xx.xx.xx.167: icmp_seq=4492 ttl=116 4.88 ms... (6 Replies)
Discussion started by: zorrox
6 Replies

10. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies
Login or Register to Ask a Question