awk with paste ... columnwise


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk with paste ... columnwise
# 1  
Old 02-05-2008
awk with paste ... columnwise

Hi,

I'm trying to plot some data using the awk to find and parse the data and then use gnuplot to plot it up. I'd like to plot one or more range cells (let the user decide!). I've been able to write up the code such that I can plot one range cell per plot, but I just can't see how to get more onto one plot. Any ideas?

Ultimately I'd like the function to be able not only wave height but period and direction as well. Calling from the command line, something like: "plotwaves <SITE> <RANGE> <TYPE(height,period,direction, or all)> <STARTtime> <STOPtime>"

Thanks for your advice!

The data is of the following format:

%CTF: 1.00
%FileType: WVMD WVM5 "Wave History"
%Manufacturer: CODAR Ocean Sensors. SeaSonde
%Site: MONT ""
%TimeStamp: 2008 02 01 00 00 00
%TimeZone: "GMT" +0.000 0
%Origin: 37.5337167 -122.5192167
%%
%%
%% Note: Parameters for Last processed item...
%TimeCoverage: 0.250 hours
%RangeResolutionKMeters: 2.9257 km
%%
%Distance: 2.926 km
%RangeCell: 1
%TableType: WAVL WVM5
%TableColumns: 14
%TableColumnTypes: TIME MWHT MWPD WAVB WNDB ACNT DIST RCLL TYRS TMON TDAY THRS TMIN TSEC
%TableRows: 727
%TableStart:
%% Time --------Wave-------- Wind
%% FromStart Height Period Dir. Dir. Spectra Distance Range Time
%% (seconds) (m) (s) (deg) (deg) Count (km) Cell Year Mo Dy Hr Mn S
0 1.7 7.4 255.4 170.6 7 2.926 1 2008 02 01 00 00 00
600 1.9 7.5 260.0 169.2 7 2.926 1 2008 02 01 00 10 00
1200 1.9 7.5 257.9 182.7 7 2.926 1 2008 02 01 00 20 00
1800 1.7 7.7 252.2 198.2 7 2.926 1 2008 02 01 00 30 00
2400 1.5 7.7 253.7 212.9 7 2.926 1 2008 02 01 00 40 00
3000 1.9 7.7 249.2 221.6 7 2.926 1 2008 02 01 00 50 00
3600 1.9 7.6 243.0 236.4 7 2.926 1 2008 02 01 01 00 00
4200 2.0 7.6 240.9 229.9 7 2.926 1 2008 02 01 01 10 00
4800 2.0 7.6 243.7 231.2 7 2.926 1 2008 02 01 01 20 00
5400 2.0 7.7 249.5 229.3 7 2.926 1 2008 02 01 01 30 00
6000 2.0 7.7 253.0 231.1 7 2.926 1 2008 02 01 01 40 00
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
%TableEnd:
%%
%Distance: 5.851 km
%RangeCell: 2
%TableType: WAVL WVM5
%TableColumns: 14
%TableColumnTypes: TIME MWHT MWPD WAVB WNDB ACNT DIST RCLL TYRS TMON TDAY THRS TMIN TSEC
%TableRows: 727
%TableStart: 2
%% Time --------Wave-------- Wind
%% FromStart Height Period Dir. Dir. Spectra Distance Range Time
%% (seconds) (m) (s) (deg) (deg) Count (km) Cell Year Mo Dy Hr Mn S
0 2.1 7.4 254.8 184.8 7 5.851 2 2008 02 01 00 00 00
600 2.1 7.4 249.4 184.8 7 5.851 2 2008 02 01 00 10 00
1200 1.9 7.4 254.5 196.9 7 5.851 2 2008 02 01 00 20 00
1800 1.8 7.4 248.8 211.7 7 5.851 2 2008 02 01 00 30 00
2400 1.8 7.4 251.4 227.4 7 5.851 2 2008 02 01 00 40 00
3000 1.7 7.4 247.0 236.6 7 5.851 2 2008 02 01 00 50 00
3600 1.8 7.3 243.9 246.7 7 5.851 2 2008 02 01 01 00 00
4200 1.7 7.3 247.4 246.7 7 5.851 2 2008 02 01 01 10 00
4800 1.7 7.3 247.4 253.7 7 5.851 2 2008 02 01 01 20 00
5400 1.8 7.3 238.2 258.0 7 5.851 2 2008 02 01 01 30 00
6000 1.8 7.3 243.3 259.5 7 5.851 2 2008 02 01 01 40 00
6600 1.8 7.4 248.0 258.0 7 5.851 2 2008 02 01 01 50 00
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
%TableEnd:
%%
%Distance: N km
%RangeCell: N
%TableType: WAVL WVM5
%TableColumns: 14
%TableColumnTypes: TIME MWHT MWPD WAVB WNDB ACNT DIST RCLL TYRS TMON TDAY THRS TMIN TSEC
%TableRows: 727
%TableStart: N
%% Time --------Wave-------- Wind
%% FromStart Height Period Dir. Dir. Spectra Distance Range Time
%% (seconds) (m) (s) (deg) (deg) Count (km) Cell Year Mo Dy Hr Mn S
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .
. . . . . . . . N . . . . .


OK, so that's the data and the code that I've got thus far, which plots one range cell, is:

Code:
#!/bin/bash
#
#print out wvlm from given time and site

site=$1
rnge=$2
daysback=$3
EsecStt=$[`date -u +%s` - (3600*24*$daysback)]
EsecStp=$[`date -u +%s`]
outputfile="/Volumes/Extras/RealTime/Plots/Waves/`echo $site`/wvlm_rng$rnge.ps"
cat `find /Volumes/Extras/RealTime/Data/Waves -type f -iname "wvlm_$site*"` | gawk -f /Users/dpath2o/cocmp/progs/trunk/dpath2o/awk/tmp.awk range=$rnge startT=$EsecStt stopT=$EsecStp
gnuplot <<EOF
set terminal postscript eps enhanced color 
set size 2,2
set xdata time
set timefmt "%s"
#set xrange [$EsecStt:$EsecStp]
set output "$outputfile"
set yrange [0:20]
set timestamp "Last updated: %d/%m/%Y, %H:%M" top
set xlabel "Date (mm/dd)" font "Helvetica,20" tc lt 4
set format x "%m/%d" 
set ylabel "meters" font "Helvetica,20" tc lt 4
set title "Wave Heights from $site and range $rnge" font "Helvetica,30" tc lt 1
plot "/Volumes/Extras/RealTime/Data/Waves/tmpWVLM" using 1:2 notitle with impulses
EOF

and the awk script
Code:
#!/opt/bin/local/gawk
{
  stt = sprintf("%04d %02d %02d %02d 00 00",$9,$10,$11,$12)
    testtime = mktime(stt)
    if ((testtime > startT ) && (testtime < stopT) && ($2 <= 20) && ($8 == range)) {
      printf("%i %f\n",testtime,$2) > "/Volumes/Extras/RealTime/Data/Waves/.tmpwvlm"
    }
}


Last edited by dpath2o; 02-06-2008 at 01:11 PM..
# 2  
Old 02-06-2008
condensed question

So the condensed version of what I'm seeking help on is ...

Suppose I have a file

#time(epoch) data1 data2 ... dataN
10000000 9.5 13.4 ... 1
10000001 9.3 13.3 ... 1
10000002 9.0 13.4 ... 1
10000003 9.1 13.2 ... 1
.
.
.
.
10000000 9.9 13.5 ... 2
10000001 999 13.6 ... 2
10000002 9.9 13.6 ... 2
.
.
.
.
10000000 ? ? ... N


And what I want to do is create a file that looks like

#time(epoch) data2 data2 ... dataN
10000000 9.5 9.9 ... ?
10000001 9.3 NAN ... ?
10000002 9.0 9.9 ... ?
.
.
.


So basically I'm looking to concatenate a file by a certain column AND filter that column such that values exceeding, let's say 20, will be given a NAN or blank, whichever gnuplot can handle.

I hope this a little more clear in terms of what I'm seeking help for as well as is a little more inline with the title ...

THANKS
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to create a report using paste or with awk

Input file will be Name: serve1 has disk :Yes dev (8): Name: serve2 has disk :No dev (8): Name: serve3 has disk :No Name: serve4 has disk :Yes dev (8): Need output like that. I was using pate -d, - - - . But that need all the line in same format in this some server it has... (4 Replies)
Discussion started by: ranjancom2000
4 Replies

2. Shell Programming and Scripting

Use sed/awk to do like copy and paste

I have rrd file which is have the gaps and I want to fill it out with some value , I've got 10 NaN record and I try to populate data from 10 records be for NaN to change instead of NaN :( <!-- 2016-05-19 14:10:00 CST / 1463638200 -->... (11 Replies)
Discussion started by: boobytrap
11 Replies

3. Shell Programming and Scripting

awk columnwise adjacent match checking program

Hi, For the below problem, awk script is needed.. input file : jill jack jill jill jack jill jack jill Required: here, i need to check from first line. reference is jack. jill need to verified with next line jack. if found print jill & jack combination hill1 jack & jill &... (0 Replies)
Discussion started by: vasanth.vadalur
0 Replies

4. Shell Programming and Scripting

copy/paste with awk

Hi everybody, I have two XML files. I am working on a script that could copy and paste the contents of the first xml file to the desired location in the second xml file. Here is my first XML file. This is the second XML file. Finaly, I wnat to obtain something like that : ... (2 Replies)
Discussion started by: lsaas
2 Replies

5. Shell Programming and Scripting

cut and paste using awk

Hi i need a favour i have a file which has some trillions of records. The file is like this 11111000000000192831840914000000000000000000000000000 45789899090000000000000000011111111111111111111111111 I want to cut specific postions in each line like cut1-3 and assisgn it to a variable and... (5 Replies)
Discussion started by: richa2.m
5 Replies

6. Shell Programming and Scripting

awk/grep copy and paste and insert in between lines.

Hi all, I'm a unix newb andI'm trying to write a script that can copy some text paste it in a certian place and then add a number. It's not really clear but I'll show an example. what the file looks like right now: Linux 2.6.24-24-generic (abc) 07/15/09 23:25:01 CPU ... (6 Replies)
Discussion started by: the1hand3r
6 Replies

7. Shell Programming and Scripting

appending several columns with awk and paste

Hello, I am trying to solve for a couple of hours now the following problem: I have n files and would like to add the third column of each file to a new file: temp1.txt 1 2 3 1 2 3 1 2 3 temp2.txt 1 2 4 1 2 4 1 2 4 1 2 4 temp3.txt (2 Replies)
Discussion started by: creamcheese
2 Replies

8. UNIX for Dummies Questions & Answers

cut and paste columns using awk

Hi, Let's say that I have a file called table, I know that if I need to see a the second column for exampls I use: awk ' {print $2}' table.txt Is there anyway to use awk to actually cut a column and put it somewhere else in the table?:confused: (8 Replies)
Discussion started by: cosmologist
8 Replies

9. Shell Programming and Scripting

how to replace paste utility with sed/awk

hi I have two file that I would like to paste line by line. I use to use paste for this. But my current platform will not have paste utility. Could anyone please suggest how I it can be done using sed,awk or even simple bourne shell scripts? Thanks Sabina (1 Reply)
Discussion started by: ssayeed
1 Replies

10. UNIX for Dummies Questions & Answers

awk or sed or paste

i have a file and i need the text to line up currently the file looks like so job scheduled complete 12 12:00 wendsday 13 1:00 wednsday its a text file but not sure how to manipulate the file for it to line up (3 Replies)
Discussion started by: leprichaun
3 Replies
Login or Register to Ask a Question

Featured Tech Videos