Sponsored Content
Top Forums Programming Resample time series data with replacement any way to do this in awk or just bash script Post 302893579 by SriniShoo on Thursday 20th of March 2014 04:01:18 AM
Old 03-20-2014
Below are the 2 solutions with slight difference. You can use as per your requirement
Code:
awk 'BEGIN{n = 5}
  {if((($1 * 100) % n) == 0) {printf "%7.5f   %-6.4f\n", $1, $4; p1 = 0}
  else if((($1 + p1) * 100) % 5 == 0)
    {printf "%7.5f   %-6.4f\n", ($1 + p1)/2, ($4 + p4)/2; p1 = 0}
  else {p1 = $1; p4 = $4}}' file

and
Code:
awk 'BEGIN{n = 5}
  {if(($1 * 100) == (n * x)) {printf "%7.5f   %-6.4f\n", $1, $4; p1 = 0; x++}
  else if((($1 + p1) * 50) == (n * x))
    {printf "%7.5f   %-6.4f\n", ($1 + p1)/2, ($4 + p4)/2; p1 = 0; x++}
  else {p1 = $1; p4 = $4}}' file

Below is the output I got with both the codes
Code:
0.00000   5.5150
0.05000   5.3750
0.10000   5.3000
0.15000   5.2075
0.20000   4.7100
0.25000   5.2800
0.30000   5.1475
0.35000   5.0550
0.40000   4.9675


Last edited by SriniShoo; 03-20-2014 at 05:02 AM.. Reason: provided output
This User Gave Thanks to SriniShoo For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

perl as awk replacement in a script.

Hey all, Im trying to write a script on windows, which Im not too familiar with. Im generally a bash scripting guy but am using perl for this case. My question is... I have this exact output: 2 Dir(s) 6,380,429,312 bytes free and I just need to get the number out... (4 Replies)
Discussion started by: trey85stang
4 Replies

2. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

3. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

4. UNIX for Dummies Questions & Answers

Simple awk script for positional replacement in text?

I have a string of letters. (They happen to be DNA, not that it's relevant to the question.) For analysis purposes, I need to replace the information at some of the sites. I need to do this based on their position, not the information in that position. I also need to ignore differences at other... (10 Replies)
Discussion started by: JFS
10 Replies

5. Shell Programming and Scripting

Subtracting time with awk - BASH/Debian GNU Linux

I'm sure this is simple and I've been looking at examples for days on end but can't seem to come to grips with awk. What I have: mplayer -v dvd:// -identify -vo null -ao null -nolirc -nojoystick -frames 0 2>/dev/null >> /tmp/MplayerOut ChapterStart=($(grep CHAPTERS: /tmp/MplayerOut |sed... (3 Replies)
Discussion started by: rickenbacherus
3 Replies

6. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

7. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

9. UNIX for Beginners Questions & Answers

Awk/bash one liner replacement for a if condition

Hi. I wrote this small bash script, i want to compare second column from file1 with file2 if a pattern matches. Files are small and I am sure that pattern occurs only once. I think this can be rewritten into a awk one liner. Appreciate if someone could give me idea. Whole NR FNR confuse me :o ... (6 Replies)
Discussion started by: ctrld
6 Replies
FILTER1D(l)															       FILTER1D(l)

NAME
filter1d - Time domain filtering of 1-D time series SYNOPSIS
filter1d [ infile ] -F<type><width> [ -Dincrement ] [ -E ] [ -H[nrec] ] [ -Iignore_val ] [ -Llack_width ] [ -Nn_cols/t_col ] [ -Qq_factor ] [ -Ssymmetry_factor ] [ -Tstart/stop/int ] [ -V ] [ -bi[s][n] ] [ -bo[s][n] ] DESCRIPTION
filter1d is a general time domain filter for multiple column time series data. The user specifies the number of columns of input and which column is the time. (See -N option below). The fastest operation occurs when the input time series are equally spaced and have no gaps or outliers and the special options are not needed. filter1d has options L, Q, and S for unevenly sampled data with gaps. infile Multi-column ASCII (or binary, see -b) file holding data values to be filtered. -F sets Filtertype, type is one of b(oxcar), c(osine arch), g(aussian), m(edian), or p(maximum likelihood Probability estimator -- a mode estimator), and specify full filter width in same units as time column, OR, use -Ffname to give the name of a one-column file of your own coefficients. Upper case type B, C, G, M, P, F will use robust filter versions: i.e., replace outliers (2.5 L1 scale off median) with median during filtering. OPTIONS
-D increment is used when series is NOT equidistantly sampled. Then increment will be the abscissae resolution, i.e., all abscissae will be rounded off to a multiple of increment. Alternatively, resample data with sample1d. -E Include Ends of time series in output. Default loses half the filter-width of data at each end. -H Input file(s) has Header record(s). Number of header records can be changed by editing your .gmtdefaults file. If used, GMT default is 1 header record. -I To ignore values; If an input value equals ignore_val it will be set to NaN. -L Checks for Lack of data condition. If input data has a gap exceeding width then no output will be given at that point [Default does not check Lack]. -N Sets number of columns in input and which column contains the independent variable (time). The left-most column is # 0, the right- most is # (n_cols - 1). [Default is n_cols = 2, t_col = 0; i.e., file has t, f(t) pairs]. -Q assess Quality of output value by checking mean weight in convolution. Enter q_factor between 0 and 1. If mean weight < q_factor, output is suppressed at this point [Default does not check Quality]. -S Checks symmetry of data about window center. Enter a factor between 0 and 1. If ( (abs(n_left - n_right)) / (n_left + n_right) ) > factor, then no output will be given at this point [Default does not check Symmetry]. -T Make evenly spaced timesteps from start to stop by int [Default uses input times]. -V Selects verbose mode, which will send progress reports to stderr [Default runs "silently"]. -bi Selects binary input. Append s for single precision [Default is double]. Append n for the number of columns in the binary file(s). -bo Selects binary output. Append s for single precision [Default is double]. EXAMPLES
To filter the data set in the file cruise.gmtd containing evenly spaced gravity, magnetics, topography, and distance (in m) with a 10 km Gaussian filter, removing outliers, and output a filtered value every 2 km between 0 and 100 km, try filter1d cruise.gmtd -T0/1.0e5/2000 -FG10000 -N4/3 -V > filtered_cruise.gmtd Data along track often have uneven sampling and gaps which we do not want to interpolate using sample1d. To find the median depth in a 50 km window every 25 km along the track of cruise v3312, stored in v3312.dt, checking for gaps of 10km and asymmetry of 0.3, try filter1d v3312.dt -FM50 -T0/100000/25 -L10 -S0.3 > v3312_filt.dt SEE ALSO
gmt(1gmt), sample1d(1gmt) 1 Jan 2004 FILTER1D(l)
All times are GMT -4. The time now is 12:36 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy