Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Resample time series data with replacement any way to do this in awk or just bash script
Top Forums Programming Resample time series data with replacement any way to do this in awk or just bash script Post 302893564 by malandisa on Wednesday 19th of March 2014 07:10:17 PM
Old 03-19-2014
Resample time series data with replacement any way to do this in awk or just bash script
I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500 is missing in my original data but I need that time slot with some interpolated value based on any interpolation method.
Code:
0          11.03    2    5.515
0.00833    11.01    2    5.505
0.01667    10.97    2    5.485
0.02500    10.9    2    5.45
0.03333    10.86    2    5.43
0.04167    10.83    2    5.415
0.05000    10.75    2    5.375
0.05833    10.73    2    5.365
0.06667    10.7    2    5.35
0.07500    10.67    2    5.335
0.08333    10.65    2    5.325
0.09167    10.63    2    5.315
0.10000    10.6    2    5.3
0.10833    10.55    2    5.275
0.11667    10.51    2    5.255
0.12500    10.5    2    5.25
0.13333    10.45    2    5.225
0.14167    10.44    2    5.22
0.15833    10.39    2    5.195
0.16667    10.35    2    5.175
0.17500    10.28    2    5.14
0.18333    10.25    2    5.125
0.19167    10.21    2    5.105
0.20000    14.13    3    4.71
0.20833    14.11    3    4.70333
0.21667    14.06    3    4.68667
0.22500    21.37    4    5.3425
0.23333    21.28    4    5.32
0.24167    21.18    4    5.295
0.25000    21.12    4    5.28
0.25833    21.05    4    5.2625
0.26667    20.95    4    5.2375
0.27500    20.87    4    5.2175
0.28333    20.79    4    5.1975
0.29167    20.67    4    5.1675
0.30000    20.59    4    5.1475
0.30833    20.54    4    5.135
0.31667    20.5    4    5.125
0.32500    20.44    4    5.11
0.33333    20.35    4    5.0875
0.34167    20.3    4    5.075
0.35000    20.22    4    5.055
0.35833    20.15    4    5.0375
0.36667    20.09    4    5.0225
0.37500    20.02    4    5.005
0.38333    19.99    4    4.9975
0.39167    19.97    4    4.9925
0.40000    19.87    4    4.9675

And here is the desired output
Code:
0          5.515
0.05000    5.375
0.10000    5.3
0.15000    some iterpolated value
0.20000    4.71
0.25000    5.28
0.30000    5.1475
0.35000    5.055
0.40000    4.9675

Here is what I have tried with awk
Code:
awk '1 == (NR +0) % 6 { printf "%-8s   %-8.4s\n",  $1, $4}' TECavgFile.txt > TECavgResample.txt

This is not producing what I need since I am just sampling every 6 datapoints. I would work if I have a complete data set without the missing datapoints where. This problem exists as long as my data has gaps as the time at which I need a sample is shifted by any gaps.

Is there any way to do this in awk or bash scripting?

Thank you for your kind assistance

Malandisa
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

perl as awk replacement in a script.

Hey all, Im trying to write a script on windows, which Im not too familiar with. Im generally a bash scripting guy but am using perl for this case. My question is... I have this exact output: 2 Dir(s) 6,380,429,312 bytes free and I just need to get the number out... (4 Replies)
Discussion started by: trey85stang
4 Replies

2. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

3. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

4. UNIX for Dummies Questions & Answers

Simple awk script for positional replacement in text?

I have a string of letters. (They happen to be DNA, not that it's relevant to the question.) For analysis purposes, I need to replace the information at some of the sites. I need to do this based on their position, not the information in that position. I also need to ignore differences at other... (10 Replies)
Discussion started by: JFS
10 Replies

5. Shell Programming and Scripting

Subtracting time with awk - BASH/Debian GNU Linux

I'm sure this is simple and I've been looking at examples for days on end but can't seem to come to grips with awk. What I have: mplayer -v dvd:// -identify -vo null -ao null -nolirc -nojoystick -frames 0 2>/dev/null >> /tmp/MplayerOut ChapterStart=($(grep CHAPTERS: /tmp/MplayerOut |sed... (3 Replies)
Discussion started by: rickenbacherus
3 Replies

6. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

7. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

8. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

9. UNIX for Beginners Questions & Answers

Awk/bash one liner replacement for a if condition

Hi. I wrote this small bash script, i want to compare second column from file1 with file2 if a pattern matches. Files are small and I am sure that pattern occurs only once. I think this can be rewritten into a awk one liner. Appreciate if someone could give me idea. Whole NR FNR confuse me :o ... (6 Replies)
Discussion started by: ctrld
6 Replies

LEARN ABOUT MOJAVE

locale::codes::langfam5.18

Locale::Codes::LangFam(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangFam(3pm)

NAME

       Locale::Codes::LangFam - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangFam;

	  $lext = code2langfam('apa');		       # $lext gets 'Apache languages'
	  $code = langfam2code('Apache languages');    # $code gets 'apa'

	  @codes   = all_langfam_codes();
	  @names   = all_langfam_names();

DESCRIPTION

       The "Locale::Codes::LangFam" module provides access to standard codes used for identifying language families, such as those as defined in
       ISO 639-5.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639-5
       language family codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language families. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langfam('apa','alpha');
	  $lext = code2langfam('apa',LOCALE_LANGFAM_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from ISO 639-5 such as 'apa' for Apache languages.

	   This is the default code set.

ROUTINES

       code2langfam ( CODE [,CODESET] )
       langfam2code ( NAME [,CODESET] )
       langfam_code2code ( CODE ,CODESET ,CODESET2 )
       all_langfam_codes ( [CODESET] )
       all_langfam_names ( [CODESET] )
       Locale::Codes::LangFam::rename_langfam  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangFam::add_langfam  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangFam::delete_langfam  ( CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangFam::delete_langfam_alias  ( NAME )
       Locale::Codes::LangFam::rename_langfam_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::delete_langfam_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.loc.gov/standards/iso639-5/id.php
	   ISO 639-5 .

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.18.2							    2013-11-04					       Locale::Codes::LangFam(3pm)
All times are GMT -4. The time now is 02:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy