Sponsored Content
Top Forums Programming Find gaps in time data and replace missing time value and column 2 value by interpolation in awk Post 302931885 by RudiC on Friday 16th of January 2015 07:08:40 AM
Old 01-16-2015
Please be aware that your data are not a linear function of time; on top of some noise it has a small curvature, and the six data points right after the gap are somewhat lower than they should be (there's a jump in values but not in time delta at point 7).
This is a quick and dirty approximation to exactly your problem and data; no error checking etc. is done. It's sort of a linear interpolation between the given boundaries although we know the boundary to the right is questionable. On top, my mawk has a problem with the D1 > D0 comparison, sometimes the delta is -1E-16, sometimes it's +71E-16, so a few extra lines are being "interpolated". I don't have a good solution at hand; either increase the to be compared value slightly (yuck!) or use sort -u on the result (yuck!)...
However, try
Code:
awk     'NR==1  {L1=$1; L2=$2; D0=0.05}
                {D1=$1-L1; D2=$2-L2
                 if (D1 > D0)   { n=D1/D0                   # D1 sometimes is sliiightly larger than D0
                                 ST=D2/n
                                 for (i=1; i<=int(n); i++) printf "%7.5f   %6.4f <---\n", L1+D0*i, L2+ST*i}
                 L1=$1; L2=$2
                 print $0}
        ' file

This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

calculate the average of time series data using AWK

Hi, I have two time series data (below) merged into a file. t1 and t2 are in unit of second I want to calculate the average of V1 every second and count how many times "1" in V2 is occur within a second Input File: t1 V1 t2 V2 10.000000... (5 Replies)
Discussion started by: nica
5 Replies

2. Shell Programming and Scripting

How to get data between the start time and end time?

Hi, Can anyone help me how can I get the line that between the start time and end time. file1.txt 15/03/2009 20:45:03 Request: - Data of this line 15/03/2009 20:45:12 Response: - Data of this line 15/03/2009 22:10:40 Request: - Data of this line 15/03/2009 22:10:42 Response: - Data of... (1 Reply)
Discussion started by: tanit
1 Replies

3. Shell Programming and Scripting

Searching for Gaps in Time

I am very new to shell scripting. We use C-Shell here and I know the issues that surround it. I hope a solution can be created using awk, sed, etc... instead of having to write a program. I have an input file that is sorted by date and time in ascending order ... (2 Replies)
Discussion started by: jclanc8
2 Replies

4. Shell Programming and Scripting

Find and Convert UTC Time to PST Time

Hello All - I have a script that grabs data from the net and outputs the following data 46029 46.144 -124.510 2010 07 26 22 50 320 4.0 6.0 2.2 9 6.8 311 1012.1 -0.9 13.3 13.5 13.3 - - 46041 47.353 -124.731 2010 07 26 22 50 250 2.0 3.0 1.6 8 6.4 - 1011.6 - ... (0 Replies)
Discussion started by: drexnefex
0 Replies

5. Shell Programming and Scripting

awk/sed to search & replace data in first column

Hi All, I need help in manipulating the data in first column in a file. The sample data looks like below, Mon Jul 18 00:32:52 EDT 2011,NULL,UAT Jul 19 2011,NULL,UAT 1] All field in the file are separated by "," 2] File is having weekly data extracted from database 3] For eg.... (8 Replies)
Discussion started by: gr8_usk
8 Replies

6. UNIX for Dummies Questions & Answers

Converting string date time to unix time in AWK

I'd like to convert a date string in the form of sun aug 19 09:03:10 EDT 2012, to unixtime timestamp using awk. I tried This is how each line of the file looks like, different date and time in this format Sun Aug 19 08:33:45 EDT 2012, user1(108.6.217.236) all: test on the 17th ... (2 Replies)
Discussion started by: bkkid
2 Replies

7. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

8. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

9. Shell Programming and Scripting

XY interpolation by time in awk

Hi I am a newbie in awk scripting. I'm working with a file with xy coordinates that were acquired with a time stamp. All the time stamps were recorded but not the XY coordinates. Let see an example: FFID X Y UNIX TIME TIMEGAP... (8 Replies)
Discussion started by: Joćo Noiva
8 Replies

10. Shell Programming and Scripting

Awk: time intervals based on epoch time

I have a list of epoch times delimited by "-" as follows: 1335078000 - 1335176700 1335340800 - 1335527400 1335771300 - 1335945600 1336201200 - 1336218000 The corresponding dates are: 20120422 1000 - 20120423 1325 20120425 1100 - 20120427 1450 20120430 1035 - 20120502 1100 ... (3 Replies)
Discussion started by: alex2005
3 Replies
TS.CONF(5)                                                             tslib                                                            TS.CONF(5)

NAME
ts.conf - Configuration file for tslib, controlling touch screens for embedded devices. DESCRIPTION
The idea of tslib is to have a core library that provides standardised services, and a set of plugins to manage the conversion and filtering as needed. The plugins for a particular touchscreen are loaded automatically by the library under the control of a static configuration file, /etc/ts.conf. /etc/ts.conf gives the library basic configuration information. Each line specifies one module, and the parameters for that module. The modules are loaded in order, with the first one processing the touchscreen data first. For example: module_raw input module variance delta=30 module dejitter delta=100 module linear MODULE PARAMETERS
variance Tries to do it's best in order to filter out random noise coming from touchscreen ADC's. This is achieved by limiting the sample movement speed to some value (e.g. the pen is not supposed to move quicker than some threshold). This is a 'greedy' filter, e.g. it gives less samples on output than receives on input. o Parameters: delta. Set the squared distance in touchscreen units between previous and current pen position (e.g. (X2-X1)^2 + (Y2-Y1)^2). This defines the criteria for determining whenever two samples are 'near' or 'far' to each other. If the distance between previous and current sample is 'far', the sample is marked as 'potential noise'. This doesn't mean yet that it will be discarded; if the next reading will be close to it, this will be considered just a regular 'quick motion' event, and it will sneak to the next layer. Also, if the sample after the 'potential noise' is 'far' from both previously discussed samples, this is also considered a 'quick motion' event and the sample sneaks into the output stream. dejitter Removes jitter on the X and Y co-ordinates. This is achieved by applying a weighted smoothing filter. The latest samples have most weight; earlier samples have less weight. This allows to achieve 1:1 input->output rate. o Parameters: delta. Squared distance between two samples ((X2-X1)^2 + (Y2-Y1)^2) that defines the 'quick motion' threshold. If the pen moves quick, it is not feasible to smooth pen motion, besides quick motion is not precise anyway; so if quick motion is detected the module just discards the backlog and simply copies input to output. linear Linear scaling module, primarily used for conversion of touch screen co-ordinates to screen co-ordinates. o Parameters: xyswap. Interchange the X and Y co-ordinates -- no longer used or needed if the new linear calibration utility ts_calibrate is used. ENVIRONMENT VARIABLES
Latest versions of the Xorg tslib input driver use hal to configure the touchscreen within Xorg. Environment variables are only needed for the tslib commands. TSLIB_TSDEVICE If the default touchscreen device cannot be found, set the TSLIB_TSDEVICE environment variable to the touchscreen device to use. Default: /dev/input/event0 TSLIB_CONSOLEDEVICE Tslib default console device. Default: /dev/tty TSLIB_CALIBFILE Stores calibration data obtained using ts_calibrate. Default: /etc/pointercal TSLIB_CONFFILE Set a different location for the ts.conf configuration file itself. Default; /etc/ts.conf. TSLIB_FBDEVICE Framebuffer device to use for the touchscreen support. Default: /dev/fb0. TSLIB_PLUGINDIR Plugin directory for tslib. Default: /usr/lib/$triplet/ts0/ where triplet is the MultiArch path, e.g. arm-linux-gnueabi. AUTHOR
This manual page was written by Neil Williams codehelp@debian.org for the Debian(TM) distribution because the original program does not have a manual page, but may be used by others. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version published by the Free Software Foundation. On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL. AUTHOR
Russell King Author. COPYRIGHT
Copyright (C) 2008 Neil Williams Debian 5 Wed 06 Aug 2008 10:46:33 ART TS.CONF(5)
All times are GMT -4. The time now is 11:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy