Calculate difference between consecutive data points in a column from a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Calculate difference between consecutive data points in a column from a file
# 1  
Old 10-07-2010
Calculate difference between consecutive data points in a column from a file

Hi,

I have a file with one column data (sample below) and I am trying to write a shell script to calculate the difference between consecutive data valuse i.e

Var = Ni -N(i-1)

0.3141
-3.6595
0.9171
5.2001
3.5331
3.7022
-6.1087
-5.1039
-9.8144
1.6516
-2.725
3.982
7.769
8.88
8.392
2.188
3.13

I seen not to even know where to start from and so I really need some a lot of help please

Thank you
Malandisa
# 2  
Old 10-07-2010
Is this what you want?
Code:
awk 's{print ((s>$0)?s-$0:$0-s)}{s=$0}' file

# 3  
Old 10-07-2010
Code:
awk 'NR==1{s=$1;next}{print $1-s;s=$1}' infile

-3.9736
4.5766
4.283
-1.667
0.1691
-9.8109
1.0048
-4.7105
11.466
-4.3766
6.707
3.787
1.111
-0.488
-6.204
0.942

# 4  
Old 10-07-2010
Thank you both Danmero and rdcwayx!

I want to make this a learning experience so please be patient with me!

rdcwayx's gives what I am looking for! danmero's approach makes everything positive (but is actually doing the difference just fine, only that it makes it positive) ....somehow I have no clue what is happening in this command! I have some idea what is happening in rdcwayx's command! Thank you again!! I am surely learning shell scripting here, I actually doubted this was possible with awk!!!

daanmero please can I ask a favour that you just explain what the different symbols actually mean and how that combination achieves the result? and is it possible to modify the command so that it preserves the negative in the results? and any advice where I can find a detailed user guide for awk that?

Thank you all!
# 5  
Old 10-08-2010
Here is the user guide for awk, i used by myself.

The GNU Awk User's Guide
# 6  
Old 10-08-2010
Quote:
Originally Posted by malandisa
Hi,

I have a file with one column data (sample below) and I am trying to write a shell script to calculate the difference between consecutive data valuse
In my opinion the difference between -1 and 1 is 2, like that a difference between any two numbers will be positive.
From here I see that you just want to subtract a number from a previous number and in this case the solution provided by rdcwayx will work just fine.
This User Gave Thanks to danmero For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grabbing data between 2 points in text file

I have a text file that shows the output of my solar inverters. I want to separate this into sections. overview , device 1 , device 2 , device 3. Each device has different number of lines. but they all have unique starting points. Overview starts with 6 #'s, Devices have 4#'s and their data starts... (6 Replies)
Discussion started by: Mikey
6 Replies

2. Shell Programming and Scripting

Calculate difference in timestamps based on unique column value

Hi Friends, Require a quick help to write the difference between 2 timestamps based on a unique column value: Input file: 08/23/2012 12:36:09,JOB_5340,08/23/2012 12:36:14,JOB_5340 08/23/2012 12:36:22,JOB_5350,08/23/2012 12:36:26,JOB_5350 08/23/2012 13:08:51,JOB_5360,08/23/2012... (4 Replies)
Discussion started by: asnandhakumar
4 Replies

3. Shell Programming and Scripting

Transpose timestamp based on column values and calculate time difference

Hello Expert, I need to transpose Date-Timestamp based on same column values and calculate time difference. The input file would be as below and required output is mentioned in the bottom INPUT File ======== 08/23/2012 12:36:09 JOB_5340 08/23/2012 12:36:14 JOB_5340 08/23/2012... (2 Replies)
Discussion started by: asnandhakumar
2 Replies

4. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

5. Shell Programming and Scripting

Calculate data and make it into new column using awk

Hi everyone, just some simple question... i've been using a awk script to calculate my data... i have 3 files: file a1.txt: 2 3 4 5 3 4 file a2.txt: 4 5 6 7 8 (1 Reply)
Discussion started by: yat
1 Replies

6. Shell Programming and Scripting

Find time difference between two consecutive lines in same file.

Hello I have a file in following format: IV 08:09:07 NM 08:12:01 IC 08:12:00 MN 08:14:20 NM 08:14:15 I need a script to compare time on each line with previous line and show the inconsecutive line. Ex.: 08:12:00 08:14:15 A better way... (6 Replies)
Discussion started by: vilibit
6 Replies

7. Shell Programming and Scripting

Calculate age of a file | calculate time difference

Hello, I'm trying to create a shell script (#!/bin/sh) which should tell me the age of a file in minutes... I have a process, which delivers me all 15 minutes a new file and I want to have a monitoring script, which sends me an email, if the present file is older than 20 minutes. To do... (10 Replies)
Discussion started by: worm
10 Replies

8. UNIX for Dummies Questions & Answers

How to get data only inside polygon created by points which is part of whole data from file?

hiii, Help me out..i have a huge set of data stored in a file.This file has has 2 columns which is latitude & longitude of a region. Now i have a program which asks for the number of points & based on this number it asks the user to enter that latitude & longitude values which are in the same... (7 Replies)
Discussion started by: reva
7 Replies

9. UNIX for Dummies Questions & Answers

calculate the difference between numbers in lines of file

Hi everyone, i have files containing lines of number: 109 107 67 62 .. .. i want to calculate the difference between numbers in lines 1 and 2, 3 and 4, 5 and 6 and so on. would someone help me please?. Thanks (12 Replies)
Discussion started by: ahidayat
12 Replies

10. Shell Programming and Scripting

Calculate the time difference between a local file and a remote file.

I m stuck with a issue. I need to calculate the time difference between two files.. one on the local machine and one on the remote machine using a script. Can any one suggest the way this can be achevied Thanks, manohar (1 Reply)
Discussion started by: meetmano143
1 Replies
Login or Register to Ask a Question