Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Calculating differences line by line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Calculating differences line by line
# 1  
Old 03-12-2013
Calculating differences line by line
i don't even know how to explain this. but i'll try my best.

so i have this data:

Code:
jime=1860,yime=1.23243
jime=1859,yime=1.23018
jime=1825,yime=1.15371
jime=1849,yime=1.20769
jime=1841,yime=1.1897
jime=1849,yime=1.20769
jime=1842,yime=1.19195
jime=1827,yime=1.15821
jime=1751,yime=0.98726
jime=1746,yime=0.976014
jime=1723,yime=0.92428
jime=1707,yime=0.888291
jime=1695,yime=0.8613
jime=1680,yime=0.82756
jime=1317,yime=0.0110665
jime=1310,yime=-0.00467853
jime=1291,yime=-0.0474151
jime=1013,yime=-0.672719
jime=982,yime=-0.742447
jime=970,yime=-0.769438
jime=956,yime=-0.800929
jime=853,yime=-1.03261
jime=819,yime=-1.10908
jime=797,yime=-1.15857
jime=782,yime=-1.19231

i need to calculate the percentages of the difference of the in fields in each line, with the next line.

for instance, what is the percentage differences (whether negative or positive) between:
Code:
jime=1859,yime=1.23018
and
jime=1825,yime=1.15371

then

Code:
jime=1825,yime=1.15371
and
jime=1849,yime=1.20769

and so on.

basically compare the line with the line that comes after it.

so in essence, the preferred output would be something like:

Code:
jime=1860,yime=1.23243
jime=1859,yime=1.23018,-0.18%
jime=1825,yime=1.15371
jime=1849,yime=1.20769,+4.46969%
jime=1841,yime=1.1897
jime=1849,yime=1.20769,.....
jime=1842,yime=1.19195
jime=1827,yime=1.15821,......
and so on

i used the following to calculate the percentages:
Code:
echo 1.15371 1.20769 | awk '{print ($1 - $2) / $2 * 100}'

Last edited by SkySmart; 03-12-2013 at 12:41 PM..
This User Gave Thanks to SkySmart For This Post:
Nohim Ys
# 2  
Old 03-12-2013
Code:
awk -F'=' 'BEGIN {
        CONVFMT="%f"
} ! ( NR % 2) {
        $0 = $0 "," ( v - $NF ) / ( $NF * 100 ) "%"
} NR % 2 {
        v = $NF
} 1 ' file

This User Gave Thanks to Yoda For This Post:
SkySmart
# 3  
Old 03-12-2013
Quote:
Originally Posted by bipinajith
Code:
awk -F'=' 'BEGIN {
        CONVFMT="%f"
} ! ( NR % 2) {
        $0 = $0 "," ( v - $NF ) / ( $NF * 100 ) "%"
} NR % 2 {
        v = $NF
} 1 ' file

thank you!! looks like this would work. maybe its my calculation that's wrong, but it looks like the percentages are off.

i get this:

Code:
jime=1860,yime=1.23243
jime=1859,yime=1.23018,0.000018%
jime=1825,yime=1.15371
jime=1849,yime=1.20769,-0.000447%
jime=1841,yime=1.1897
jime=1849,yime=1.20769,-0.000149%

like take for instance, the above bolded should be:

Code:
echo 1.23243 1.23018 | awk '{print ($1 - $2) / $2 * 100}'
0.1829%

# 4  
Old 03-12-2013
Try this:
Code:
awk -F= '{prev=$NF; print; getline; print $0","($NF-prev)/$NF*100"%"}' input

This User Gave Thanks to mirni For This Post:
SkySmart
# 5  
Old 03-12-2013
Round brackets where not placed correctly:
Code:
awk -F'=' 'BEGIN {
        CONVFMT="%.4f"
} ! ( NR % 2) {
        $0 = $0 "," (( v - $NF ) / $NF ) * 100  "%"
} NR % 2 {
        v = $NF
} 1 ' file

If you want to subtract the other way around:
Code:
awk -F'=' 'BEGIN {
        CONVFMT="%.4f"
} ! ( NR % 2) {
        $0 = $0 "," (( $NF - v ) / $NF ) * 100  "%"
} NR % 2 {
        v = $NF
} 1 ' file

This User Gave Thanks to Yoda For This Post:
SkySmart
# 6  
Old 03-12-2013
the contents of the datafile changed:

Code:
jime=1860,yime=1.23243,lime=[Mar-12-(UTC)]
jime=1859,yime=1.23018,lime=[Mar-12-(UTC)]
jime=1825,yime=1.15371,lime=[Mar-12-(UTC)]
jime=1849,yime=1.20769,lime=[Mar-12-(UTC)]
jime=1841,yime=1.1897,lime=[Mar-12-(UTC)]
jime=1849,yime=1.20769,lime=[Mar-12-(UTC)]

i tried manipulating the fields so the script can adapt to this new datafile layout. "$NF" to work on the second column:

Code:
awk -F= '{prev=$3; print; getline; print $0","($NF-prev)/$NF*100"%"}' datafile

but i keep getting:

Code:
awk: (FILENAME=mile FNR=2) fatal: division by zero attempted

# 7  
Old 03-12-2013
You can split on either = or , and hard-code 4th column:
Code:
awk -F"[=,]" '{prev=$4; print; getline; print $0","($4-prev)/$4*100"%"}' input

This User Gave Thanks to mirni For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies

2. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

3. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

4. Shell Programming and Scripting

Calculating average for every Nth line in the Nth column

Is there an awk script that can easily perform the following operation? I have a data file that is in the format of 1944-12,5.6 1945-01,9.8 1945-02,6.7 1945-03,9.3 1945-04,5.9 1945-05,0.7 1945-06,0.0 1945-07,0.0 1945-08,0.0 1945-09,0.0 1945-10,0.2 1945-11,10.5 1945-12,22.3... (3 Replies)
Discussion started by: ncwxpanther
3 Replies

5. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

6. Shell Programming and Scripting

How to make diff show differences one line at a time and not group them?

Is there a way to tell diff to show differences one line at a time and not to group them? For example, I have two files: file1: line 1 line 2 line 3 diff line 4 diff line 5 diff line 6 line 7 file2: line 1 line 2 line 3 diff. line 4 diff. line 5 diff. line 6 line 7 (13 Replies)
Discussion started by: mmr11408
13 Replies

7. Solaris

Line too long error Replace string with new line line character

I get a file which has all its content in a single row. The file contains xml data containing 3000 records, but all in a single row, making it difficult for Unix to Process the file. I decided to insert a new line character at all occurrences of a particular string in this file (say replacing... (4 Replies)
Discussion started by: ducati
4 Replies

8. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies
Login or Register to Ask a Question