Improve script made to calculate value based on present and previous line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Improve script made to calculate value based on present and previous line
# 1  
Old 11-22-2010
Improve script made to calculate value based on present and previous line

Hi all,

I have made at small script to make a simple calculation on a file which is formatted in this way:
Quote:
[Name] [Value]
gi|110645304|ref|NC_002516.2| 747
gi|110645304|ref|NC_002516.2| 748
gi|110645304|ref|NC_002516.2| 749
gi|110645304|ref|NC_002516.2| 750
gi|110645304|ref|NC_002516.2| 751
gi|110645304|ref|NC_002516.2| 752
gi|110645304|ref|NC_002516.2| 753
gi|110645304|ref|NC_002516.2| 754
I want to create a new file in which the value of particular line minus the previous line is printed.

So my wanted output is:
Quote:
[Name] [Value]
gi|110645304|ref|NC_002516.2| 747
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
gi|110645304|ref|NC_002516.2| 1
I have made the following program to do the job but it is very slow (and my file has 6 million lines):
Code:
#!/bin/bash

FILE="$1"

VALUEBEFORE=0

while read line
do

VALUE=`echo $line | awk '{print($2)}'`
NAME=`echo $line | awk '{print($1)}'`

TEST=`echo $VALUE $VALUEBEFORE | awk '{print($1 - $2)}'`

VALUEBEFORE=`echo $line | awk '{print($2)}'`

echo "$NAME   $TEST"

done < $FILE

Can any suggest how the script can work faster?
Thanks.
# 2  
Old 11-22-2010
You can give this a try:
Code:
#!/bin/bash

FILE="$1"

awk 'NR==1{print;next}!val{val=$NF;print ;next}{p=val;val=$NF;$NF= $NF-p}1' $FILE > newfile


Last edited by Franklin52; 11-22-2010 at 11:17 AM..
This User Gave Thanks to Franklin52 For This Post:
# 3  
Old 11-22-2010
Thanks Franklin52

I saw you also posted this code which is more simple:
Code:
awk 'p{print $0-p}{p=$0}' file

So I am using this instead as I better understand what is happening as I did not really understand the "!val".
Best
# 4  
Old 11-22-2010
Simpler and surely faster, no calls to external programs.
Code:
#!/bin/bash
OLDVAL=0
while read NAME VALUE
do
   echo "$NAME $((VALUE-OLDVAL))"
   OLDVAL=$VALUE
done <"$1"

# 5  
Old 11-22-2010
Code:
awk -v s=0 'NR==1{print;next}{print $1,$2-s;s=$2}' infile

# 6  
Old 11-22-2010
Code:
awk  'NR-1{x=y;y=$NF;$NF=$NF-x}1' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

UNIX script to append multiple text files into one file based on pattern present in filaname

Hi All-I am new to Unix , I need to write a script. Can someone help me with a requirement where I have list of files in a directory, I want to Merge the files if a pattern of string matches in filenames? AAAL_555A_ORANGE1_F190404.TXT AAAL_555A_ORANGE2_F190404.TXT AAAL_555A_ORANGE3_F190404.TXT... (6 Replies)
Discussion started by: Shankar455
6 Replies

2. Shell Programming and Scripting

Returning multiple outputs of a single line based on previous repeated lines

Hello, I am trying to return a time multiple times from a file that has varying output just before the time instance, i.e. cat jumped cat jumped cat jumped time = 1.1 cat jumped cat jumped time = 1.2 cat jumped cat jumped time = 1.3 In this case i would like to output a time.txt... (6 Replies)
Discussion started by: ryddner
6 Replies

3. Shell Programming and Scripting

awk script -print line when $2 > $2 of previous line

Hi all, From a while loop I am reading a sorted file where I want to print only the lines that have $1 match and $2 only when the difference from $2 from the previous line is > 30. Input would be like ... AN237 010 193019 0502 1 CSU Amoxycillin AN237 080 ... (2 Replies)
Discussion started by: gafoleyo73
2 Replies

4. UNIX for Dummies Questions & Answers

Script to count character present in a line

Suppose i have multiple line like below in a file. ASDFAFAGAHAHAHA AGAHAHAJGAFAGAH AHAHAKAHAHAHAKA I need a bash script to count a character and also Also count the number of character present in each line . suppose for line 1: A=x, S=y, D=x and so on and total character=15. where x y and z is... (3 Replies)
Discussion started by: XXLMMN
3 Replies

5. Shell Programming and Scripting

Perl: Conditional replace based on previous and current value in a line

I need to read the contents of a file. Then I need to grep for a keyword and replace part of the grepped line based on the condition of previous and present line. Example input file: V { port1 = P; port2 = 0; shift_port = P0; /* if next shift_port is P0 I need... (9 Replies)
Discussion started by: naveen@
9 Replies

6. Emergency UNIX and Linux Support

invoke one script based on previous script execution

I am database guy and not very good at shell scripts. I am seeking help to sharp my script coding. I have 5 scripts 1. master script. I use this one to call other four scripts to do database work. 2. db_backup_1 and log_backup_1 3. db_backup_2 and log_backup_2 in master script, I want to... (4 Replies)
Discussion started by: duke0001
4 Replies

7. Shell Programming and Scripting

Awk script to create new file based on previous line

Need help creating a script that does the following: Sort a file Compare the previous line "last field" with current line "last field" If they are the same, print output to a file If they are different print output to a new file The script should keep creating new files if the previous... (5 Replies)
Discussion started by: Muga801
5 Replies

8. Shell Programming and Scripting

Printing previous line based on pattern using sed

Hi, I have a written a shell script to get the previous line based on the pattern. For example if a file has below lines: ---------------------------------------------- #UNBLOCK_As _per #As per 205.162.42.92 #BLOCK_As_per #----------------------- #input checks abc.com... (5 Replies)
Discussion started by: Anjan1
5 Replies

9. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

10. Shell Programming and Scripting

Automatically Rerun a script based on previous execution output

Hi Experts, I have a shell script called "updatevs" that is scheduled to run at 6.00 am everyday via cronjob. The cronjob will execute this script and output to a log file. The functionality of this script is to read the database and run a set of commands. This script is generally successful... (6 Replies)
Discussion started by: forumthreads
6 Replies
Login or Register to Ask a Question