Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

optimising script

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers optimising script
# 1  
Old 10-16-2010
optimising script
I have a script which is as below

Code:
input=$1
acts_to_be_removed=$2
if [ "$#" != "2" ]
 
then
echo usage input_file acts_to_be_removed
fi
 
for rec in `cat ${input}`
do
ACCOUNT_NO=`echo ${rec} | tr ' ' '|'|  cut -c14-31|cut -d'|' -f1 | cut -d "/" -f1`
grep ${ACCOUNT_NO} ${acts_to_be_removed} 
if [ $? -ne 0 ]
then
grep ${ACCOUNT_NO} ${input} >> ${input}.OUT
fi


The script takes ages to complete which I believe is due to the file acts_to_be_removed that has around 90000 records inside.

The file "input"has around 300 lines

Is there any way to optimise the script such that it works faster.



Any help will be much appreciated.
Last edited by DukeNuke2; 10-16-2010 at 06:09 AM..
# 2  
Old 10-16-2010
Instead of sharing your code, you should post the requirements that the script should meet, and some sample input and output data.
# 3  
Old 10-16-2010
Sorry bartus. Please find the details below

input file1 - X has data like this

Code:
751089A15061001731137/515737110UPHILL            48259090070116                  E                  11879260070246                        09270131

The lines that are in bold are the accounts to be grepped.

Input file 2 -Y has 99000 records like below
Code:
95196674
95202662
95206041
95206564
95208562
95223675
95243600
95252346
95274633

script has to grep the accounts in file Y in file X and if X does not have the accounts then that line from X should be appended to a OUTPUT file.

Please let me know in case if you require any further info
Last edited by Scott; 10-16-2010 at 07:01 AM.. Reason: Code tags
# 4  
Old 10-16-2010
Code:
$ ruby -ne 'BEGIN{a=File.read("Y").split("\n")};print if not a.include?($_[13,8])' X

# 5  
Old 10-16-2010
Thanks Kurumi.

However I get an error when i execute the command which states
Code:
"sh: ruby:  not found."


Please help
Last edited by Scott; 10-16-2010 at 07:02 AM.. Reason: Code tags
# 6  
Old 10-16-2010
Try:
Code:
sed 's/^/^.{13}/' file2 | grep -Evf - file1 > file1.out

 
Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Shell Programming and Scripting

Shell script works fine as a standalone script but not as part of a bigger script

Hello all, I am facing a weird issue while executing a code below - #!/bin/bash cd /wload/baot/home/baotasa0/sandboxes_finance/ext_ukba_bde/pset sh UKBA_publish.sh UKBA 28082015 3 if then echo "Param file conversion for all the areas are completed, please check in your home directory"... (2 Replies)
Discussion started by: ektubbe
2 Replies

3. UNIX for Dummies Questions & Answers

Calling a script from master script to get value from called script

I am trying to call a script(callingscript.sh) from a master script(masterscript.sh) to get string type value from calling script to master script. I have used scripts mentioned below. #masterscript.sh ./callingscript.sh echo $fileExist #callingscript.sh echo "The script is called"... (2 Replies)
Discussion started by: Raj Roy
2 Replies

4. Shell Programming and Scripting

Script will keep checking running status of another script and also restart called script at night

I am using blow script :-- #!/bin/bash FIND=$(ps -elf | grep "snmp_trap.sh" | grep -v grep) #check snmp_trap.sh is running or not if then # echo "process found" exit 0; else echo "process not found" exec /home/Ketan_r /snmp_trap.sh 2>&1 & disown -h ... (1 Reply)
Discussion started by: ketanraut
1 Replies

5. Shell Programming and Scripting

create a shell script that calls another script and and an awk script

Hi guys I have a shell script that executes sql statemets and sends the output to a file.the script takes in parameters executes sql and sends the result to an output file. #!/bin/sh echo " $2 $3 $4 $5 $6 $7 isql -w400 -U$2 -S$5 -P$3 << xxx use $4 go print"**Changes to the table... (0 Replies)
Discussion started by: magikminox
0 Replies

6. Shell Programming and Scripting

optimising the a perl script

HI, I am in urgent need of optimising the a perl script which is able to handle to 1000 request in one min and the requirement is to handle 5000 requests in one minute. I have that file with me. Could any one let me know how can we find out which portion of the code is taking much time, or... (3 Replies)
Discussion started by: rish1980
3 Replies
Login or Register to Ask a Question