Improve script and get new output file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Improve script and get new output file
# 1  
Old 06-12-2015
Improve script and get new output file

Gents,

Using the following script, I got the changes as desired in the output file called (spread_2611.x01.new). Complete file as input (spread_2611.x01).

Can you please have a look to my script and improve it please. Smilie

Also I would like to I get a additional selecting only the records were the changes were done, please.

Here my script and attached the data to run the script.

Code:
#!/bin/bash
             read -p "First : " fsw
             read -p "Last  : " lsw

file="datatochange.txt"

touch $file

        for swath in $(seq $fsw $lsw)
              do 
awk '{\
	ori_line=substr($5,1,5);\
	ori_point=substr($5,6,5);\
	off_line=substr($1,2,5);\
	off_point=substr($1,7,5);\
	printf ("spread_'${swath}'.x01 %5d.00   %5d.00 %5d.00   %5d.00\n",ori_line,ori_point,off_line,off_point)}' $swath"offb1-Sx.sps" >> $file

done

awk '{F=$1;a[F];e=(F~/x01$/)?"  ":z;s=$2"  "$3;p=$2"  "$3;r=$4"  "$5;
print "s/" s "/" r "/" >> F".sed"
}END{
for(i in a){print "sed -f "i".sed "i " >>"i".new">>"changeindex"}
}' datatochange.txt

sh changeindex

awk '{print $1}' datatochange.txt | uniq > datatochange1.txt 

awk '{a[$1]++ ; print $1}' datatochange1.txt | while read i
mv "$i" "$i.old" && mv "$i.new" "$i"
rm -f "$i.sed"
done

Later I will use this script to change many files at the same time.

Thanks for your help
# 2  
Old 06-12-2015
Having a hard time to read and try to understand your snippet. What problems do you encounter that need improvement? Real errors? Performance problem?
One error I found by just looking: there's a do missing in the last while loop.
This User Gave Thanks to RudiC For This Post:
# 3  
Old 06-12-2015
Dear RudiC
Well It works Fine. When i say improve
I though you can modify It or get other way to do the same output more faster or efficient as your say my script is difficult to readSmilie.

Also i am missing a output For only the records where the changes where done, i can get It with grep using input and output. But maybe there is other way

Thanks For your help
# 4  
Old 06-12-2015
Well, may I doubt "It works Fine"? Parts of it, maybe, but still that "do" is missing, for certain leading to an error message.
Every programmer has his/her own style, and none of those is THE style, but - please - adopt some indentation habits that structure your code making it easier to read and understand.
For your clarity and by sheer courtesy to us, do the first step yourself and tidy that thing up. If correctly, structuredly, and nicely presented, optimization opportunities will lend themselves to implemention...
This User Gave Thanks to RudiC For This Post:
# 5  
Old 06-13-2015
Dear RudiC
Thanks For all your advises and help
I am not a programer, just i am learning. Now i am reading Pro Bash Programing to learn well bash.

I will try to optmize It.

Regates and Thanks again

---------- Post updated 06-13-15 at 01:23 PM ---------- Previous update was 06-12-15 at 09:42 PM ----------

Dear RudiC

Here my last update, hope it is more clear now. Smilie

Code:
#!/bin/bash
 
             read -p "First: " fxl

             read -p "Last : " lxl

#------------------------------------------------------------------------------------------------------
file="datatochange.txt"
touch $file
#------------------------------------------------------------------------------------------------------

        for valueNB in $(seq $fxl $lxl)
	do
        printf " |----------->> Processing value $valueNB \n"
 
	
## create Dbase ---
awk '{\
	ori_line=substr($5,1,5);\
	ori_point=substr($5,6,5);\
	off_line=substr($1,2,5);\
	off_point=substr($1,7,5);\
	printf ("'${sw_spread}' %5d.00   %5d.00 %5d.00   %5d.00\n",ori_line,ori_point,off_line,off_point)}' ${sw_offset} > $file

## create file with vps to be replaced && generate new files with changes done ----
	awk '{F=$1;a[F]?"   ":s=$2"  "$3;r=$4"  "$5;x=1;y=3;
	print "s/" s x "/" r y "/" >> F".sed"
	}END{
	for(i in a){print "sed -f "i".sed "i " > "i".new">"changefile"}
	}' $file
	sh changefile
	mv "$i" "$i.ori" && mv "$i.new" "$i"

## create file with only vps replaced && Concatenated QC files: Spread and Vps2Cancell ----	

	off_spread=$(mktemp)

	awk '{print substr($0,23,18)}' $i.sed > $off_spread
	grep -hFf $off_spread $i >> QC_spread_DB_$fxl"_"$lxl.x01
	awk 'BEGIN {OFS= ","}{print $6,"S,"substr($5,1,5),substr($5,6,5)}' $xl_guia >> QC_vps2cancell_DB_$fxl"_"$lxl.csv

## Deleting files
	rm -f "$i.sed"	*change*
	done

Kindly can you give other idea to change the code in red, to be more faster.. It works well but is very slow when I run the script for many files.

Thanks for your help

Last edited by Don Cragun; 06-13-2015 at 12:05 AM.. Reason: Remove duplicate post.
# 6  
Old 06-13-2015
A good step into the right direction! Some comments:
Once you adopted a style, you should stay consistent (which doesn't mean you can't evolve over time). In above, in "create DBase", you have "continuation backslashes", in the other, you don't (as they obviously aren't needed). You should stick to one single syntax.
And this }END{ is absolutely horrible to read. awk uses
pattern {action}
pairs, c.f. man awk. You could reflect that in your style. Why don't you look around in here and find a neat style that you like and can adopt and adapt?

---------- Post updated at 23:13 ---------- Previous update was at 23:02 ----------

Regarding your performance issue: Having a loop running awk to produce a shell script full of sed commands which then is executed by sh smells like to be sloooow.
And - your script has changed from post#1 to post#5. Sure it's still working?

There certainly are ways to improve, but I don't think I'll be working on THAT script. Explain verbosely and exactly what you need to be done, post sample input and output files, and someone in here might come up with a good solution for you.

Last edited by RudiC; 06-13-2015 at 06:38 PM..
These 2 Users Gave Thanks to RudiC For This Post:
# 7  
Old 06-13-2015
Dear RudiC
Thanks For your comments, the script works Fine But slow.
I will attach examples or input and output Files, they are big so i cant post Here,
Thanks again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Improve script

Gents, Is there the possibility to improve this script to be able to have same output information. I did this script, but I believe there is a very short code to get same output here my script awk -F, '{if($10>0 && $10<=15) print $6}' tmp1 | sort -k1n | awk '{a++} END { for (n in a )... (23 Replies)
Discussion started by: jiam912
23 Replies

2. Shell Programming and Scripting

How to improve an script?

Gents. I have 2 different scripts for the same purpose: raw2csv_1 Script raw2csv_1 finish the process in less that 1 minute raw2csv_2 Script raw2csv_2 finish the process in more that 6 minutes. Can you please check if there is any option to improve the raw2csv_2. To finish the job... (4 Replies)
Discussion started by: jiam912
4 Replies

3. Shell Programming and Scripting

Improve sftp script

Dear all, I have written two scripts to transfer files to another server outside the company. One is a batch script , and the other script calls the batch script, send the files and archive the file sent. The problem is, that I want to get the list of files which have been uploaded the the... (10 Replies)
Discussion started by: arrals_vl
10 Replies

4. UNIX for Dummies Questions & Answers

How to improve the performance of this script?

Hi , i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script #!/bin/bash file=$1 ofile=$2 cp $file $ofile mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13 mydates=$(echo "$mydates" | sort |... (5 Replies)
Discussion started by: vikatakavi
5 Replies

5. Shell Programming and Scripting

Looking to improve the output of this awk one-liner

I have the following awk one-liner I came up with last night to gather some data. and it works pretty well (apologies, I'm quite new with awk, and don't know how to format this pretty-printed). You can see the output with it. awk '{if ($8 == 41015 && $21 == "requests") arr+=$20;if ($8 == 41015... (3 Replies)
Discussion started by: DeCoTwc
3 Replies

6. Shell Programming and Scripting

Var Check Script (Help improve if possible)

I am working on a script to check the var on all of my systems. Can someone help me fix it to work better or give me suggestions. #!/bin/ksh IN=/path/to/list_of_workstations.txt while read hostnames do if ping $hostnames 1 | grep alive > /dev/null then percent=`ssh -q... (3 Replies)
Discussion started by: whotippedmycow
3 Replies

7. Shell Programming and Scripting

Want to improve the performance of script

Hi All, I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately. Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies

8. Shell Programming and Scripting

Improve the performance of a shell script

Hi Friends, I wrote the below shell script to generate a report on alert messages recieved on a day. But i for processing around 4500 lines (alerts) the script is taking aorund 30 minutes to process. Please help me to make it faster and improve the performace of the script. i would be very... (10 Replies)
Discussion started by: apsprabhu
10 Replies

9. Shell Programming and Scripting

Any way to improve performance of this script

I have a data file of 2 gig I need to do all these, but its taking hours, any where i can improve performance, thanks a lot #!/usr/bin/ksh echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')" function showHelp { cat << EOF >&2 syntax extreme.sh FILENAME Specify filename to parse EOF... (3 Replies)
Discussion started by: sirababu
3 Replies

10. Shell Programming and Scripting

Can I improve this script ???

Hi all, Still a newbie and learning as I go ... as you do :) Have created this script to report on disc usage and I've just included the ChkSpace function this morning. It's the first time I've read a file (line-by-bloody-line) and would like to know if I can improve this script ? FYI - I... (11 Replies)
Discussion started by: Cameron
11 Replies
Login or Register to Ask a Question