How to convert this script so that it runs the dbv at most 5x at a time instead of serially?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to convert this script so that it runs the dbv at most 5x at a time instead of serially?
# 1  
Old 12-29-2018
How to convert this script so that it runs the dbv at most 5x at a time instead of serially?

Hi,

Attached is a generated script of the database files if/when we want to run dbv = dbverify.

The script runs fine by itself but it does take awhile to finish. In a worse case scenario, it can take up to 7 hours to finish as the dbv runs serially.

I need help and guidance to somehow have some parallelism in terms of running the dbv. For instance, I want to at least have 5xdbv running at some point in time. So for a start, run 5 of these dbv's, and then 'monitor' in the background if some has finished, then start more dbv until there's 5xdbv running again and so on and so forth.

As there are several databases on the same server, I cannot simply ps -ef | grep dbv to check for the number of dbv commands running. That is, I may have two of these scripts that I want to run for two different databases. So the 5xdbv run is for each database. If there is two of these scripts then there should be 10xdbv running at one time, 5 for each database.

Not sure where or how to start Smilie Any advice will be much appreciated. Thanks in advance. BTW, prefers something that I can use on both Solaris and Linux

P.S.:
Belated Merry Christmas, Happy New Year to All.
# 2  
Old 12-29-2018
You should know the answer to the following questions before you start.
What effect does increasing the block size from 8192 to 32768 have?
What percentage of disk i/o capacity does a step take? You can use sar to determine this. and from this you should be able to calculate the number of megabytes per hour that you can process at a given disk usage threshold.



Pick two or three of the largest tables and manually run them in series. Record the wall clock time. Run them again in parallel and record the wall clock time. If you have other users on the system while you are running this experiment. record the response times when running serial and parallel.
These 2 Users Gave Thanks to jgt For This Post:
# 3  
Old 12-29-2018
Hi jgt,

I don't want to change the block size. I just want to be able to run several dbv's instead of one at a time.

Anyway, I think Run multiple sessions of DBV has what I am after.

Thanks.
# 4  
Old 12-31-2018
Quote:
Originally Posted by newbie_01
I don't want to change the block size. I just want to be able to run several dbv's instead of one at a time.
I think we (that includes jgt) have understood that. The problem is: verifing the DB creates a certain amount of usage on your disk subsystem because it causes (quite some amount of) disk I/O. if you run several instances of it in parallel it depends on the amount of disk I/O a single instance creates if these several instances run faster linearly with the number of instances.

i.e. if a single instance of the script taxes the disk 10% then running 5 simultaneous instances will drive that up to 50% and run perhaps somewhere in the ballpark of 1/5 of the time. If a single instance taxes the disk already 100% then running 5 instances in parallel will perhaps run about as fast a single instance because you can't tax any resource more than 100%. The disk I/O will be queued (thus serialised again) and this will put a cap at runtime. Perhaps reality is somewhere in between those extremes but you may want to find out beforehand where you stand.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 5  
Old 01-01-2019
Hi,

Yeah we have tested with running 5 dbv's at a time and the load is acceptable to the server.

--- Post updated at 04:25 PM ---

Hi jgt

Finally got to understand what you are wanting me to check first. Got confused when you mentioned about the block size thingy.
Thanks.
# 6  
Old 01-05-2019
FYI, had decided to keep it 'simple' use wait instead and run the dbv in the background using &. It's kinda 'parallel' but running it in group of 5 'serially' :-)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Script only runs first time through crontab

Hello, I am trying to run a script through crontab and it runs the first time and then it does not run. I tried to run a simple script (as shown below) and I see the same issue. #!/bin/ksh clear echo "Good Morning, World." > /tmp/test123 Crontab Entry: 30 09 * * *... (9 Replies)
Discussion started by: hasn318
9 Replies

2. Shell Programming and Scripting

Convert UTC time into current UNIX sever time zone

Hi guys thanks for the help for my previous posts.Now i have a requirement that i download a XMl file which has UTC time stamp.I need to convert UTC time into Unix server timezone. For ex if the time zone of unix server is CDT then i need to convert into CDT.whatever may be the system time... (5 Replies)
Discussion started by: mohanalakshmi
5 Replies

3. Shell Programming and Scripting

Shell script to convert epoch time to real time

Dear experts, I have an epoch time input file such as : - 1302451209564 1302483698948 1302485231072 1302490805383 1302519244700 1302492787481 1302505299145 1302506557022 1302532112140 1302501033105 1302511536485 1302512669550 I need the epoch time above to be converted into real... (4 Replies)
Discussion started by: aismann
4 Replies

4. Shell Programming and Scripting

Find and Convert UTC Time to PST Time

Hello All - I have a script that grabs data from the net and outputs the following data 46029 46.144 -124.510 2010 07 26 22 50 320 4.0 6.0 2.2 9 6.8 311 1012.1 -0.9 13.3 13.5 13.3 - - 46041 47.353 -124.731 2010 07 26 22 50 250 2.0 3.0 1.6 8 6.4 - 1011.6 - ... (0 Replies)
Discussion started by: drexnefex
0 Replies

5. Shell Programming and Scripting

Script to add time convert to seconds

Hi, What i am looking for and i am new to this too, is a bash script that will add time in the format hh:mm:ss and produce the answer in minutes or seconds. It needs to be a loop since there are hundreds of times in my file. This is data is from a CDR that calculates duration of time used. ... (2 Replies)
Discussion started by: trotella
2 Replies

6. Shell Programming and Scripting

Capturing the output of dbv

Hello, We have an oracle database running on a Linux host (RHEL5)...I'm trying to run Oracle dbv (database verify utility) and capture its output to a file using the following syntax but the standart output does NOT get redirected to the file... dbv blocksize=32768 ... (2 Replies)
Discussion started by: luft
2 Replies

7. Shell Programming and Scripting

how to convert date time to epoch time in solaris

Hi, Is there any easy way to convert date time(stored in shell variable ) to epoch time in solaris box? As +%s is working on linux but not on solaris, also -d option is not working. Any suggestion please? (6 Replies)
Discussion started by: anshuman0507
6 Replies

8. Shell Programming and Scripting

Convert Epoch Time to Standard Date and Time & Vice Versa

Hi guys, I know that this topic has been discuss numerous times, and I have search the net and this forum for it. However, non able to address the problem I faced so far. I am on Solaris Platform and unable to install additional packages like the GNU date and gawk to make use of their... (5 Replies)
Discussion started by: DrivesMeCrazy
5 Replies

9. Shell Programming and Scripting

Script to convert GMT to Asia/Hong Kong time

hi friends, this is my first time with this type of script so please pardon my ignorance. i have this script in which a piece of code needs to be added which can have the Asia/Hong kong time as well. system date and format is GMT so no problem with GMT and even EST is covered..i have ato add new... (5 Replies)
Discussion started by: xejatt
5 Replies

10. Shell Programming and Scripting

Convert from standard epoch time from a shell script?

Is there an easy method to do an on the fly conversion of a standard epoch time (seconds from 1970) to more readable date format? Does Unix have anything built in to do this? (4 Replies)
Discussion started by: LordJezo
4 Replies
Login or Register to Ask a Question