Sponsored Content
Top Forums Shell Programming and Scripting jobs run parallel - server consumption? Post 302441544 by agama on Saturday 31st of July 2010 08:24:36 PM
Old 07-31-2010
The bottom line is that it depends on exactly what those 20 scripts are doing. If they are all starting a process which consumes a large amount of memory on the machine, or each is doing a large file transfer, then yes it is possible that you will impact the machine by running all 20 at the same time. On the other hand, these scripts might not significantly contribute to the load on the machine and running them all concurrently will be fine.

If you decide that they must all not be executed concurrently, then you could build a simplistic 'starter' into your driving script that starts the maximum that you deem safe to run concurrently, waits for those to finish, and starts more; this would continue until all scripts have been executed. If you need more sophisticated scheduling (dependencies and/or load average consideration), then you'll need to think about using a scheduler to manage the jobs.

This may be vague, but without knowing any details about the potential resource consumption of the scripts (CPU, memory, I/O), or the hardware (installed memory, number of cores, etc.), it's difficult to guess.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to run processes in parallel?

In a korn shell script, how can I run several processes in parallel at the same time? For example, I have 3 processes say p1, p2, p3 if I call them as p1.ksh p2.ksh p3.ksh they will run after one process finishes. But I want to run them in parallel and want to display "Process p1... (3 Replies)
Discussion started by: sbasak
3 Replies

2. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

3. Shell Programming and Scripting

background jobs exit status and limit the number of jobs to run

i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help. $cat run_job.ksh #!/usr/bin/ksh #################################### typeset -u SCHEMA_NAME=$1 ... (1 Reply)
Discussion started by: GrepMe
1 Replies

4. Shell Programming and Scripting

Run a command in parallel

Hi all, How do i run a command in parallel 50 times and capturing the result of each run in a separate file Eg: myApp arg1 > run1.txt myApp arg1 > run2.txt ::::::::::::::::::::::::: ::::::::::::::::::::::::: myApp arg1 > run50.txt The above way is sequential. ... (3 Replies)
Discussion started by: jakSun8
3 Replies

5. HP-UX

need to find the power consumption of a RX7640 server.

Hi All, Need to find the power consumption of a RX7640 server. Please help. Regards, Sag. (3 Replies)
Discussion started by: sag71155
3 Replies

6. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

7. Shell Programming and Scripting

Run in series and Parallel

I have a list with four dates say load_date.lst contains 2010-01-01 2010-01-31 2010-03-01 2010-03-31 2010-05-01 2010-05-31 2010-07-01 2010-07-31 And I have directory /lll/src/sql with set of sql's 1_load.sql 2_load.sql 3_load.sql I want to run the sql'in series with respective to... (3 Replies)
Discussion started by: sol_nov
3 Replies

8. Shell Programming and Scripting

Running jobs in parallel

I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
Discussion started by: gctex
17 Replies

9. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

10. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies
serialize(2)							System Calls Manual						      serialize(2)

NAME
serialize() - force target process to run serially with other processes SYNOPSIS
DESCRIPTION
The system call is used to force the target process referenced by the pid value passed in to run serially with other processes also marked for serialization. If the value of pid is zero, then the currently running process is marked for serialization. Once a process has been marked by the process stays marked until process completion, unless is reissued on the serialized process with timeshare set to 1. If timeshare is set to 1, the process specified in pid will be returned to normal timeshare scheduling algorithms. This call is used to improve process throughput since process throughput usually increases for large processes when they are executed seri- ally instead of allowing each program to run for only a short period of time. By running large processes one at a time, the system makes more efficient use of the CPU as well as system memory, since each process does not end up constantly faulting in its working set, to only have the pages stolen when another process starts running. As long as there is enough memory in the system, processes marked by behave no differently from other processes in the system. However, once memory becomes tight, processes marked by are run one at a time with the highest priority processes being run first. Each process runs for a finite interval of time before another serialized process is allowed to run. RETURN VALUE
returns zero upon successful completion, or nonzero if the system call failed. ERRORS
If fails, it sets (see errno(2)) to the following value: The pid passed in does not exist. WARNINGS
The user has no way of forcing an execution order on serialized processes. AUTHOR
was developed by HP. SEE ALSO
serialize(1), privileges(5). serialize(2)
All times are GMT -4. The time now is 05:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy