Sponsored Content
Top Forums Shell Programming and Scripting jobs run parallel - server consumption? Post 302441544 by agama on Saturday 31st of July 2010 08:24:36 PM
Old 07-31-2010
The bottom line is that it depends on exactly what those 20 scripts are doing. If they are all starting a process which consumes a large amount of memory on the machine, or each is doing a large file transfer, then yes it is possible that you will impact the machine by running all 20 at the same time. On the other hand, these scripts might not significantly contribute to the load on the machine and running them all concurrently will be fine.

If you decide that they must all not be executed concurrently, then you could build a simplistic 'starter' into your driving script that starts the maximum that you deem safe to run concurrently, waits for those to finish, and starts more; this would continue until all scripts have been executed. If you need more sophisticated scheduling (dependencies and/or load average consideration), then you'll need to think about using a scheduler to manage the jobs.

This may be vague, but without knowing any details about the potential resource consumption of the scripts (CPU, memory, I/O), or the hardware (installed memory, number of cores, etc.), it's difficult to guess.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to run processes in parallel?

In a korn shell script, how can I run several processes in parallel at the same time? For example, I have 3 processes say p1, p2, p3 if I call them as p1.ksh p2.ksh p3.ksh they will run after one process finishes. But I want to run them in parallel and want to display "Process p1... (3 Replies)
Discussion started by: sbasak
3 Replies

2. Shell Programming and Scripting

Diff: Server n parallel jobs

Hi, Could any one please explain the difference between DataStage server edition jobs and DS parallel extender jobs...? In which scenarios or application areas do we use either of these jobs.? Regards Suresh (0 Replies)
Discussion started by: sureshg_sampat
0 Replies

3. Shell Programming and Scripting

background jobs exit status and limit the number of jobs to run

i need to execute 5 jobs at a time in background and need to get the exit status of all the jobs i wrote small script below , i'm not sure this is right way to do it.any ideas please help. $cat run_job.ksh #!/usr/bin/ksh #################################### typeset -u SCHEMA_NAME=$1 ... (1 Reply)
Discussion started by: GrepMe
1 Replies

4. Shell Programming and Scripting

Run a command in parallel

Hi all, How do i run a command in parallel 50 times and capturing the result of each run in a separate file Eg: myApp arg1 > run1.txt myApp arg1 > run2.txt ::::::::::::::::::::::::: ::::::::::::::::::::::::: myApp arg1 > run50.txt The above way is sequential. ... (3 Replies)
Discussion started by: jakSun8
3 Replies

5. HP-UX

need to find the power consumption of a RX7640 server.

Hi All, Need to find the power consumption of a RX7640 server. Please help. Regards, Sag. (3 Replies)
Discussion started by: sag71155
3 Replies

6. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

7. Shell Programming and Scripting

Run in series and Parallel

I have a list with four dates say load_date.lst contains 2010-01-01 2010-01-31 2010-03-01 2010-03-31 2010-05-01 2010-05-31 2010-07-01 2010-07-31 And I have directory /lll/src/sql with set of sql's 1_load.sql 2_load.sql 3_load.sql I want to run the sql'in series with respective to... (3 Replies)
Discussion started by: sol_nov
3 Replies

8. Shell Programming and Scripting

Running jobs in parallel

I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
Discussion started by: gctex
17 Replies

9. Shell Programming and Scripting

waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Hello, I am running GNU bash, version 3.2.39(1)-release (x86_64-pc-linux-gnu). I have a specific question pertaining to waiting on jobs run in sub-shells, based on the max number of parallel processes I want to allow, and then wait... (1 Reply)
Discussion started by: srao
1 Replies

10. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies
PIDOF(8)						Linux System Administrator's Manual						  PIDOF(8)

NAME
pidof -- find the process ID of a running program. SYNOPSIS
pidof [-s] [-c] [-n] [-x] [-m] [-o omitpid[,omitpid..]] [-o omitpid[,omitpid..]..] program [program..] DESCRIPTION
Pidof finds the process id's (pids) of the named programs. It prints those id's on the standard output. This program is on some systems used in run-level change scripts, especially when the system has a System-V like rc structure. In that case these scripts are located in /etc/rc?.d, where ? is the runlevel. If the system has a start-stop-daemon (8) program that should be used instead. OPTIONS
-s Single shot - this instructs the program to only return one pid. -c Only return process ids that are running with the same root directory. This option is ignored for non-root users, as they will be unable to check the current root directory of processes they do not own. -n Avoid stat(2) system function call on all binaries which are located on network based file systems like NFS. Instead of using this option the the variable PIDOF_NETFS may be set and exported. -x Scripts too - this causes the program to also return process id's of shells running the named scripts. -o omitpid Tells pidof to omit processes with that process id. The special pid %PPID can be used to name the parent process of the pidof pro- gram, in other words the calling shell or shell script. -m When used with -o, will also omit any processes that have the same argv[0] and argv[1] as any explicitly omitted process ids. This can be used to avoid multiple shell scripts concurrently calling pidof returning each other's pids. EXIT STATUS
0 At least one program was found with the requested name. 1 No program was found with the requested name. NOTES
pidof is actually the same program as killall5; the program behaves according to the name under which it is called. When pidof is invoked with a full pathname to the program it should find the pid of, it is reasonably safe. Otherwise it is possible that it returns pids of running programs that happen to have the same name as the program you're after but are actually other programs. Note that that the executable name of running processes is calculated with readlink(2), so symbolic links to executables will also match. SEE ALSO
shutdown(8), init(8), halt(8), reboot(8), killall5(8) AUTHOR
Miquel van Smoorenburg, miquels@cistron.nl 01 Sep 1998 PIDOF(8)
All times are GMT -4. The time now is 03:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy