Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Query on Running Multiple processes in background

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Query on Running Multiple processes in background
# 1  
Old 09-24-2010
Query on Running Multiple processes in background
HI All ,

Pardon me for asking some very basic questions,
I would be grateful if someone can help.

I am trying to execute a shell script which runs multiple processes in background. It includes various operations like copying , DB operations etc etc.

Now problem is that the complete script execution is taking a long long time, While we have optimized the DB part , I am on the lookout for other culprits.

Iam executing the script on T5K server . When executing multiple processes in background concurrently ....
1) Are all the processes allocated to a single CPU or multiple CPU ?
2) Can there be a possibility that a lot of time is wasted in scheduling the processes ?
# 2  
Old 09-24-2010
My crystal ball is still in for maintenance. Without actually seeing the script, we can't tell you why it's slow.
# 3  
Old 09-24-2010
Hi

I cannot post the entire script as it is used for official purpose. But would post a snippet.

Code:
MACHINE1=bip201


date
echo "Copying the data files to be imported . . ."

rcp ${MACHINE1}:/tmp/mi/format/a/a.icc /tmp/mi/import/SMFC/a/ &
rcp ${MACHINE1}:/tmp/mi/format/b.icc /tmp/mi/import/SMFC/b/ &
rcp ${MACHINE1}:/tmp/mi/format/c1/c1.icc /tmp/mi/import/SMFC/c1/ &
rcp ${MACHINE1}:/tmp/mi/format/c2/c2.icc /tmp/mi/import/SMFC/c2/ &
rcp ${MACHINE1}:/tmp/mi/format/c3/c3.icc /tmp/mi/import/SMFC/c3/ &
rcp ${MACHINE1}:/tmp/mi/format/c4/c4.icc /tmp/mi/import/SMFC/c4/ &
rcp ${MACHINE1}:/tmp/mi/format/d1/d1.icc /tmp/mi/import/SMFC/d1/ &
rcp ${MACHINE1}:/tmp/mi/format/d2/d2.icc /tmp/mi/import/SMFC/d2/ &
rcp ${MACHINE1}:/tmp/mi/format/d3/d3.icc /tmp/mi/import/SMFC/d3/ &
rcp ${MACHINE1}:/tmp/mi/format/d4/d4.icc /tmp/mi/import/SMFC/d4/ &


echo "Waiting for copy to complete"
wait
echo "************COPY COMPLETE*************************************************"
date

echo "************************ STARTING IMPORT ON SMFC *************************"
echo "Starting import of A & B"
cd /tmp/mi/import/SMFC/a/
perl importICCmig.pl -x SMFC a.cfg a.icc > import_log_a &
echo "'a' import started with PID = $!"

# 4  
Old 09-24-2010
What you havent said yet is what about the load of the server, is it swapping?
What else is running concurrently?
Ive seen cases where the server was doing almost nothing... because it was slowed down by the network (rcp...)
Now to gain performance, a cpio via network is faster than rcp but will give quite an overhead (CPU load...)

There is plenty to look at and you havent given much information (what OS, version how many CPU / RAM , swap NEtworks speed, are you on RAID disks, on SAN ...)
With what you gave so far no one can actually say what you are to expect...
# 5  
Old 09-24-2010
Thanks for the reply vbe !

But before I dwell further ,
I have some queries:

In case I run multiple processes from a script ,
1) Are all the processes allocated to a single CPU or multiple CPU ?
2) Can there be a possibility that a lot of time is wasted in scheduling the processes , allocated to single CPU?

3) In case of a fork() , are the child processes allocated to multiple CPUs for execution ? If not , can we do so ?
# 6  
Old 09-24-2010
Quote:
Originally Posted by gpta_varun
Thanks for the reply vbe !

But before I dwell further ,
I have some queries:

In case I run multiple processes from a script ,
1) Are all the processes allocated to a single CPU or multiple CPU ?
2) Can there be a possibility that a lot of time is wasted in scheduling the processes , allocated to single CPU?

3) In case of a fork() , are the child processes allocated to multiple CPUs for execution ? If not , can we do so ?
How should we know? We don't even know what OS is running on your server. This is important. Some have programmable restrictions in the number of CPU's it will let a user monopolize simultaneously.

Assuming there's not some sort of restrictions in place, multiple processes will run on multiple cores. But you know what they say about assumptions.

I don't see a lot of point running umpteen instances of RCP in parallel. The transfer's probably not CPU-bound, more likely being bottlenecked by disk or network, so more cores won't help. You'd have 9 transfers at X/9 MBPS instead of one transfer at X MBPS.
# 7  
Old 09-24-2010
Code:
You'd have 9 transfers at X/9 MBPS instead of one transfer at X MBPS

Unless you have a segmented high speed line like we do.... but the point is important we need the OS.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multi threading - running multiple processes at the same time

so i've been using this a lot in a lot of my scripts: ( columnA & columnAPID=$! & columnB & columnBPID=$! & columnC & columnCPID=$! &) & wait ${columnAPID} wait ${columnBPID} wait ${columnCPID} It seems to work as ive seen it dramatically reduce run time of my scripts. however, i'm... (5 Replies)
Discussion started by: SkySmart
5 Replies

2. UNIX for Advanced & Expert Users

List all background processes

How do I list the process in a Unix based system which are running in background? The following are options that I'm aware of, but they may not be appropiate. a. using ps -ef , and getting records of processes for which STATUS='S'(uninterruptible sleep) b. using jobs -l, and filtering... (5 Replies)
Discussion started by: kumarjt
5 Replies

3. Shell Programming and Scripting

Need help on background processes

Hi, I have a schell script parent.ksh from which I am calling three background processes a.ksh,b.ksh and c.ksh. Once these three processes completes the next step in parent.ksh should execute. How to achieve this? Please help me.... Thanks... (1 Reply)
Discussion started by: ravinunna
1 Replies

4. Shell Programming and Scripting

Background Processes

Ok guys so I have my first dummy shell almost done except for one tiny part: I do not know how to run a process in the background, from the code! I already know how to do that in a normal shell: $ program & However, no clue when it comes to how to program that thing. :eek: A very... (2 Replies)
Discussion started by: Across
2 Replies

5. Solaris

About running processes in background

Hi, I need to establish a procedure that will start an application in background each time my remote Solaris server is (re)started. This would be a kind of daemon. I am no sysadmin expert, so I am looking for pointers. How should I proceed? What are the main steps? Thanks, JVerstry (9 Replies)
Discussion started by: JVerstry
9 Replies

6. Shell Programming and Scripting

Keep a certain number of background processes running

I've got a bit of code I'm trying to work on... What i want to happen is ... at all times have four parallel mysql dump and imports running. I found the follow code snippet on the forum and modified it to work by starting four concurrent processes but it waits until all four are done before... (7 Replies)
Discussion started by: dgob123
7 Replies

7. UNIX for Dummies Questions & Answers

Running multiple processes in Linux

Hi guys, I want to run the multiple scripts at the same time using a ksh script. For example, I have three scripts to run: a.ksh, b.ksh and c.ksh How to start the above 3 scripts simultaneously and then on the completion of the above scripts I have other tasks to schedule. Thanks Gary (6 Replies)
Discussion started by: abcabc1103
6 Replies

8. Shell Programming and Scripting

Multiple SQLPLUS background processes not working properly

Hi All, I am running 25 background process from a Unix shell script which calls a single Oracle procedure with different paramenters each time. These 25 process creates 25 different files. When i run these 25 Background SQLPLUS processes, few files are not created completly but if i run 25... (1 Reply)
Discussion started by: rawat_me01
1 Replies

9. Shell Programming and Scripting

Running two processes in background

hi there, here's what i need in my korn-shell: ... begin korn-shell script ... nohup process_A.ksh ; nohup process_B.ksh & ... "other stuff" ... end lorn-shell script in plain english i want process A and process B to run in the background so that the script can continue doing... (6 Replies)
Discussion started by: jacob_gs
6 Replies

10. UNIX for Advanced & Expert Users

Background processes

How do you capture the return code from a background process? I am dumping data to a fifo and then processing it in a c program. I need to know that the sql finished successfully to ensure no missing data. Thanks. ex. sqlplus user/password < get_data.sql > data_fifo.txt & bin/process_data... (2 Replies)
Discussion started by: korndog
2 Replies
Login or Register to Ask a Question