Many processes running at the same time


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Many processes running at the same time
# 8  
Old 02-12-2018
Use a lockfile. In bash, one can do this:-
Code:
#!/bin/bash

# Use a lock file.  Lock released on exit unless earlier.
mylockfile="/var/lock/whatever"

[ ! -f $mylockfile ] && touch $mylockfile

exec 77>> $mylockfile       # Direct file descriptor 77 (or whatever) to the lock file

flock -w 30 -x 77           # Wait for up to 30 seconds to acquire the lock on fd 77

if [ $? -ne 0 ] 
then
   logger "$0 bypass due to lock"
   exit 99
else
   logger "$0 has the lock"
fi

# Carry on with the rest of the script.....



Does this help?
Robin
# 9  
Old 02-12-2018
Quote:
Originally Posted by beautymind
@RudiC , I don't know what is the problem because is it expected to finish in close to an hour.
That's why hicksd8 asked you to post the script you are using, including the environment it is executed in. As is, your waiting "until the job terminated in order to have only one process running" might wait for 41 (or even more) days.
# 10  
Old 02-12-2018
@apmcd47 Thanks for the help
@hicksd8 and @RudiC Unfortunately I can't post the script ( I am intern and I do not have the right to release the code)

I think that the script takes so long unlike in the past for the folowing reason :
The script is supposed to download files each hour from 12 sites . In the past there was only one site so the job was done in less than an hour but now with 12 sites it's no longer the case.

Last edited by beautymind; 02-12-2018 at 09:57 AM..
# 11  
Old 02-12-2018
How come you're having 1000 processs?
# 12  
Old 02-12-2018
@Rudic 1000 process running because there are many scripts each script calls an other ,
Now what is changed . we don't have only one site as before , we have 12 sites so it takes more time
# 13  
Old 02-13-2018
You have been asked to show us your script twice. If it needs to run over 800 processes to process one site, there is something seriously wrong. If you are unwilling to show us your script there isn't much we can do to help.

Please show us your script (in CODE tags) or close this thread.
# 14  
Old 02-13-2018
Unfortunately I can't post the script because it is strictly forbidden as I'm working on a confidential project.
My question is : how can I do so that the process takes less time ? knowing that the script load files on HDFS
Thanks in advance
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Processes running response time

Hi All I have been asked to write scripts within our monitoring tool for a vast requirement set. One of the requirements is below: • Lowest, Highest & Average response times of the Documentum process threads serving client requests Essentially they want a view where we can see the... (4 Replies)
Discussion started by: simpsa27
4 Replies

2. Shell Programming and Scripting

Multi threading - running multiple processes at the same time

so i've been using this a lot in a lot of my scripts: ( columnA & columnAPID=$! & columnB & columnBPID=$! & columnC & columnCPID=$! &) & wait ${columnAPID} wait ${columnBPID} wait ${columnCPID} It seems to work as ive seen it dramatically reduce run time of my scripts. however, i'm... (5 Replies)
Discussion started by: SkySmart
5 Replies

3. Linux

Running processes

Hi guys is it normal to have 5-10 cron/syslog processes running... in my case i got 10 cron process running. (4 Replies)
Discussion started by: batas
4 Replies

4. Solaris

Running processes on GZ/LZ

Hi guys just a question is it normal to see running process on a non-global zone in the global zone... processes such as cron. (3 Replies)
Discussion started by: batas
3 Replies

5. Shell Programming and Scripting

how to know the running processes.

Hi can anybody help me regarding this.. i want know the output of ps -ef with explanation. how can we know the running processess. this is the output of ps -elf F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 19 T root 0 0 0 0 SY ... (1 Reply)
Discussion started by: rajesh_pola
1 Replies

6. Shell Programming and Scripting

determine the active processes on the system which are running since long time

Hi , Please help me shell script to determine the active processes on the system which are running since long time (2 Replies)
Discussion started by: itian2010
2 Replies

7. Shell Programming and Scripting

Help in running two processes in parellel

I have a script something like this: #!/usr/bin/ksh CLASSPATH=/apps/opt/db2udb/admin/db2bdt/sqllib/java/db2java.zip:/apps/opt/db2udb/admin/db2bdt/sqllib/java/db2jcc.jar:/apps/opt/db2udb/admin/db2bdt/sqllib/function:$CLASSPATH export CLASSPATH ... (7 Replies)
Discussion started by: ss3944
7 Replies

8. Solaris

About running processes in background

Hi, I need to establish a procedure that will start an application in background each time my remote Solaris server is (re)started. This would be a kind of daemon. I am no sysadmin expert, so I am looking for pointers. How should I proceed? What are the main steps? Thanks, JVerstry (9 Replies)
Discussion started by: JVerstry
9 Replies

9. Shell Programming and Scripting

monitoring running processes

I have a script that runs continuously and will deliver a file to multiple servers via scp. On occasions one of the scp's will hang and as a result not complete in sending the remaining files and not loop around again. If I run the scp commands with a & they'll complete, but I want to make sure... (2 Replies)
Discussion started by: nhatch
2 Replies

10. Programming

parsing currently running processes

Hey guys, I'm writing a monitoring program that reads the pattern and the max and min number of instances of a process and then proceeds to parse the currently running processes for the pattern. I just want to know how I should go about this. I'll give you an idea of the flow of the program:... (7 Replies)
Discussion started by: blowtorch
7 Replies
Login or Register to Ask a Question