LSF Job Routing Algorithm


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users LSF Job Routing Algorithm
# 1  
Old 03-19-2013
LSF Job Routing Algorithm

I have a requirement where i need to restrict users into reusing the first 3 exec machine they are using so as to reduce license usage of our tools.

So, lets say that at any instant an user is using 3 machine - mac1,mac2,mac3. Now if he is going to run a new job, he is expected to re-use one of the three machine already in use.

Up to this point , I hope my requirement is clear. Please prompt me if you have any doubt

Now, I aim to design an algorithm which decides which is the best machine to use. I need following input to design the algo:-

1. Among mac1,2,3 , what parameters can be used to decide what machine is best to exec a job.

There are few other requirements as well, but i would like to try and fail before posting them here. The one posted above is the biggest bottleneck i have

---------- Post updated at 04:42 PM ---------- Previous update was at 04:41 PM ----------

I went through lsload command, but frankly I need some sort of guidelines to decide what load factors/user logins/memory/etc signals health of a exec machine.

Last edited by jim mcnamara; 03-19-2013 at 09:19 AM..
# 2  
Old 03-19-2013
Assume you know the load profile your jobs generate e.g., low cpu, lots of I/O, some network traffic.

In this case I/O would be the determining factor. When you have no clue what some job will consume you try to balance cpu usage. Sometimes load balancing larger jobs means waiting for resources. If you know that job is a hog ahead of time.

The reason load balancing network site requests is usually attainable without too many issues -- they mostly have similar profiles, some db access, a small amount of I/O.

Since you are asking an extremely general question, you got an extremely general answer. What you are doing is playing "OS" in a sense - you are doing what the OS does - try to schedule jobs. In such a way that "everybody is happy". There are round-robin schedulers, fair share schedulers - as examples. google for each of those to see what is entailed.-----
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Passing a variable to #BSUB -n in a LSF file

Hi There! I'm writing this LSF script to make it easier to send jobs to a cluster when varying certain parameters. At one point I'd like to do something like: set NPROC = 10 and later on call BSUB using something like: #BSUB -n $NPROC unfortunately for me, this throws an error: ... (4 Replies)
Discussion started by: lrayo
4 Replies

2. Programming

Please help me to develop algorithm

Hi guys , in my study book from which I re-learn C is task to generate all possible characters combination from numbers entered by the user. I know this algorithm must use combinatorics to calculate all permutations. Problem is how to implement algortihm. // This program reads the four numbers... (0 Replies)
Discussion started by: solaris_user
0 Replies

3. Shell Programming and Scripting

autosys job configuration for job failure.

We need to configure autosys that when a job fails continously for 3 times, we need to call another job. Is this possible in Autosys, or can anyone advice on the alternative. (2 Replies)
Discussion started by: sangea
2 Replies

4. Shell Programming and Scripting

Script to Start a Job after finding the Old job completed

Hi Experts, I need a script advice to schedule 12 jobs ( SAS Codes execute back ground ). Algorithem: 1. Script checks first job. 2. Finds first job is done; invoke second job. 3. finds second job is done; invoke third job. .. Request you to please assist. (3 Replies)
Discussion started by: Jerald Nathan
3 Replies

5. UNIX and Linux Applications

LSF Server: could not load license

Hi All, Please help me to solve the problem when i submit a job on the client: #bsub -q normal -(blah blah....) I got the error messages: #batch system daemon not responding ... still trying #batch system daemon not responding ... still trying #batch system daemon not responding ... still... (0 Replies)
Discussion started by: teppyvn
0 Replies

6. UNIX for Dummies Questions & Answers

How to insert child job under a box job?

I have this box job and it contains only one job under it which is to load a file. I want to insert a "File Watcher", "Copy File" to it? Have no clue how to do that...any help plzzz... (4 Replies)
Discussion started by: xejatt
4 Replies

7. Shell Programming and Scripting

algorithm

PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 21444 tomusr 213M 61M sleep 29 10 1:20:46 0.1% java/43 21249 root 93M 44M sleep 29 10 1:07:19 0.2% java/56 is there anyway i can use a command to get the total of the SIZE? 306M (Derive from... (5 Replies)
Discussion started by: filthymonk
5 Replies

8. Solaris

killing a unix job after the job process gets completed

Hi, Thanks in advance. i need to kill a unix background running job after that job process completes. i can kill a job by giving the following unix command kill -9 processid how to kill the job after the current process run gets completed ? Appreciate your valuable help. Thanks... (7 Replies)
Discussion started by: dtazv
7 Replies

9. Shell Programming and Scripting

killing unix job after the job process completes

Hi, Thanks in advance. i need to kill a unix background running job after that job process completes. i can kill a job by giving the following unix command kill -9 processid how to kill the job after the current process run gets completed ? Appreciate your valuable help. ... (1 Reply)
Discussion started by: dtazv
1 Replies
Login or Register to Ask a Question