Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Complicated job distribution
Top Forums Shell Programming and Scripting Complicated job distribution Post 302387888 by amcrisan on Monday 18th of January 2010 06:29:30 PM
Old 01-18-2010
Complicated job distribution
Hello,

This is going to sound convoluted, however, this is how I need to do things.

I have three awesome machines and a couple hundred jobs. EACH of these jobs takes approximately 4 hours to complete. I want to give jobs to each of these machines, and their respective CPUs in an intelligent way. I basically would like to emulate a cluster, however, I cannot actually use a cluster (we don't want to hook up the machines to our already existing cluster, and don't also want to install a cluster on these machines...as this is really an emergency,hopefully limited one time, manner of usage).

Basically, what I would like to script is:
From one machine: assign jobs to the available CPUs on that machine, ALSO, from the one machine, assign jobs to the CPUs available on the other machines.

It needs to:
1) Apply no more than one job per CPU, and use only a total of 10 CPUs on each machine.
2) Not execute more jobs to a CPU until the job which is working away on it complete.

..as you can see, I basically need to emulate a cluster and it's queueing, deploying machinery...without installing a cluster. All of these machines are on the same network and share the same file system. Is this possible at all?

Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Very complicated script..

I have a script that I need to create tha involves moving files and renaming them(see previous post) Are there any websites with user made shell scripts? (5 Replies)
Discussion started by: rocinante
5 Replies

2. Shell Programming and Scripting

More complicated log parsing

Hey Guys, I am trying to grep within a file to find and output certain parts of lines to other file(s). The output files need to have a dynamic file name based on a field in the main log. The problem is that every line of the log is not the same, and often not even similar. To explain... (25 Replies)
Discussion started by: sjug
25 Replies

3. UNIX for Dummies Questions & Answers

complicated(?) grep

Hi all, I need help to figure out this grep. I would like to search all files in a directory for the occurrence of abc AND xyz. I started with this: grep -i abc * | grep xyz But this only returns lines which contain both values on the same line. So I am thinking that I need to do... (5 Replies)
Discussion started by: khearnen
5 Replies

4. Shell Programming and Scripting

Sort complicated two fields

Hi experts, I am trying sort command with my data but still not getting the expected results. For example, I have 5 fields data here c,18:12:45,c,c,c d,12:34:34,d,d,d a,13:50:10,a,a,a b,13:50:50,b,b,b a,13:50:50,a,a,a b,14:10:01,b,b,b c,10:12:45,c,c,c I want to get ... (3 Replies)
Discussion started by: lalelle
3 Replies

5. Shell Programming and Scripting

Seems Complicated but would b happy if its possible

Hey guys, I m here again with (may b) a silly query n sorry for that, I am new to UNIX n all so don't kno much @ it. Thro' a script I am generating an output (It is a TXT file) After generating it, I get it to windows n mails it by dragging n dropping it in lotus mail new mail window. Is there... (6 Replies)
Discussion started by: anushree.a
6 Replies

6. Shell Programming and Scripting

complicated search within file

Hi, I have following problem. I have a file with time stamps and some data describing what happened between time stamps. Something like this: 10:00 meeting with K meeting with L 11:00 lunch 12:00 work with K 13:00 From this file I have to get a file with... (7 Replies)
Discussion started by: mmike
7 Replies

7. Linux

Complicated Join command!

Hi I have a serious issue when trying to join to files so I have two files, one for meals and one for people Meal1:Turkey:Potato Chips:Twinkie:Coke:5.95 Meal2:Ham & Cheese:Doritos:Cookie:Sprite:6.49 Meal3:Vegetarian:Cheese Crackers:Brownie:Pepsi:5.75 Meal4:Tuna:Cheese Puffs:Eclair:Diet... (1 Reply)
Discussion started by: ehshi1992
1 Replies

8. Shell Programming and Scripting

complicated alias command

hi guys i m making one alias which will set variable , invoke sqlplus and also set prompt of sqlplus,,i have made successfully upto invoking sqlplus in unix but cant pass command in sqlplus here is the command alias sett='export ORACLE_SID=devdb2;sqlplus system/system@test' now this... (3 Replies)
Discussion started by: tapia
3 Replies

9. Solaris

Solaris? Is it really this complicated?

Hello Everybody, I am a mid-level Linux administrator trying to get a hold of Solaris. So far, my experience has been painful. I think it's confusing and it's just a big mess... To be more specific, I have noticed that whenever I install a package... it never followed a specific pattern.... (4 Replies)
Discussion started by: EssenceNY
4 Replies

10. Shell Programming and Scripting

Complicated string

I have a data file example 10302|77747373|442422442|290209|244242|"234|2352"|92482892 It has about 5000 rows the same way in that field. Needs to look like this.... I need to remove the quotes but the more difficult thing is to remove the pipe between the quotes because there is a pipe... (6 Replies)
Discussion started by: xgringo
6 Replies

LEARN ABOUT OPENSOLARIS

disown

disown(1)							   User Commands							 disown(1)

NAME

       disown - ksh93 built-in function to disassociate a job with the current shell

SYNOPSIS

       disown [job ...]

DESCRIPTION

       The  ksh93  disown command prevents the current shell from sending a HUP signal to each of the specified jobs when the current shell termi-
       nates a login session.

       If job is omitted, disown sends the HUP signal to the most recently started or stopped background job.

OPERANDS

       The following operands are supported:

       job    Specifies the job or jobs on which disown operates.

	      Specify job as one of the following:

	      number	  Refers to a process ID.

	      -number	  Refers to a process group ID.

	      %number	  Refers to a job number.

	      %string	  Refers to a job whose name begins with string.

	      %?string	  Refers to a job whose name contains string.

	      %+ or %%	  Refers to the current job.

	      %-	  Refers to the previous job.

EXIT STATUS

       0     Successful completion.

       >0    One or more specified jobs does not exist.

EXAMPLES

       Example 1 Disowning a Job

       The following example disowns job 1:

	 example% disown %1

AUTHORS

       David Korn, dgk@research.att.com

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWcsu			   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Uncommitted		   |
       +-----------------------------+-----------------------------+

SEE ALSO

       bg(1), jobs(1), ksh93(1), wait(1), attributes(5)

SunOS 5.11							    18 Apr 2007 							 disown(1)
All times are GMT -4. The time now is 03:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy