Memory Barriers for (Ubuntu) Linux (i686) Post: 302431048

Sponsored Content

Special Forums UNIX and Linux Applications High Performance Computing Memory Barriers for (Ubuntu) Linux (i686) Post 302431048 by Corona688 on Sunday 20th of June 2010 09:14:36 PM

06-20-2010

Registered User

Quote:

Originally Posted by gorga

Let me put it this way, I have a list of jobs to do, as I progress through the list I find some jobs aren't ready. I can either block/sleep etc and wait for the job to become ready, or I can move onto the next task and execute that one instead.

...and when you don't assume best-case, you could be going through the next few thousand not-ready tasks. Meanwhile every idle worker's doing the same thing. I begin to understand why you're concerned about contention for memory.

Quote:

If I build my system with locks, I'll adhere to the first approach, whereas what I'm trying to achieve (aided by atomic flags and memory barriers) is the second.

But because it doesn't block, you'll be wasting time scanning the list anyway, time that could have been spent doing actual work. And since your system's as busy idle as it is when actually busy you'll have a difficult time guessing how much. If I read you correctly, the jobs are all tiny. How tiny? How much more work is it to do a job than to scan the list? (And don't assume it could never, ever become mostly empty, that's the goal, not the proof.) If they're even in the same ballpark, you're going to be wasting a worrying proportion of CPU time scanning your list.

Anyway, the queue needn't block like you're describing. Put jobs in the queue when they become ready, don't just stick them there in advance, that way threads won't block when picking up jobs unless you're actually out of jobs -- in which case you want them to block. If the queue's big enough and jobs can be added fast enough, things can run smoothly. You can also do other things to streamline the queue -- hand out jobs 16 at a time instead of one at a time, switch between multiple queues, etc.

Last edited by Corona688; 06-20-2010 at 10:33 PM..

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

4 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Memory-waste in Ubuntu/Debian?

I have 512 mem on this laptop, though 'top' tells me I only have 380. However, Ubuntu is using 288 mb of memory, when I only have 3 terminals, running lynx, vim(for this file) and (of course) top. Considering it I have lynx running a 600 page txt file, which of course would eat some memory but 300?...

2. Linux

i686, x86 64, ppc

Hi, i am quite new to linux. I am interested in fedora linux distro. Fedora Project I dont know which one to choose, either i686, x86 64 or ppc. I prefer a live cd, coz its easy to use. And what is the difference between "Fedora Desktop Live Media" and "Fedora KDE Live Media".

3. Programming

Getting the total virtual memory for ubuntu in c++

Hi guys , i need to get the total virtual memory in ubuntu but i need to write a C++ code for that, any idea on how to go about doing it? any references? or website that i can refer to ?

4. Ubuntu

XP and Linux (Ubuntu) on same disk, Can I install Ubuntu on not-yet partitioned portion of disk?

My PC (Esprimo, 3 yeas old) has one hard drive having 2 partitions C: (80 GB NTFS, XP) and D: (120 GB NTFS, empty) and and a 200 MB area that yet is not-partitioned. I would like to try Ubuntu and to install Ubuntu on the not-partitioned area . The idea is to have the possibility to run...

LEARN ABOUT OSF1

queuedefs

queuedefs(4)						     Kernel Interfaces Manual						      queuedefs(4)

NAME

       queuedefs - Queue description file for at, batch, and cron commands

DESCRIPTION

       The  queuedefs  file describes the characteristics of the queues managed by cron or specifies other characteristics for cron.  Each noncom-
       ment line in this file describes either one queue or a cron characteristic.  Each uncommented line should be in one of the  following  for-
       mats.

	    q.[njobj][nicen][nwaitw] max_jobs=mjobs log=lcode

       The  fields  in these lines are as follows: The name of the queue.  Defined queues are as follows: The default queue for jobs started by at
       The default queue for jobs started by batch The default queue for jobs run from a crontab file

	      Queues d to z are also available for local use.  The maximum number of jobs that can be run simultaneously in  the  queue;  if  more
	      than  njob jobs are ready to run, only the first njob jobs will be run.  The others will be initiated as currently running jobs ter-
	      minate.  The nice(1) value to give to all jobs in the queue that are not run with a user ID of superuser.  The number of seconds	to
	      wait  before rescheduling a job that was deferred because more than njob jobs were running in that queue, or because the system-wide
	      limit of jobs executing (max_jobs) has been reached.  The maximum number of active jobs from all queues that  may  run  at  any  one
	      time.  The default is 25 jobs.  Logging level of messages sent to a log file.  The default is 4.	Defined levels are as follows:

	      level-code   level

		  0	   None

		  1	   Low

		  2	   Medium

		  3	   High

		  4	   Full

       Lines beginning with # are comments, and are ignored.

EXAMPLES

       The  following  file specifies that the b queue, for batch jobs, can have up to 50 jobs running simultaneously; that those jobs will be run
       with a nice value of 20.  If a job cannot be run because too many other jobs are running, cron will wait 60 seconds before trying again	to
       run it.	All other queues can have up to 100 jobs running simultaneously;  they will be run with a nice value of 2.  If a job cannot be run
       because too many other jobs are running cron will wait 60 seconds before trying again to run it.  b.50j20n60w

       The following file specifies that a total of 25 active jobs will be allowed by cron over all the queues at any one time, and cron will  log
       all messages to the log file.  The last two lines are comments that are ignored.  max_jobs=25 log=4 # This is a comment # And so is this

FILES

       Main cron directory The default location for the queue description file.

RELATED INFORMATION

       Commands: at(1), cron(8), crontab(1), nice(1)
       delim off

																      queuedefs(4)