Shell Programming and Scripting

Say you got a for loop where each execution has 0 dependence on the other. Thus ideally you'd like to executed them all concurrently rather than iteratively (if you had enough CPUs). We don't quite have that many CPUs but I would like to instead partition the iterations between them.

Or maybe even partition between CPU cores.

How to do either using bash commands? Thanks

You'd run the commands inside in the background, and each instance could run separately on one core. You'd have to be careful to limit it to the number of cores available.

Different commands in the same pipe chain can run concurrently with no further effort on your part, by the way.

Without seeing more details on what you want to run parallel, it's hard to give more details.

I'll rephrase. Say you have:

Code:

for VARIABLE in 1 2 3 4 5 .. N
do
    command1
    command2
    commandN
done

How to tell each iteration to execute commands 1, 2, and N on a CPU and CPU core of my choosing? Lets not worry about how to partition them yet.

IT would help if you gave your UNIX OS flavor, too. Output of

Code:

uname -a

You can do what you request on some different UNIXes but the commands are totally different.

It's Yellow Dog Linux, although I could conceivably run it from a Red Had Enterprise Linux machine as well.

That's decided by the operating system. You run n commands simultaneously and if there's enough CPU's, it will run them concurrently.

Code:

for X in 1 2 3 4 5
do
        command &
done

wait

I don't know if there's a standard utility to do so, but 'processor affinity mask' is what you're looking for if you really want to. But there's almost never any point -- the OS knows what CPU's are busy, and you don't.

Use the taskset command. You need some privilege to use it.

The syntax is:

Code:

for i in something
do
 taskset [cpumask]  command $i  &
 taskset [cpumask]  command2 $i  &
 taskset [cpumask]  command3 $i  &
wait
done

[cpumaskl] is a hexadecimal mask : 0x00000001 is processor #1
you may also need to redirect stdout and stderr for the taskset.... & line, depending on what the command is.
Try the man page for taskset.