any additional ideas on how to make what i have in my original post more efficient?
i read somewhere that i can "pipe" the functions, but i haven't seen an example on how to do this.
I do not know what you mean by "pipe functions".
What i can suggest, though, is to make the parallelisation scheme more flexible, but if this is possible at all depends on the exact problem you are working on and how much parallelisation it allows for.
Suppose you have some "stream" of values you want to work on coming from somewhere, like in the following loop "VAL" is fed consecutive values:
To work on several values of VAL at the same time we first need to define a fanout value: it only makes sense to have so many parallel instances of some_command running at the same time. How many exactly depends on your system, the nature of some_command, the nature of your input data (the values of VAL) and perhaps a few other things but there will be *some* value whic is optimal. Let us call this value FANOUT. Here is what we are going to do, first in pseudocode:
here is a sketch for a solution in shell (ksh actually):
It should be feasible to pack this into a generic function and call that with the command to execute as parameters. Something like this (not tested):
Save for that i share Don Craguns astute observations: if you show us "something like" your problem you are likely to get "something like" an answer, most probably producing only "something like" the desired result instead of the real thing.
I've been doing some reading lately about threading (Posix threads) and I'm really curious about a couple things that I've read. I'm not sure if many people here have threading experience, but I thought it would be nice to be able to discuss some questions about it.
(For the record, I did... (1 Reply)
Hi,
I have written a code which will run a set of process using
fork.
I want to know from You how can i start another job when one of my job in my loop is completed
My code is
#include<stdio.h>
#include<ctype.h>
main() {
int pid,cid;
ChildProcess();
... (1 Reply)
Hi All,
Here's my question
I have a 385 MB file containing 5,000,000 records.
I need to read from the file and load into a table.
Initially i thought of doing it in a single thread (execution of a single program) but when calculated accounted 16 hours of time on a standard benchmark.
Hence... (5 Replies)
I am not sure if multi threading is the correct term, but here is what I am trying to do. I have a while loop that displays the number 1, pauses, displays the number 2, pauses , displays the number 3 ad infinitum. It just keeps counting.
While the screen displays the sequence of numbers counting... (4 Replies)
Hi, If we create 10 threads to invoke runQuery method at same time, Will queryProcessor will be overriden sometime or 10 different copies will be created?
We are not using any sunchronzation mechnism in runQuery(). so there is not gurantee on QueryProcessor class variables right OR each 10... (1 Reply)
In this piece i implemented the gossip method. The first thread is invoked from inside the (msg is first sent from node -1 to 0 from main()) and the other threads are invoked from inside of the thread function itself. I used two mutexes and a condition variable to control the synchronization. ... (4 Replies)
Hi, I am trying to get my head round Multi Threading and I have a few queries to try and clear up my confusion
Q1. Is multi threading a hardware / chip level concept, an OS level or an application level concept ? I am trying to work out where SMT architecture fits in.
Q2. What's the multi... (3 Replies)
Hello everybody ,
I launched cron to execute a task every hour but the job takes more than hour that's why I'm getting more than 1000 cron processes running at the same time !!!
My question is how to tell cron not to execute unless the job terminated in order to have only one process running .... (14 Replies)
Hi All
I have been asked to write scripts within our monitoring tool for a vast requirement set.
One of the requirements is below:
• Lowest, Highest & Average response times of the Documentum process threads serving client requests
Essentially they want a view where we can see the... (4 Replies)
Hi,
Can we apply multi threading in Unix. I am using bash shell.
We have a generic script to load the data to table based on file input. For each file there is an individual table to load.
For each file found in directory I want to load the data in parallel to target table using
... (3 Replies)