I have to search files on directory "text files"
then search on all of them for a word or sentence....the user inter
my problem is,,,, if I want to create a child for each file...and point a file by pointer to search...and I don't know how much files i have in directory ,,, so i don't know how much pointers should i create....
anyone have an idea how to create a number of pointers dynamicly with number of files or any other solution using C
First off there is a limit to the number of open files
Code:
ulimit -n
That is as many as you should try to open.
Secondly, systems limits the number of threads you can create. You should use threads, not processes for each file. The reason is that if your create lots of processes you can actually lock up the computer for a while in extreme cases.
Threads are limited, too. Example use of getconf to find your POSIX_THREAD maximum.
Code:
getconf -a | grep POSIX_THREAD_THREADS_MAX
_POSIX_THREAD_THREADS_MAX: 64
The parent program counts as one.
This User Gave Thanks to jim mcnamara For This Post:
Another thing to consider -- having 10,000 threads won't make your disk 10,000 times faster. You'd need enough RAM to keep all the file data cached to get benefit from multithreading.
Hi All,
I am trying to achieve threading inside each process of multiprocessing. I have 2 queues one for multiprocess (process) & another inside each process. when i execute it got hung after below output. My goal here is to go through p_source queue & for each process picks up all t_source... (0 Replies)
Hi there,
I have a code that can take in any function with two arguements and do processing. However, I would like to implement a feature whether it can limit a number of process running concurrently so as not take up too much resources. I have tried researching for pool.map however I am unable... (1 Reply)
Hello all,
I recently wrote a simple script for the analysis jobs I do at work. I have to run multiple files through 5 different stages of an analysis program, the script simply runs all of the files through each stage automatically. My question is this: The computer I'm using has 12 cores, each... (8 Replies)
I'm writing C programs to be executed on a multi-processor UNIX (GNU/Linux, kernel 2.6.11) Do I need to add a special kind of code to somewhere or run a special utility to execute the program file to be executed by all processors? Or is it handled automatically by kernel? (1 Reply)
12-17-2012
