Multiprocessing Help Post: 302725611

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Multiprocessing Help Post 302725611 by Tyler_92 on Friday 2nd of November 2012 11:35:40 AM

11-02-2012

Registered User

Thanks everyone for the helpful responses! I was messing around some last night, I think where I will go from here is change my loop, which looks like this:

Code:

#Stage 1 Data Mode
#$runlist_size is number of files found in text file
#Analyze every other file starting with first
sub Stage1_DataMode
{
    for (my $i=0; $i<$runlist_size; $i=$i+2)
    {
        system "vaStage1 -Stage1_RunMode=data -config=${CONFIG}Stg1.config $VBFDIR$runlist[$i].cvbf ${OUTPUT}$runlist[$i].root";
    }
}

to this:

Code:

#Stage 1 Data Mode
#$runlist_size is number of files found in text file
#Analyze every other file starting with first
sub Stage1_DataMode
{
    for (my $i=0; $i<$runlist_size; $i=$i+2)
    {
        system "vaStage1 -Stage1_RunMode=data -config=${CONFIG}Stg1.config $VBFDIR$runlist[$i].cvbf ${OUTPUT}$runlist[$i].root &";
    }
}

Adding the "&" to send files to the background. Then I will do some testing to see how many files I should send through at once, and change the loop accordingly. I'll also do some experimenting with parallel, that looks like a nice tool.

Tyler_92

View Public Profile for Tyler_92

Find all posts by Tyler_92

4 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Multiprocessing under Linux

I'm writing C programs to be executed on a multi-processor UNIX (GNU/Linux, kernel 2.6.11) Do I need to add a special kind of code to somewhere or run a special utility to execute the program file to be executed by all processors? Or is it handled automatically by kernel?

2. Programming

Multiprocessing multipointers

I have a complex problem..... I have to search files on directory "text files" then search on all of them for a word or sentence....the user inter my problem is,,,, if I want to create a child for each file...and point a file by pointer to search...and I don't know how much files i have in...

3. Shell Programming and Scripting

Multiprocessing in Python

Hi there, I have a code that can take in any function with two arguements and do processing. However, I would like to implement a feature whether it can limit a number of process running concurrently so as not take up too much resources. I have tried researching for pool.map however I am unable...

4. Shell Programming and Scripting

<< Threading inside multiprocessing using queues >>

Hi All, I am trying to achieve threading inside each process of multiprocessing. I have 2 queues one for multiprocess (process) & another inside each process. when i execute it got hung after below output. My goal here is to go through p_source queue & for each process picks up all t_source...

LEARN ABOUT DEBIAN

rivet

RIVET(1)							   User Commands							  RIVET(1)

NAME

       rivet - Robust Independent Validation of Experiment and Theory

SYNOPSIS

       rivet [options] myfifo

OPTIONS

       --version
	      show program's version number and exit.

       -h, --help
	      show this help message and exit.

       -l NATIVE_LOG_STRS
	      set a log level in the Rivet library.

       -a ANA, --analysis=ANA
	      add an analysis to the processing list.

       --list-analyses
	      show the list of available analyses' names. With -v, it shows the descriptions, too.

       --list-used-analyses
	      list the analyses used by this command (after subtraction of inappropriate ones).

       --show-analysis=SHOW_ANALYSES, --show-analyses=SHOW_ANALYSES
	      show the details of an analysis.

       --analysis-path=PATH
	      specify the analysis search path (cf.  $RIVET_ANALYSIS_PATH).

       --analysis-path-append=PATH
	      append to the analysis search path (cf.  $RIVET_ANALYSIS_PATH).

       --pwd  append the current directory (pwd) to the analysis search path (cf. $RIVET_ANALYSIS_PATH).

       -H HISTOFILE, --histo-file=HISTOFILE
	      specify the output histo file path (default = Rivet.aida).

       -x XS, --cross-section=XS
	      specify the signal process cross-section in pb.

       -n NUM, --nevts=NUM
	      restrict the max number of events to read.

       --runname=NAME
	      give an optional run name, to be prepended as a 'top level directory' in histo paths.

       --ignore-beams
	      Ignore input event beams when checking analysis compatibility.

       --event-timeout=NSECS
	      max time in whole seconds to wait for an event to be generated from the specified source (default = 21600)

       --run-timeout=NSECS
	      max  time  in  whole  seconds  to  wait for the run to finish. This can be useful on batch systems such as the LCG Grid where tokens
	      expire on a fixed wall-clock and can render long Rivet runs unable to write out the final histogram file (default = unlimited).

       --histo-interval=HISTO_WRITE_INTERVAL
	      [experimental!] specify the number of events between histogram file updates. Default is to only write out at the	end  of  the  run.
	      Note  that intermediate histograms will be those from the analyze step only: analysis finalizing is currently not executed until the
	      end of the run.

       -v, --verbose
	      print debug (very verbose) messages.

       -q, --quiet
	      be very quiet.

EXAMPLES

	      mkfifo hepmc.fifo

	      my-generator -o myfifo &

	      rivet --analysis=ANALYSIS_NAME hepmc.fifo

ENVIRONMENT
:
       RIVET_ANALYSIS_PATH
	      list of paths to be searched for plugin analysis libraries at runtime.

       RIVET_REF_PATH
	      list of paths to be searched for reference data files.

       RIVET_INFO_PATH
	      list of paths to be searched for analysis metadata files.

AUTHOR

       This manual page was written by Lifeng Sun <lifongsun@gmail.com> for the Debian system (but may be used by others).

Rivet								     June 2012								  RIVET(1)