Sponsored Content
Full Discussion: Multiprocessing Help
Top Forums UNIX for Dummies Questions & Answers Multiprocessing Help Post 302725067 by drl on Thursday 1st of November 2012 04:13:57 PM
Old 11-01-2012
Hi.

Here is an example of GNU parallel running on an (old) OSX box. This call spawns as many processes as needed for each filename with a suffix of txt. There are also controls for CPUs as well as number of cores.
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate multiple processes simultaneously, with "GNU parallel".
# See: http://www.gnu.org/software/parallel
# See: http://freecode.com/projects/align

pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C parallel tree

pl " Structure of directories:"
tree d1 
tree d2

pl " Results of parallel processes:"
# ls d1/* d2/* |
find d1 d2 -name '*.txt' |
parallel --jobs 0 --ungroup 'echo -n job {#}, process $$", wc = "; wc {}' 2>/dev/null |
align

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = POSIX, LANG = POSIX
(Versions displayed with local utility "version")
OSX 10.3.9
GNU bash 2.05b.0
grep - ( /usr/bin/grep, 29 Aug 2008 )
parallel - ( local: ~/bin/parallel, 1 Nov 11:35 )
tree - ( local: RepRev 1.2, ~/bin/tree, 1 Nov 11:48 )

-----
 Structure of directories:
d1
`--a.txt
`--b.txt
`--binary-1.exe
`--c.txt
d2
`--frog-town.jpg
`--x.txt
`--y.txt

-----
 Results of parallel processes:
job 1, process 2620, wc =  4  16   70 d1/a.txt
job 2, process 2622, wc = 16  16  123 d1/b.txt
job 3, process 2624, wc = 26 265 1464 d1/c.txt
job 4, process 2626, wc =  4  16   70 d2/x.txt
job 5, process 2628, wc = 16  16  123 d2/y.txt

See URLs mentioned in comments if you do not have the items in your repository.

Best wishes ... cheers, drl
 

4 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Multiprocessing under Linux

I'm writing C programs to be executed on a multi-processor UNIX (GNU/Linux, kernel 2.6.11) Do I need to add a special kind of code to somewhere or run a special utility to execute the program file to be executed by all processors? Or is it handled automatically by kernel? (1 Reply)
Discussion started by: rayne
1 Replies

2. Programming

Multiprocessing multipointers

I have a complex problem..... I have to search files on directory "text files" then search on all of them for a word or sentence....the user inter my problem is,,,, if I want to create a child for each file...and point a file by pointer to search...and I don't know how much files i have in... (2 Replies)
Discussion started by: fwrlfo
2 Replies

3. Shell Programming and Scripting

Multiprocessing in Python

Hi there, I have a code that can take in any function with two arguements and do processing. However, I would like to implement a feature whether it can limit a number of process running concurrently so as not take up too much resources. I have tried researching for pool.map however I am unable... (1 Reply)
Discussion started by: alvinoo
1 Replies

4. Shell Programming and Scripting

<< Threading inside multiprocessing using queues >>

Hi All, I am trying to achieve threading inside each process of multiprocessing. I have 2 queues one for multiprocess (process) & another inside each process. when i execute it got hung after below output. My goal here is to go through p_source queue & for each process picks up all t_source... (0 Replies)
Discussion started by: kamauv234
0 Replies
Perl::Metrics::Simple(3pm)				User Contributed Perl Documentation				Perl::Metrics::Simple(3pm)

NAME
Perl::Metrics::Simple - Count packages, subs, lines, etc. of many files. SYNOPSIS
use Perl::Metrics::Simple; my $analyzer = Perl::Metrics::Simple->new; my $analysis = $analyzer->analyze_files(@paths, @refs_to_file_contents); $file_count = $analysis->file_count; $package_count = $analysis->package_count; $sub_count = $analysis->sub_count; $lines = $analysis->lines; $main_stats = $analysis->main_stats; $file_stats = $analysis->file_stats; VERSION
This is VERSION 0.12 DESCRIPTION
Perl::Metrics::Simple provides just enough methods to run static analysis of one or many Perl files and obtain a few metrics: packages, subroutines, lines of code, and an approximation of cyclomatic (mccabe) complexity for the subroutines and the "main" portion of the code. Perl::Metrics::Simple is far simpler than Perl::Metrics. Installs a script called countperl. USAGE
See the countperl script (included with this distribution) for a simple example of usage. CLASS METHODS
new Takes no arguments and returns a new Perl::Metrics::Simple object. is_perl_file Takes a path and returns true if the target is a Perl file. OBJECT METHODS
analyze_files( @paths, @refs_to_file_contents ) Takes an array of files and or directory paths, and/or SCALAR refs to file contents and returns an Perl::Metrics::Simple::Analysis object. find_files( @directories_and_files ) Uses list_perl_files to find all the readable Perl files and returns a reference to a (possibly empty) list of paths. list_perl_files Takes a list of one or more paths and returns an alphabetically sorted list of only the perl files. Uses is_perl_file so may throw an exception if a file is unreadable. is_perl_file($path) Takes a path to a file and returns true if the file appears to be a Perl file, otherwise returns false. If the file name does not match any of @Perl::Metrics::Simple::PERL_FILE_SUFFIXES then the file is opened for reading and the first line examined for a a Perl 'shebang' line. An exception is thrown if the file cannot be opened in this case. should_be_skipped($path) Returns true if the path should be skipped when looking for Perl files. Currently skips .svn, CVS, and _darcs directories. BUGS AND LIMITATIONS
See: http://rt.cpan.org/NoAuth/Bugs.html?Dist=Perl-Metrics-Simple SUPPORT
Via CPAN: Disussion Forum http://www.cpanforum.com/dist/Perl-Metrics-Simple Bug Reports http://rt.cpan.org/NoAuth/Bugs.html?Dist=Perl-Metrics-Simple AUTHOR
Matisse Enzer CPAN ID: MATISSE Eigenstate Consulting, LLC matisse@eigenstate.net http://www.eigenstate.net/ LICENSE AND COPYRIGHT
Copyright (c) 2006-2009 by Eigenstate Consulting, LLC. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
The countperl script included with this distribution. PPI Perl::Critic Perl::Metrics http://en.wikipedia.org/wiki/Cyclomatic_complexity perl v5.10.1 2010-05-13 Perl::Metrics::Simple(3pm)
All times are GMT -4. The time now is 03:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy