Multiprocessing in Python


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multiprocessing in Python
# 1  
Old 01-05-2018
Multiprocessing in Python

Hi there,

I have a code that can take in any function with two arguements and do processing. However, I would like to implement a feature whether it can limit a number of process running concurrently so as not take up too much resources. I have tried researching for pool.map however I am unable to find solution pertaining to this.

Tried searching for pool.map it shows something like pool (func, iterable, chunksize=NONE), but unsure how to replace it

Code:
def multProc(targetin, scanip, port):
    jobs = []
    #pool = multiprocessing.Pool(multiprocessing.cpu_count()) ##8 
    p = multiprocessing.Process(target=targetin, args=(scanip,port))
    jobs.append(p)
    p.start()
    return

# 2  
Old 01-06-2018
Check the multiprocessing.Pool class. The example in the docs give a good insight on how to implement.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Programming

Create a C source and compile inside Python 1.4.0 to 3.7.0 in Python for ALL? platforms...

Hi all... As you know I like making code backwards compatible for as many platforms as possible. This Python script was in fact dedicated for the AMIGA A1200 using Pythons 1.4.0, 1.5.2, 1.6.0, 2.0.1, and 2.4.6 as that is all we have for varying levels of upgrades from a HDD and 4MB FastRam... (1 Reply)
Discussion started by: wisecracker
1 Replies

2. Windows & DOS: Issues & Discussions

How to execute python script on remote with python way..?

Hi all, I am trying to run below python code for connecting remote windows machine from unix to run an python file exist on that remote windows machine.. Below is the code I am trying: #!/usr/bin/env python import wmi c = wmi.WMI("xxxxx", user="xxxx", password="xxxxxxx")... (1 Reply)
Discussion started by: onenessboy
1 Replies

3. Shell Programming and Scripting

<< Threading inside multiprocessing using queues >>

Hi All, I am trying to achieve threading inside each process of multiprocessing. I have 2 queues one for multiprocess (process) & another inside each process. when i execute it got hung after below output. My goal here is to go through p_source queue & for each process picks up all t_source... (0 Replies)
Discussion started by: kamauv234
0 Replies

4. Shell Programming and Scripting

**python** unable to read the background color in python

I am working on requirement on spreadsheet in python scripting. I have a spreadsheet containing cell values and with background color. I am able to read the value value but unable to get the background color of that particular cell. Actually my requirement is to read the cell value along... (1 Reply)
Discussion started by: giridhar276
1 Replies

5. SuSE

"ssh suse-server 'python -V' > python-version.out" not redirecting

Okay, so I have had this problem on openSUSE, and Debian systems now and I am hoping for a little help. I think it has something to do with Python but I couldn't find a proper Python area here. I am trying to redirect the output of "ssh suse-server 'python -V'" to a file. It seems that no matter... (3 Replies)
Discussion started by: Druonysus
3 Replies

6. Programming

Multiprocessing multipointers

I have a complex problem..... I have to search files on directory "text files" then search on all of them for a word or sentence....the user inter my problem is,,,, if I want to create a child for each file...and point a file by pointer to search...and I don't know how much files i have in... (2 Replies)
Discussion started by: fwrlfo
2 Replies

7. UNIX for Dummies Questions & Answers

Multiprocessing Help

Hello all, I recently wrote a simple script for the analysis jobs I do at work. I have to run multiple files through 5 different stages of an analysis program, the script simply runs all of the files through each stage automatically. My question is this: The computer I'm using has 12 cores, each... (8 Replies)
Discussion started by: Tyler_92
8 Replies

8. Programming

Python: bash-shell-like less functionality in the python shell

Hello, Is there some type of functional way to read things in the Python shell interpreter similar to less or more in the bash (and other) command line shells? Example: >>> import subprocess >>> help(subprocess) ... ... I'm hoping so as I hate scrolling and love how less works with... (0 Replies)
Discussion started by: Narnie
0 Replies

9. UNIX for Dummies Questions & Answers

Multiprocessing under Linux

I'm writing C programs to be executed on a multi-processor UNIX (GNU/Linux, kernel 2.6.11) Do I need to add a special kind of code to somewhere or run a special utility to execute the program file to be executed by all processors? Or is it handled automatically by kernel? (1 Reply)
Discussion started by: rayne
1 Replies
Login or Register to Ask a Question