o
ai|
@ sv d Z ddgZddlmZ ddlmZ ddlmZ ddlmZ ddl
m Z m Z m
Z
mZmZ ddlmZ eejej f Zed Zed
Zzdd lZW n eyS d ZY nw d
ZdZedede
e fddZ d de egef de e dede
e fddZ d de egef de e dede
e fddZ d de egef de e dede
e fdd Z ere Z Z d S eZ e Z d S ) ab Convenient parallelization of higher order functions.
This module provides two helper functions, with appropriate fallbacks on
Python 2 and on systems lacking support for synchronization mechanisms:
- map_multiprocess
- map_multithread
These helpers work like Python 3's map, with two differences:
- They don't guarantee the order of processing of
the elements of the iterable.
- The underlying process/thread pools chop the iterable into
a number of chunks, so that for very long iterables using
a large value for chunksize can make the job complete much faster
than using the default value of 1.
map_multiprocessmap_multithread )contextmanager)Poolpool)CallableIterableIteratorTypeVarUnion)DEFAULT_POOLSIZESTNTFi r returnc c sB z| V W | | | dS | | | w )z>Return a context manager making sure the pool closes properly.N)closejoin terminater r /builddir/build/BUILDROOT/alt-python310-pip-21.3.1-5.el8.x86_64/opt/alt/python310/lib/python3.10/site-packages/pip/_internal/utils/parallel.pyclosing. s
r funciterable chunksizec C s
t | |S )zMake an iterator applying func to each element in iterable.
This function is the sequential fallback either on Python 2
where Pool.imap* doesn't react to KeyboardInterrupt
or when sem_open is unavailable.
)map)r r r r r r
_map_fallback; s
r c C s< t t }|| ||W d S 1 sw Y dS )zChop iterable into chunks and submit them to a process pool.
For very long iterables using a large value for chunksize can make
the job complete much faster than using the default value of 1.
Return an unordered iterator of the results.
N)r ProcessPoolimap_unorderedr r r r r r r _map_multiprocessG s
$r c C s> t tt}|| ||W d S 1 sw Y dS )zChop iterable into chunks and submit them to a thread pool.
For very long iterables using a large value for chunksize can make
the job complete much faster than using the default value of 1.
Return an unordered iterator of the results.
N)r
ThreadPoolr
r r r r r _map_multithreadU s
$r" )r ) __doc____all__
contextlibr multiprocessingr r r Zmultiprocessing.dummyr! typingr r r
r r Z pip._vendor.requests.adaptersr
r r Zmultiprocessing.synchronize ImportErrorZ
LACK_SEM_OPENTIMEOUTr intr r r" r r r r r r sl