*j S S/r SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK
J r SSK
J r J
r
SSKJr SrS rS
rS r\R( " 5 rS rS
r " S S\5 r " S S5 rS r " S S\5 r S#S jr S r " S S\ 5 r " S S \!5 r" " S S\!5 r#\#r$ " S S \#5 r% " S S \!5 r& " S S!\&5 r' " S" S\"5 r(g)$Pool
ThreadPool N )util) get_context TimeoutError)waitINITRUNCLOSE TERMINATEc $ [ [ U 6 5 $ N)listmapargss ;/opt/alt/python313/lib64/python3.13/multiprocessing/pool.pymapstarr / s T
c N [ [ R " U S U S 5 5 $ )Nr r )r itertoolsstarmapr s r starmapstarr 2 s" !!$q'473 44r c \ rS rSrS rS rSrg)RemoteTraceback9 c Xl g r tb)selfr s r __init__RemoteTraceback.__init__: s r c U R $ r r r! s r __str__RemoteTraceback.__str__< s wwr r N)__name__
__module__ __qualname____firstlineno__r" r& __static_attributes__ r r r r 9 s r r c \ rS rSrS rS rSrg)ExceptionWithTraceback? c [ R " [ U5 X5 nSR U5 nXl SU- U l g )N z
"""
%s""") tracebackformat_exceptiontypejoinexcr )r! r7 r s r r" ExceptionWithTraceback.__init__@ s6
'
'S 3
;
WWR[ 2%r c > [ U R U R 44$ r ) rebuild_excr7 r r% s r
__reduce__!ExceptionWithTraceback.__reduce__E s TXXtww ///r r7 r N)r( r) r* r+ r" r; r, r- r r r/ r/ ? s &
0r r/ c & [ U5 U l U $ r )r __cause__r= s r r: r: H s #B'CM Jr c 8 ^ \ rS rSrSrU 4S jrS rS rSrU =r $ )MaybeEncodingErrorP zRWraps possible unpickleable errors, so they can be
safely sent through the socket.c > [ U5 U l [ U5 U l [ [ U ] U R U R 5 g r )reprr7 valuesuperrA r" )r! r7 rE __class__s r r" MaybeEncodingError.__init__T s3 9%[
$04::Fr c @ SU R < SU R < S3$ )NzError sending result: 'z '. Reason: '')rE r7 r% s r r& MaybeEncodingError.__str__Y s =AZZ=AXXG Gr c @ SU R R < SU < S3$ )N)rG r( r% s r __repr__MaybeEncodingError.__repr__] s !^^ 4 4d;;r )r7 rE )
r( r) r* r+ __doc__r" r& rO r,
__classcell__rG s @r rA rA P s 'G
G< n[% UUS 5 n[ R " S U- 5 U" XSU445 S nANS nAff = f) Nr zMaxtasks {!r} is not valid_writerr z)worker got EOFError or OSError -- exitingz worker got sentinel -- exitingTFz0Possible encoding error while sending result: %sz worker exiting after %d tasks)
isinstanceintAssertionErrorformatputgethasattrrU close_readerEOFErrorOSErrorr debug Exception_helper_reraises_exceptionr/
__traceback__rA )inqueueoutqueue initializerinitargsmaxtaskswrap_exceptionrZ r[ completedtaskjobifuncr kwdsresultewrappeds r workerrt a s z(C'@'@+3q= 9 @ @ JKK
,,C
++Cw "" X I
xI,@ 5D
< JJ7 8 #' D D$ /$ /0F
, ! 4877s7V7d7TQ 7
xI,@8 JJ.:;3 '" JJB C $.H"H*1aoo>QZF ,(F1I6G JJI
%)* + + ,sB 0D*
E #
F *&EE
F!*FF
G "4GG c U e)z@Pickle-able helper function for use by _guarded_task_generation.r- )exs r rc rc s
Hr c @ ^ \ rS rSrSrSS.U 4S jjrU 4S jrSrU =r$ )
_PoolCache z
Class that implements a cache for the Pool class that will notify
the pool management threads every time the cache is emptied. The
notification is done by the use of a queue that is provided when
instantiating the cache.
Nnotifierc 2 > Xl [ TU ] " U0 UD6 g r )r{ rF r" )r! r{ r rp rG s r r" _PoolCache.__init__ s
$'$'r c j > [ TU ] U5 U ( d U R R S 5 g g r )rF __delitem__r{ rZ )r! itemrG s r r _PoolCache.__delitem__ s,
D! MM d # r ) r( r) r* r+ rQ r" r r, rR rS s @r rx rx s! +/ ( (
$
$r rx c \ rS rSrSrSr\S 5 r S*S jr\ R \ 4S jr S r
S
r\S 5 r\S 5 rS
r\S 5 r\S 5 rS rS rS0 4S jrS+S jrS+S jr S,S jrS rS-S jrS-S jr S0 SS4S jr S,S jr S,S jr \S+S j5 r \!S 5 r"\S 5 r#\S 5 r$\S 5 r%S! r&S" r'S# r(S$ r)\S% 5 r*\!S& 5 r+S' r,S( r-S)r.g).r zK
Class which supports an async version of applying functions to arguments.
Tc & U R " U0 UD6$ r Process)ctxr rp s r r Pool.Process s {{D)D))r Nr- c / U l [ U l U=( d
[ 5 U l U R 5 [ R " 5 U l U R R 5 U l [ U R S9U l X@l X l
X0l Uc [ R " 5 =( d SnUS: a [# S5 eUb& [% U[& 5 ( a US:: a [# S5 eUb [) U5 ( d [+ S5 eXl U R/ 5 U R9 5 n[: R< " [> R@ U R U R U R U RB U R, U R U RD U RF U R U R U R U RH XpR 4S9U l% SU RJ l& [N U RJ l U RJ RQ 5 [: R< " [> RR U R U RT U RF U R U R 4S9U l+ SU RV l& [N U RV l U RV RQ 5 [: R< " [> RX U RF U RZ U R 4S9U l. SU R\ l& [N U R\ l U R\ RQ 5 [^ R` " X Rb U R U RD U RF U R U R U RJ U RV U R\ U R 4 S S
9U l2 [N U l g ! [0 aW U R H" nUR2 b M UR5 5 M$ U R H nUR7 5 M e f = f) Nrz r z&Number of processes must be at least 1r z/maxtasksperchild must be a positive int or Nonez initializer must be a callabletargetr T )r exitpriority)3_poolr
_stater _ctx
_setup_queuesqueue SimpleQueue
_taskqueue_change_notifierrx _cache_maxtasksperchild _initializer _initargsosprocess_cpu_count
ValueErrorrV rW callable TypeError
_processes_repopulate_poolrb exitcode terminater6 _get_sentinels threadingThreadr _handle_workersr _inqueue _outqueue_wrap_exception_worker_handlerdaemonr start
_handle_tasks
_quick_put
_task_handler_handle_results
_quick_get_result_handlerr Finalize_terminate_pool
_terminate)r! processesrg rh maxtasksperchildcontextp sentinelss r r"
Pool.__init__ sI
,{} ++- !% 5 5 7 $*?*?@ !1 '! ,,.3!I q= EF F ' .448HA8M !RSS "8K+@+@ <= =# ! ! # '') ( / /''++t 4<<//4::t}}dnn##T^^T5K5K&& 3H3HJ '+ #&) # ""$ ' - -%%//4??DNN**dkk+
%)!$'! " ( / /''..$//4;;? '+ #&) # ""$ -- &&//4==$..$**'')=)=t?Q?Q&& 5 ] ZZ::%KKM ZZ
s O1 1$Q9Qc U R U:X a; U" SU < 3[ U S9 [ U SS 5 b U R R S 5 g g g )Nz&unclosed running multiprocessing pool )sourcer )r ResourceWarninggetattrr rZ )r! _warnr s r __del__ Pool.__del__
sT ;;# :4(C!$
0t /6B%%))$/ C r c
U R nSUR SUR SU R S[ U R
5 S3 $ )NrM .z state=z pool_size=rN )rG r) r* r lenr )r! clss r rO
Pool.__repr__ sQ nnCNN#1S%5%5$6 7
& _ -Q0 1r c j U R R /nU R R /n/ UQUQ$ r )r r^ r )r! task_queue_sentinelsself_notifier_sentinelss r r Pool._get_sentinels s< $ 6 6 7 #'#8#8#@#@"A @%@(?@@r c l U Vs/ s H" n[ US5 ( d M UR PM$ sn$ s snf )Nsentinel)r\ r )workersrt s r _get_worker_sentinelsPool._get_worker_sentinels s< 8 %+"6:6 8 8 8s 11c Sn[ [ [ U 5 5 5 HC nX nUR c M [ R
" SU- 5 UR
5 SnX ME U$ )zCleanup after any worker processes which have exited due to reaching
their specified lifetime. Returns True if any workers were cleaned up.
Fcleaning up worker %dT)reversedranger r r ra r6 )poolcleanedrn rt s r _join_exited_workersPool._join_exited_workers! s^
%D *+AWF*
2Q67
G , r c U R U R U R U R U R U R
U R U R U R U R U R 5
$ r ) _repopulate_pool_staticr r r r r r r r r r r% s r r Pool._repopulate_pool1 s_ ++DIIt||,0OO,0JJ
,0NNDbI I Js) A # A
AAA AA c U R 5 US:X aN [ U 5 nU R R U R UR
X5 UR 45 U$ US: a [ SR U5 5 e[ R XU5 n[ U 5 nU R R U R UR
[ U5 UR 45 S U 5 $ )z@
Equivalent of `map()` -- can be MUCH slower than `Pool.map()`.
r z Chunksize must be 1+, not {0:n}c 3 6 # U H o H o"v M M g 7fr r- .0chunkr s r Pool.imap.. ?fUDDf ) r IMapIteratorr rZ r _job _set_lengthr rY r
_get_tasksr r! ro r r rq task_batchess r imap Pool.imap s
>!$'F OO 11&++tN&&
M1} 5<<!#$ $ ??49EL!$'F OO 11&++292>@ &&
@f? ?r c U R 5 US:X aN [ U 5 nU R R U R UR
X5 UR 45 U$ US: a [ SR U5 5 e[ R XU5 n[ U 5 nU R R U R UR
[ U5 UR 45 S U 5 $ )z<
Like `imap()` method but ordering of results is arbitrary.
r z Chunksize must be 1+, not {0!r}c 3 6 # U H o H o"v M M g 7fr r- r s r r &Pool.imap_unordered.. r
r ) r IMapUnorderedIteratorr rZ r r r r rY r r r r s r imap_unorderedPool.imap_unordered s
>*40F OO 11&++tN&&
M1} 5<@ &&
@f? ?r c U R 5 [ XU5 nU R R UR SXU4/S45 U$ )z+
Asynchronous version of `apply()` method.
r N)r ApplyResultr rZ r )r! ro r rp r r rq s r r Pool.apply_async sH
T^< v{{At4 @ A4 HI
r c 2 U R X[ X4U5 $ )z)
Asynchronous version of `map()` method.
)r r r s r map_asyncPool.map_async s
tw r c U R 5 [ US5 ( d [ U5 nUc9 [ [ U5 [ U R
5 S- 5 u pGU( a US-
n[ U5 S:X a Sn[ R XU5 n[ X[ U5 UUS9n U R R U R U R UU5 S45 U $ )zI
Helper function to implement map, starmap and their async counterparts.
__len__N r r r )
r r\ r divmodr r r r MapResultr rZ r r )
r! ro r mapperr r r extrar rq s
r r Pool._map_async s
x++H~H %c(mS_q5H I IQ x=A ItyA 4CM8*8: --fkk.4.:<
r c [ XS9 UR 5 ( d( UR 5 UR 5 ( d M' g g )N)timeout)r emptyr[ )r change_notifierr- s r _wait_for_updatesPool._wait_for_updates s6 Y(!'')) ! "''))r c [ R " 5 nUR [ :X d U( a UR [ :w ap U R X4XVUXU
X5
/ U R
U5 QU
QnU R UU5 UR [ :X a MS U( a UR [ :w a Mp UR S 5 [ R " S5 g )Nzworker handler exiting) r current_threadr r r
r r r0 rZ r ra )r cache taskqueuer r r r re rf rg rh r rj r r/ threadcurrent_sentinelss r r Pool._handle_workers s
))+ mms"u )1K sYg 'h /
A !O#";";D"A NI N ! !"3_ E mms"u )1K
d
+,r c f [ R " 5 n[ U R S 5 H u pgS n U H8 nUR [
:w a [ R " S5 OK U" U5 M: U( a/ [ R " S5 U( a US OSn U" U S- 5 S =n=pjM S =n=pj O [ R " S5 [ R " S5 UR S 5 [ R " S 5 U H n U" S 5 M
[ R " S 5 g ! [ a@ n US S u p XJ R U SU 45 S n A GM/ ! [ a S n A GMA f = fS n A ff = f! S =n=pjf = f! [ a [ R " S
5 Nf = f) Nz'task handler found thread._state != RUN Fzdoing set_length()r r ztask handler got sentinelz/task handler sending sentinel to result handlerz(task handler sending sentinel to workersz/task handler got OSError when sending sentinelsztask handler exiting)
r r3 iterr[ r r r ra rb _setKeyErrorrZ r` )
r5 rZ rf r r4 r6 taskseq
set_lengthrl rr rm idxr s
r r Pool._handle_tasks s ))+#'
t#< GD
, #D }}+
#LM !D $ "
#78)-d1g2"37+ (,++w '+++w1 $=4
JJ2 3
J JJH I LL
JJA BD
)*= % !#'8! !JOOC% < < ' ! ! ! (,++w J JJH I Jsl 1F&D6.:FAF
6
F E;E% F%
E8 /E;0F7E8 8E;;F FF
F0/F0c 8 [ R " 5 n U" 5 nUR [ :w a [ R
" S5 O:Uc [ R
" S5 O Uu pVn X% R Xg5 S =n=pWMm U( ar UR [ :w a^ U" 5 nUc [ R
" S5 M> Uu pVn X% R Xg5 S =n=pWU( a UR [ :w a M^ [ U S5 ( aR [ R
" S5 [ S5 H+ nU R R 5 ( d O U" 5 M- [ R
" S[ U5 UR 5 g ! [ [ 4 a [ R
" S5 g f = f! [ a GNHf = f! [ [ 4 a [ R
" S5 g f = f! [ a GN'f = f! [ [ 4 a Nf = f) Nz.result handler got EOFError/OSError -- exitingz,result handler found thread._state=TERMINATEzresult handler got sentinelz&result handler ignoring extra sentinelr^ z"ensuring that outqueue is not full
z7result handler exiting: len(cache)=%s, thread._state=%s)r r3 r` r_ r ra r r r< r= r
r\ r r^ pollr )rf r[ r4 r6 rl rm rn objs r r Pool._handle_results= s ))+
u
}}#
IJ|
89 KCC
' $ #D #3+ .
2
u
|
CD KCC
' $ #D #3
2" 8Y ' ' JJ; <
rA#++0022 E #
L%j&-- )e X&
KL
X&
KL
X&
s_ F *F8 G G5 %.H H &F54F58
GG &G21G25
HHHHc # ~ # [ U5 n [ [ R " X5 5 nU( d g X4v M. 7fr )r; tupler islice)ro itsizer s r r Pool._get_tasksy s8
"Xi&&r01A)O s ;=c [ S5 e)Nz:pool objects cannot be passed between processes or pickled)NotImplementedErrorr% s r r; Pool.__reduce__ s !J r c [ R " S5 U R [ :X a< [ U l [ U R
l U R R S 5 g g )Nz closing pool)r ra r r r r r rZ r% s r r]
Pool.close sH
>" ;;# DK*/D ' ! ! % %d + r c f [ R " S5 [ U l U R 5 g )Nzterminating pool)r ra r
r r r% s r r Pool.terminate s
%& r c [ R " S5 U R [ :X a [ S5 eU R [
[ 4; a [ S5 eU R R 5 U R R 5 U R R 5 U R H nUR 5 M g )Nz joining poolzPool is still runningzIn unknown state) r ra r r r r r
r r6 r r r )r! r s r r6 Pool.join s
>" ;;# 45 5
[[ 2
2 /0 0 !!# ! !!#A
FFH r c [ R " S5 U R R 5 UR 5 ( a U R
R
5 ( ah U R
R 5 [ R " S5 UR 5 ( a" U R
R
5 ( a Mf g g g g )Nz7removing tasks from inqueue until task handler finishedr )
r ra _rlockacquireis_aliver^ rD r timesleep)re task_handlerrK s r _help_stuff_finishPool._help_stuff_finish s
LM ##%%'//*>*>*@*@ OO " JJqM ##%%'//*>*>*@*@%*@%r c
[ R " S5 [ Ul UR S 5 [ Ul [ R " S5 U R X'[
U5 5 UR 5 ( d [
U 5 S:w a [ S5 e[ Ul UR S 5 UR S 5 [ R " S5 [ R " 5 ULa UR 5 U( aR [ US S5 ( a> [ R " S5 U H" n
U
R b M U
R 5 M$ [ R " S5 [ R " 5 ULa UR 5 [ R " S 5 [ R " 5 ULa UR 5 U( a [ US S5 ( aj [ R " S
5 U HM n
U
R 5 ( d M [ R " S U
R - 5 U
R 5 MO g g g ) Nzfinalizing poolz&helping task handler/workers to finishr z/Cannot have cache with result_handler not alivezjoining worker handlerr zterminating workerszjoining task handlerzjoining result handlerzjoining pool workersr )r ra r
r rZ r] r rY rX r r3 r6 r\ r r pid) r r5 re rf r r/ worker_handlerr\ result_handlerr4 r s r r Pool._terminate_pool s
$%
!* D! '
;< wc$i@ ''))E
a AC
C !* D! T
+, # # %^ ; ! GDG[11 JJ, -::%KKM
)* # # %\ 9
+, # # %^ ; ! GDG[11 JJ- .::<<JJ 6 >?FFH 24r c & U R 5 U $ r )r r% s r __enter__Pool.__enter__ s r c $ U R 5 g r )r )r! exc_typeexc_valexc_tbs r __exit__
Pool.__exit__ s r )r r r r r r r r r r r r r r r r r r )NNr- NNr )NNN)r )/r( r) r* r+ rQ r staticmethodr r" warningswarnr r rO r r r r r r r r r r r r r r r r r! r r0 classmethodr r r r r; r] r r6 r] r re rk r, r- r r r r s O* * CE04Of %MMs 01 A
8 8
B ' '$ 9 96 1 " 8IM FJ 9 J@:@8 &(b4 BF KO : " "
- -( ++ ++Z 9) 9)v
,
3 3 jr c d \ rS rSrS rS rS rS
S jrS
S jrS r \
" \ R 5 r
S rg) r i c Xl [ R " 5 U l [ [
5 U l UR U l X l X0l X R U R ' g r )
r r Event_eventnext job_counterr r _callback_error_callback)r! r r r s r r" ApplyResult.__init__ sG
oo' % kk ! - !% DII r c 6 U R R 5 $ r )rt is_setr% s r readyApplyResult.ready s {{!!##r c x U R 5 ( d [ SR U 5 5 eU R $ )Nz{0!r} not ready)r| r rY _successr% s r
successfulApplyResult.successful s. zz|| . 5 5d ;< <}} r Nc : U R R U5 g r )rt r r! r- s r r ApplyResult.wait s !r c U R U5 U R 5 ( d [ eU R ( a U R $ U R er )r r| r r _valuer s r r[ ApplyResult.get s: 'zz|| ==;; ++ r c Uu U l U l U R ( a, U R ( a U R U R 5 U R ( a, U R ( d U R U R 5 U R R 5 U R U R S U l g r ) r r rw rx rt setr r r r! rn rE s r r< ApplyResult._set ss %("
t{ >>dmm NN4;; '
- KK "
r )r rw rx rt r r r r r )r( r) r* r+ r" r| r r r[ r< rp types GenericAlias__class_getitem__r, r- r r r r s3 &$
" $E$6$67r r c \ rS rSrS rS rSrg)r( i c
[ R XUUS9 SU l S /U- U l X l US:: a9 SU l U R R 5 U R U R g X2- [ X2- 5 - U l g )Nr& Tr ) r r" r r
_chunksize _number_leftrt r r r bool)r! r r lengthr r s r r" MapResult.__init__ sz T,: <
fvo # > !D KKOO DII& & 1D9K4L LD r c U =R S- sl Uu p4U( a U R ( a X@R XR - US- U R - & U R S:X ae U R ( a U R U R 5 U R
U R U R R 5 S U l g g U( d U R ( a
SU l X@l U R S:X ae U R ( a U R U R 5 U R
U R U R R 5 S U l g g )Nr r F) r r r r rw r r rt r r rx )r! rn success_resultsuccessrq s r r< MapResult._set) s Q ( t}}CIKK//)1Q3*? @ A%>>NN4;;/KK * ! !
& t}} %
$ A%''((5KK * ! !
&r )r r r r r N)r( r) r* r+ r" r< r, r- r r r( r( s
M"r r( c : \ rS rSrS rS rS S jr\rS rS r Sr
g)
r iE c D Xl [ R " [ R " 5 5 U l [ [ 5 U l UR U l [ R " 5 U l SU l S U l
0 U l X R U R ' g )Nr )r r ConditionLock_condru rv r r collectionsdeque_items_index_length _unsorted)r! r s r r" IMapIterator.__init__G sl
(( )9:
% kk !'') !% DII r c U $ r r- r% s r __iter__IMapIterator.__iter__R s r Nc U R U R R 5 nS S S 5 Wu p4U( a U$ Ue! [ a U R U R
:X a S U l [ S eU R R U5 U R R 5 n N! [ a0 U R U R
:X a S U l [ S e[ S ef = ff = f! , ( d f N= fr )
r r popleft
IndexErrorr r r
StopIterationr r )r! r- r r rE s r ru IMapIterator.nextU s
ZZ
1{{**, L !
1;;$,,.!%DJ'T1
(1 ;;..0D! 1{{dll2%)
+5&D0 1
1 Zs5 C+A A
C(B*(C+*:C$$C((C++
C9c U R U R U:X a U R R U5 U =R S-
sl U R U R ; aq U R R U R 5 nU R R U5 U =R S-
sl U R U R ; a Mq U R R
5 OX R U' U R U R :X a U R U R S U l
S S S 5 g ! , ( d f g = fNr ) r r r r r popnotifyr r r r r s r r< IMapIterator._setm s
ZZ{{a ""3' q kkT^^3 ..,,T[[9CKK&&s+KK1$K kkT^^3
!!#$'q!{{dll*KK * !
ZZs C
EA!E
Ec U R Xl U R U R :X a8 U R R 5 U R U R
S U l S S S 5 g ! , ( d f g = fr )r r r r r r r )r! r s r r IMapIterator._set_length~ sO
ZZ!L{{dll*
!!#KK * !
ZZs AA//
A=)r r r r r r r r r ) r( r) r* r+ r" r ru __next__r< r r, r- r r r r E s" &, H"""r r c \ rS rSrS rSrg)r i c T U R U R R U5 U =R S-
sl U R R 5 U R U R
:X a U R U R S U l S S S 5 g ! , ( d f g = fr ) r r r r r r r r r r s r r< IMapUnorderedIterator._set sj
ZZ KK s # KK1 K JJ {{dll*KK * !
ZZs BB
B')r N)r( r) r* r+ r< r, r- r r r r s "r r c d \ rS rSrSr\S 5 rS S jrS rS r \S 5 r
\S 5 r S
r S r
g)
r i Fc SSK Jn U" U0 UD6$ )Nr r )dummyr )r r rp r s r r ThreadPool.Process s "%%%r Nc 0 [ R XX#5 g r )r r" )r! r rg rh s r r" ThreadPool.__init__ s
d{=r c [ R " 5 U l [ R " 5 U l U R R U l U R R U l g r )r r r r rZ r r[ r r% s r r ThreadPool._setup_queues sD ))+
**, --++ ..,,r c 0 U R R /$ r )r r^ r% s r r ThreadPool._get_sentinels s %%--..r c / $ r r- )r s r r ThreadPool._get_worker_sentinels s r c U R SS9 M ! [ R a Of = f[ U5 H nU R S 5 M g )NF)block)r[ r Emptyr rZ )re r\ rK rn s r r] ThreadPool._help_stuff_finish sM % ( {{ tA KK s ++c 0 [ R " U5 g r )rZ r[ )r! r r/ r- s r r0 ThreadPool._wait_for_updates s
7r )r r r r )NNr- )r( r) r* r+ r rm r r" r r r r] r0 r, r- r r r r sU O& &>- / r )Nr- NF))__all__r r r r r rZ r3 r rn r2 r r r
connectionr r
r r r
countrv r r rb r r/ r: rA rt rc dictrx objectr r AsyncResultr( r r r r- r r r s <
' oo 5i 0 0< <" GK +