U f@sddlmZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddlmZddlmZddlmZdd lmZmZmZmZmZmZmZmZdd lmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+dd l,m-Z-dd l.m/Z/ddl0m1Z1ddl2m3Z3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z:m;Z;mZ>m?Z?ddl=m@ZAddl=mBZBddlCmDZDejErddlFZFddlGmHZHddlmIZImJZJeKeLZMejNeZdS)@HTTPConnectionPoola Thread-safe connection pool for one host. :param host: Host used for this HTTP Connection (e.g. "localhost"), passed into :class:`http.client.HTTPConnection`. :param port: Port used for this HTTP Connection (None is equivalent to 80), passed into :class:`http.client.HTTPConnection`. :param timeout: Socket timeout in seconds for each individual connection. This can be a float or integer, which sets the timeout for the HTTP request, or an instance of :class:`urllib3.util.Timeout` which gives you more fine-grained control over request timeouts. After the constructor has been parsed, this is always a `urllib3.util.Timeout` object. :param maxsize: Number of connections to save that can be reused. More than 1 is useful in multithreaded situations. If ``block`` is set to False, more connections will be created but they will not be saved once they've been used. :param block: If set to True, no more than ``maxsize`` connections will be used at a time. When no free connections are available, the call will block until a connection has been released. This is a useful side effect for particular multithreaded situations where one does not want to use more than maxsize connections per host to prevent flooding. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param retries: Retry configuration to use by default with requests in this pool. :param _proxy: Parsed proxy URL, should not be used directly, instead, see :class:`urllib3.ProxyManager` :param _proxy_headers: A dictionary with proxy headers, should not be used directly, instead, see :class:`urllib3.ProxyManager` :param \**conn_kw: Additional parameters are used to create fresh :class:`urllib3.connection.HTTPConnection`, :class:`urllib3.connection.HTTPSConnection` instances. httpz4type[BaseHTTPConnection] | type[BaseHTTPSConnection] ConnectionClsNrFr5r6_TYPE_TIMEOUT | Noneintbooltyping.Mapping[str, str] | NoneRetry | bool | int | None Url | NonezProxyConfig | None typing.Any) r8r9rmaxsizeblockheadersretries_proxy_proxy_headers _proxy_configconn_kwc Kst|||t||t|ts.t|}|dkrPool reached maximum size and no more connections are allowed.zLConnection pool is full, discarding connection: %s. Connection pool size: %s) rprsrrUFullrNrcrr|warningr8qsizerr@r@rA _put_conn(s.  zHTTPConnectionPool._put_conncCsdS)U Called right before a request is made, after the socket is created. Nr@rr@r@rA_validate_connSsz!HTTPConnectionPool._validate_conncCsdSrIr@rr@r@rA_prepare_proxyXsz!HTTPConnectionPool._prepare_proxy _TYPE_TIMEOUTr)cCs2|tkr|jSt|tr$|St|SdS)z:Helper that always returns a :class:`urllib3.util.Timeout`N)r'rclonermr)rn)r?rr@r@rA _get_timeout\s   zHTTPConnectionPool._get_timeoutz&BaseSSLError | OSError | SocketTimeout)errurl timeout_valuer:cCsNt|tr t||d|d|t|drJ|jtkrJt||d|d|dS)zAIs the error actually a timeout? Will raise a ReadTimeout or passRead timed out. (read timeout=rDerrnoN)rm SocketTimeoutrhasattrr_blocking_errnos)r?rrrr@r@rA_raise_timeouths    z!HTTPConnectionPool._raise_timeoutTz_TYPE_BODY | Nonez Retry | Noner ) rmethodrbodyrdrerchunked response_connpreload_contentdecode_contentenforce_content_lengthr:c  Cs&|jd7_||} | t| j|_zNz||Wn:tt fk rx}z|j |||jdW5d}~XYnXWn|t t t t ttfk r}zR|}t|t tfrt|}t|t t t tfr|r|jr|jst||jj}|W5d}~XYnXz|j|||||| | | dWnVtk r.YnBt k rn}z"|jtjkr^|jtjkr^W5d}~XYnX| j}|js|dkrt||d|d||_z |}Wn:t t fk r}z|j |||dW5d}~XYnX||_| |_||_ t!"d|j|j#|j$|||j%|j&|j' |S) a8 Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param response_conn: Set this to ``None`` if you will handle releasing the connection or set the connection to have the response release it. :param preload_content: If True, the response's body will be preloaded during construction. :param decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. :param enforce_content_length: Enforce content length checking. Body returned by server must match value of Content-Length header, if present. Otherwise, raise error. rrrrN)rrdrrrrrrrDz %s://%s:%s "%s %s HTTP/%s" %s %s)(rur start_connectr)Zresolve_default_timeoutr~rrrr rOSErrorrrr&rrmrkhas_connected_to_proxyrr4requestr r EPROTOTYPE ECONNRESET read_timeout is_closedr getresponsere _connection_poolr|r}r8r9versionstatuslength_remaining)r?rrrrrdrerrrrrr timeout_objenew_erresponser@r@rA _make_request{sN       z HTTPConnectionPool._make_requestcCs(|jdkrdS|jd}|_t|dSrP)rpry)r?old_poolr@r@rArN0s zHTTPConnectionPool.close)rr:cCs|drdSt|^}}}}}|p&d}|dk rr|8|YSX|8|.|t/:d||j1||||f||||| | | | | |d |S|S)a Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. If ``None`` (default) will retry 3 times, see ``Retry.DEFAULT``. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param bool preload_content: If True, the response's body will be preloaded into memory. :param bool decode_content: If True, will attempt to decode the body based on the 'content-encoding' header. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``preload_content`` which defaults to ``True``. :param bool chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. N)rdefaultrFTrr)rrrdrrerrrzConnection aborted.)errorr _stacktracez1Retrying (%r) after connection broken by '%r': %s)rrrrrrri/GET)rrzRedirecting %s -> %s) rerrrrrrrrrz Retry-Afterz Retry: %s)?r-r4rdrmr%from_intrerrrr.r+rr"rkrlcopyupdaterqr$rNrrrr~rrrr rrrrrrrrrr&rrrr incrementsysexc_infosleepr|rurlopenget_redirect_locationrrZ_prepare_for_method_changerraise_on_redirect drain_connsleep_for_retryr}r]ris_retryraise_on_status)r?rrrrdrerrrrrrrrrr parsed_urldestination_schemerrelease_this_connhttp_tunnel_requiredr clean_exitrrrrrredirect_locationhas_retry_afterr@r@rArRsq                         zHTTPConnectionPool.urlopen)N)rFrQrRrSr4r rZrTr'rBrrrrrrrrrNrrr@r@r@rArXxs^ 3 $=&+ (6 rXcseZdZUdZdZeZded<dedddddddddddddddddfdd d d d d ddd ddddddddddddddfdd Z dddddZ dddd Z d!ddfd"d# Z Z S)$HTTPSConnectionPoola Same as :class:`.HTTPConnectionPool`, but HTTPS. :class:`.HTTPSConnection` uses one of ``assert_fingerprint``, ``assert_hostname`` and ``host`` in this order to verify connections. If ``assert_hostname`` is False, no verification is done. The ``key_file``, ``cert_file``, ``cert_reqs``, ``ca_certs``, ``ca_cert_dir``, ``ssl_version``, ``key_password`` are only used if :mod:`ssl` is available and are fed into :meth:`urllib3.util.ssl_wrap_socket` to upgrade the connection socket into an SSL socket. httpsztype[BaseHTTPSConnection]rZNrFr5r6r[r\r]r^r_r`r3zint | str | Nonezssl.TLSVersion | Nonez"str | typing.Literal[False] | Nonerar7)r8r9rrbrcrdrerfrgkey_file cert_file cert_reqs key_passwordca_certs ssl_versionssl_minimum_versionssl_maximum_versionassert_hostnameassert_fingerprint ca_cert_dirrir:c sftj||||||||| f || |_| |_| |_| |_||_||_||_||_ ||_ ||_ ||_ dSrI) superrBrrrrrrrrrrr)r?r8r9rrbrcrdrerfrgrrrrrrrrrrrri __class__r@rArBs0  zHTTPSConnectionPool.__init__rrcCs@|jr|jjdkrd}nd}|j||j|j|jd|dS)z5Establishes a tunnel connection through HTTP CONNECT.rrY)r4r8r9rdN)rkr4 set_tunnelr>r9rqconnect)r?rZ tunnel_schemer@r@rArsz"HTTPSConnectionPool._prepare_proxyr1rCcCs|jd7_td|j|j|jp$d|jr8|jtkr@td|j}|j}|jdk rr|jjdk rr|jj}|jj}|jf|||j j |j |j |j |j|j|j|j|j|j|j|jd|jS)zL Return a fresh :class:`urllib3.connection.HTTPConnection`. rz)Starting new HTTPS connection (%d): %s:%s443zCCan't connect to HTTPS URL because the SSL module is not available.N)r8r9rrrrrrrrrrrr)rtr|r}r8r9rZr ImportErrorrkrr~rrrrrrrrrrrri)r? actual_host actual_portr@r@rArsDzHTTPSConnectionPool._new_connr0cs@t||jr||js<|js1)rr5ra)rkwr:cKs^t|^}}}}}|pd}|p(t|d}|dkrFt|fd|i|St|fd|i|SdS)a Given a url, return an :class:`.ConnectionPool` instance of its host. This is a shortcut for not having to parse out the scheme, host, and port of the url before creating an :class:`.ConnectionPool` instance. :param url: Absolute URL string that must include the scheme. Port is optional. :param \**kw: Passes additional parameters to the constructor of the appropriate :class:`.ConnectionPool`. Useful for specifying things like timeout, maxsize, headers, etc. Example:: >>> conn = connection_from_url('http://google.com/') >>> r = conn.request('GET', '/') rYPrr9N)r-rrrrX)rrr4rzr8r9r@r@rAconnection_from_urlVs rr7r3)r8r4r:cCsdSrIr@r8r4r@r@rAr,ssr,cCsdSrIr@rr@r@rAr,xscCs2t||}|r.|dr.|dr.|dd}|S)z? Normalize hosts for comparisons and use with sockets. []r)r<rendswithrr@r@rAr,}s  z(HTTPConnectionPool | HTTPSConnectionPool)rppathr:cCst|j|j|j|djS)zZReturns the URL from a given connection pool. This is mainly used for testing and logging.)r4r8r9r)r*r4r8r9r)rprr@r@rA_url_from_poolsrzqueue.LifoQueue[typing.Any])rpr:cCs:z|jdd}|r|qWntjk r4YnXdS)z2Drains a queue of connections and closes each one.FrN)rrNrUr)rprr@r@rArys  ry)N)[ __future__rrloggingrUrtypingrrwsocketrrtypesrZ_base_connectionr _collectionsrZ_request_methodsr connectionr r r r rrrrr exceptionsrrrrrrrrrrrrrrr Zutil.connectionr!Z util.proxyr"Z util.requestr#r$Z util.retryr%Zutil.ssl_match_hostnamer&Z util.timeoutr'r(r)Zutil.urlr*r+r,r<r-Z util.utilr. TYPE_CHECKINGsslZtyping_extensionsr/r0r1 getLoggerrFr|Unionfloatrr2EAGAIN EWOULDBLOCKrrXrroverloadrryr@r@r@rAsh      ( <          2 S