o
ai3M @ s6 d dl mZ d dlZd dlZd dlZddlmZ ddlmZm Z m
Z
ddl m Z m
Z
mZmZmZ ddlmZ ddlmZ dd lmZ dd
lmZ dd lmZ dd lm Z g d
Z e e Z dZ!dZ"e#de"Z$dZ%e#de%Z&dd Z'e(e'e$e(e'e$dZ)ee dZ*G dd deZ+G dd de+Z,dd Z-dS ) )absolute_importN )RecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPoolport_by_scheme)LocationValueError
MaxRetryErrorProxySchemeUnknownProxySchemeUnsupportedURLSchemeUnknown)six)urljoin)RequestMethods) connection_requires_http_tunnel)Retry) parse_url) PoolManager ProxyManagerproxy_from_url)key_file cert_file cert_reqsca_certs ssl_version ca_cert_dir ssl_context key_password)
key_schemekey_hostkey_port key_timeout key_retries
key_strict key_blockkey_source_address key_key_filekey_key_password
key_cert_file
key_cert_reqs key_ca_certskey_ssl_versionkey_ca_cert_dirkey_ssl_context key_maxsize key_headers
key__proxykey__proxy_headerskey__proxy_configkey_socket_optionskey__socks_optionskey_assert_hostnamekey_assert_fingerprintkey_server_hostnamePoolKey)r use_forwarding_for_https ProxyConfigc C s | }|d |d< |d |d< dD ]}||v r,|| dur,t|| ||< q|d}|dur>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
N
c K s8 t | | || _t|dd d| _t| _t | _d S )Nc S s | S N)close)prB rB rS s z&PoolManager.__init__..) dispose_func)r __init__connection_pool_kwr poolspool_classes_by_schemekey_fn_by_schemerC )self num_poolsr= r_ rB rB rS r^ s
zPoolManager.__init__c C s | S rY rB rc rB rB rS __enter__ s zPoolManager.__enter__c C s | dS )NF)clear)rc exc_typeexc_valexc_tbrB rB rS __exit__ s zPoolManager.__exit__c C sb | j | }|du r| j }dD ]}||d q|dkr(tD ]}||d q |||fi |S )a
Create a new :class:`urllib3.connectionpool.ConnectionPool` based on host, port, scheme, and
any additional pool keyword arguments.
If ``request_context`` is provided, it is provided as keyword arguments
to the pool class used. This method is used to actually create the
connection pools handed out by :meth:`connection_from_url` and
companion methods. It is intended to be overridden for customization.
N)r; r< portrV )ra r_ rC rK SSL_KEYWORDS)rc r; r< rl rN pool_clsrP kwrB rB rS _new_pool s
zPoolManager._new_poolc C s | j dS )z
Empty our store of pools and direct them all to close.
This will not affect in-flight connections, but they will not be
re-used after completion.
N)r` rg re rB rB rS rg s zPoolManager.clearrV c C sT |st d| |}|pd|d< |s t|d d}||d< ||d< | |S )a
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.
If ``port`` isn't given, it will be derived from the ``scheme`` using
``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
provided, it is merged with the instance's ``connection_pool_kw``
variable and used to create the new connection pool, if one is
needed.
zNo host specified.rV r; P rl r< )r _merge_pool_kwargsr rG rD connection_from_context)rc r< rl r; pool_kwargsrN rB rB rS connection_from_host s
z PoolManager.connection_from_hostc C s: |d }| j|}|st|||}| j||dS )z
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context.
``request_context`` must at least contain the ``scheme`` key and its
value must be a key in ``key_fn_by_scheme`` instance variable.
r; rN )rD rb rG r connection_from_pool_key)rc rN r; pool_key_constructorpool_keyrB rB rS rs s z#PoolManager.connection_from_contextc C s | j j4 | j |}|r|W d S |d }|d }|d }| j||||d}|| j |< W d |S 1 s;w Y |S )a
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the provided pool key.
``pool_key`` should be a namedtuple that only contains immutable
objects. At a minimum it must have the ``scheme``, ``host``, and
``port`` fields.
Nr; r< rl rv )r` lockrG rp )rc ry rN poolr; r< rl rB rB rS rw s
z$PoolManager.connection_from_pool_keyc C s t |}| j|j|j|j|dS )a
Similar to :func:`urllib3.connectionpool.connection_from_url`.
If ``pool_kwargs`` is not provided and a new pool needs to be
constructed, ``self.connection_pool_kw`` is used to initialize
the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs``
is provided, it is used instead. Note that if a new pool does not
need to be created for the request, the provided ``pool_kwargs`` are
not used.
)rl r; rt )r ru r< rl r; )rc urlrt urB rB rS connection_from_url s z PoolManager.connection_from_urlc C sR | j }|r'| D ]\}}|du r"z||= W q ty! Y q w |||< q |S )a
Merge a dictionary of override values for self.connection_pool_kw.
This does not modify self.connection_pool_kw and returns a new dict.
Any keys in the override dictionary with a value of ``None`` are
removed from the merged dictionary.
N)r_ rC rF KeyError)rc overridebase_pool_kwargsrP valuerB rB rS rr . s
z PoolManager._merge_pool_kwargsc C s" | j du rdS t| j | j|j S )z
Indicates if the proxy requires the complete destination URL in the
request. Normally this is only needed when not using an HTTP CONNECT
tunnel.
NF)proxyr proxy_configr; )rc
parsed_urlrB rB rS !_proxy_requires_url_absolute_formB s
z-PoolManager._proxy_requires_url_absolute_formc C sD | j du s |dkr dS | j jdkrdS tjr | jjs tddS dS )z
Validates that were not attempting to do TLS in TLS connections on
Python2 or with unsupported SSL implementations.
NrW zfContacting HTTPS destinations through HTTPS proxies 'via CONNECT tunnels' is not supported in Python 2)r r; r
PY2r r9 r )rc
url_schemerB rB rS $_validate_proxy_scheme_url_selectionO s z0PoolManager._validate_proxy_scheme_url_selectionTc K s t |}| |j | j|j|j|jd}d|d< d|d< d|vr(| j |d< | |r8|j ||fi |}n |j ||j
fi |}|oH| }|sM|S t ||}|j
dkrYd}|d} t| tsjtj| |d } | jr||stt|d }
|
D ]} | | jv r|d | d
q}z | j||||d } W n ty | jr| | Y S w | |d< ||d< t d || | | j ||fi |S )
aN
Same as :meth:`urllib3.HTTPConnectionPool.urlopen`
with custom cross-host redirect logic and only sends the request-uri
portion of the ``url``.
The given ``url`` parameter must be absolute, such that an appropriate
:class:`urllib3.connectionpool.ConnectionPool` can be chosen for it.
)rl r; Fassert_same_hostredirectr= i/ GETretries)r N)response_poolzRedirecting %s -> %s) r r r; ru r< rl r= rC r urlopen request_uriget_redirect_locationr statusrG
isinstancer from_intremove_headers_on_redirect is_same_hostrI r
iterkeysrD rK incrementr raise_on_redirect
drain_connloginfo) rc methodr| r ro r} connr redirect_locationr r= headerrB rB rS r ` sP
zPoolManager.urlopen)rX NrY NrV NT)__name__
__module__ __qualname____doc__r r r^ rf rk rp rg ru rs rw r~ rr r r r rB rB rB rS r s"
r c sR e Zd ZdZ d fdd Zd fdd Zdd
d Zd fd
d Z ZS )r a>
Behaves just like :class:`PoolManager`, but sends all requests through
the defined proxy, using the CONNECT method for HTTPS URLs.
:param proxy_url:
The URL of the proxy to be used.
:param proxy_headers:
A dictionary containing headers that will be sent to the proxy. In case
of HTTP they are being sent with each request, while in the
HTTPS/CONNECT case they are sent only once. Could be used for proxy
authentication.
:param proxy_ssl_context:
The proxy SSL context is used to establish the TLS connection to the
proxy when using HTTPS proxies.
:param use_forwarding_for_https:
(Defaults to False) If set to True will forward requests to the HTTPS
proxy to be made on behalf of the client instead of creating a TLS
tunnel via the CONNECT method. **Enabling this flag means that request
and response headers and content will be visible from the HTTPS proxy**
whereas tunneling keeps request and response headers and content
private. IP address, target hostname, SNI, and port are always visible
to an HTTPS proxy even when this flag is disabled.
Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
rX NFc
s t |trd|j|j|jf }t|}|jdvr t|j|js-t|jd} |j | d}|| _
|p3i | _ || _ t
||| _| j
|d< | j |d< | j|d< tt| j||fi | d S )Nz
%s://%s:%irU rq )rl _proxyr>
_proxy_config)r r r; r< rl r r
r rG _replacer
proxy_headersproxy_ssl_contextr: r superr r^ )
rc proxy_urlrd r= r r r9 r_ r rl __class__rB rS r^ s(
zProxyManager.__init__rV c sD |dkrt t| j||||dS t t| j| jj| jj| jj|dS )NrW )rt )r r ru r r< rl r; )rc r< rl r; rt r rB rS ru s
z!ProxyManager.connection_from_hostc C s0 ddi}t |j}|r||d< |r|| |S )z
Sets headers needed by proxies: specifically, the Accept and Host
headers. Only sets headers not provided by the user.
Acceptz*/*Host)r netlocupdate)rc r| r= headers_r rB rB rS _set_proxy_headers s
z ProxyManager._set_proxy_headersTc sV t |}t| j| j|js |d| j}| |||d< tt | j
||fd|i|S )z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.r= r ) r r r r r; rG r= r r r r )rc r r| r ro r} r= r rB rS r
s
zProxyManager.urlopen)rX NNNFr rY r ) r r r r r^ ru r r
__classcell__rB rB r rS r s *%
r c K s t dd| i|S )Nr rB )r )r| ro rB rB rS r s r ).
__future__r collections functoolslogging _collectionsr connectionpoolr r r
exceptionsr r r
r r packagesr
Z packages.six.moves.urllib.parser requestr Z
util.proxyr Z
util.retryr Zutil.urlr __all__ getLoggerr r rm _key_fields
namedtupler8 _proxy_config_fieldsr: rT partialrb ra r r r rB rB rB rS s<
6
s