U
.eau @ s d Z ddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl
Z
ddl Z ddl m Z ddl
mZ ddlmZ ddlmZ ddlmZmZmZmZmZmZmZmZm Z m Z m Z m Z m Z m!Z!m"Z" dd l#m$Z$ dd
l%m&Z& dd l'm(Z(m)Z)m*Z*m+Z+ d Z,e- Z.d
ddZ/ej0dkr2dd Z1dd Zdd Z2dd Z3dhddZ4d d Z5d d Z6d d! Z7d"d# Z8d$d% Z9d&d' Z:did(d)Z;d*d+ Zd0d1 Z?d2d3 Z@d4d5 ZAd6d7 ZBd8d9 ZCeDd:ZEd;d< ZFd=d> ZGd?d@ ZHdAdB ZIdCdD ZJdEdF ZKejLdGdH ZMdIdJ ZNdjdKdLZOdMdN ZPdkdPdQZQdRdS ZRdTdU ZSdVTdWZUeUdX ZVeUdY ZWdZd[ ZXd\d] ZYd^d_ ZZe[d`Z\e[daZ]dbdc Z^ddde Z_dfdg Z`dS )lz
requests.utils
~~~~~~~~~~~~~~
This module provides utility functions that are used within Requests
that are also useful for external consumption.
N __version__)certs)to_native_string)parse_http_list)quoteurlparsebytesstr OrderedDictunquote
getproxies proxy_bypass
urlunparse
basestring
integer_typesis_py3proxy_bypass_environmentgetproxies_environmentMappingcookiejar_from_dict)CaseInsensitiveDict)
InvalidURL
InvalidHeaderFileModeWarningUnrewindableBodyError)z.netrc_netrcP i )ZhttpZhttpsZwin32c C s zt rdd l}ndd l}W n tk
r0 Y dS X z6||jd}t||dd }||dd }W n tk
r~ Y dS X |r|sdS | d}|D ]T}|dkrd| kr d S |
dd
}|
d d }|
d
d}t || t j
r d S qdS )Nr Fz;Software\Microsoft\Windows\CurrentVersion\Internet SettingsZ ProxyEnableZ
ProxyOverride;z.Tz\.*z.*?)r winreg_winreg ImportErrorOpenKeyHKEY_CURRENT_USERintZ QueryValueExOSErrorsplitreplacerematchI)hostr$ ZinternetSettingsZ proxyEnableZ
proxyOverrideZtest r1 >/usr/lib/python3.8/site-packages/pip/_vendor/requests/utils.pyproxy_bypass_registry/ sF
r3 c C s t rt| S t| S dS )zReturn True, if the host should be bypassed.
Checks proxy settings gathered from the environment, if specified,
or the registry.
N)r r r3 )r0 r1 r1 r2 r V s r c C s t | dr| } | S )z/Returns an internal sequence dictionary update.items)hasattrr4 )dr1 r1 r2 dict_to_sequenceb s
r7 c C s4 d }d}t | dr t| }nbt | dr.| j}nPt | dr~z | }W n tjk
rZ Y n$X t|j}d| jkr~t
dt t | drz | }W n$ t
tfk
r |d k r|}Y n^X t | dr|d krz&| dd | }| |pd W n t
tfk
r d}Y nX |d kr&d}td|| S )
Nr __len__lenfilenoba% Requests has determined the content-length for this request using the binary size of the file: however, the file has been opened in text mode (i.e. without the 'b' flag in the mode). This may lead to an incorrect content-length. In Requests 3.0, support will be removed for files in text mode.tellseek )r5 r9 r: ioUnsupportedOperationosfstatst_sizemodewarningswarnr r< r* IOErrorr= max)oZ total_lengthZcurrent_positionr: r1 r1 r2 super_lenk sB
rJ Fc C s& zddl m }m} d}tD ]J}ztjd|}W n tk
rP Y W dS X tj|r |} qhq |dkrvW dS t | }d}t
| t r| d}|j
|d } z:||| }
|
r|
d rdnd} |
| |
d fW W S W n" |tfk
r | r Y nX W n ttfk
r Y nX dS ) z;Returns the Requests tuple auth for a given url from netrc.r )netrcNetrcParseErrorNz~/{} :asciir r> )rK rL NETRC_FILESrA path
expanduserformatKeyErrorexistsr
isinstancer decodenetlocr+ ZauthenticatorsrG r& AttributeError) urlZ raise_errorsrK rL Z
netrc_pathfZlocZriZsplitstrr0 r Zlogin_ir1 r1 r2 get_netrc_auth s8
r[ c C sB t | dd}|r>t|tr>|d dkr>|d dkr>tj|S dS )z0Tries to guess the filename of the given object.nameNr <>)getattrrU r rA rP basename)objr\ r1 r1 r2 guess_filename s
rc c C s t j| r| S t j| \}}|rPt j|sPt j|\}}d||g}q t|s^| S t|}|| krx| S t
}t jj|f|d }t j|s|j ||d}|S )zReplace nonexistent paths that look like they refer to a member of a zip
archive with the location of an extracted copy of the target, or else
just return the provided path unchanged.
/)rP ) rA rP rT r+ joinzipfileZ
is_zipfileZZipFileZnamelisttempfileZ
gettempdirextract)rP archivememberprefixZzip_fileZtmpZextracted_pathr1 r1 r2 extract_zipped_paths s
rl c C s. | dkr dS t | ttttfr&tdt| S )a Take an object and test to see if it can be represented as a
dictionary. Unless it can not be represented as such, return an
OrderedDict, e.g.,
::
>>> from_key_val_list([('key', 'val')])
OrderedDict([('key', 'val')])
>>> from_key_val_list('string')
ValueError: cannot encode objects that are not 2-tuples
>>> from_key_val_list({'key': 'val'})
OrderedDict([('key', 'val')])
:rtype: OrderedDict
N+cannot encode objects that are not 2-tuples)rU r r
boolr)
ValueErrorr valuer1 r1 r2 from_key_val_list s
rr c C s@ | dkr dS t | ttttfr&tdt | tr8| } t| S )a Take an object and test to see if it can be represented as a
dictionary. If it can be, return a list of tuples, e.g.,
::
>>> to_key_val_list([('key', 'val')])
[('key', 'val')]
>>> to_key_val_list({'key': 'val'})
[('key', 'val')]
>>> to_key_val_list('string')
ValueError: cannot encode objects that are not 2-tuples.
:rtype: list
Nrm ) rU r r
rn r) ro r r4 listrp r1 r1 r2 to_key_val_list s
rt c C sX g }t | D ]F}|dd |dd kr4dkrHn nt|dd }|| q |S )a Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
It basically works like :func:`parse_set_header` just that items
may appear multiple times and case sensitivity is preserved.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
To create a header from the :class:`list` again, use the
:func:`dump_header` function.
:param value: a string with a list header.
:return: :class:`list`
:rtype: list
Nr r^ ")_parse_list_headerunquote_header_valueappend)rq resultitemr1 r1 r2 parse_list_header8 s ( r{ c C sx i }t | D ]f}d|kr"d||< q |dd\}} | dd | dd krVdkrjn nt| dd } | ||< q |S )a^ Parse lists of key, value pairs as described by RFC 2068 Section 2 and
convert them into a python dict:
>>> d = parse_dict_header('foo="is a fish", bar="as well"')
>>> type(d) is dict
True
>>> sorted(d.items())
[('bar', 'as well'), ('foo', 'is a fish')]
If there is no value for a key it will be `None`:
>>> parse_dict_header('key_without_value')
{'key_without_value': None}
To create a header from the :class:`dict` again, use the
:func:`dump_header` function.
:param value: a string with a dict header.
:return: :class:`dict`
:rtype: dict
=Nr r^ ru )rv r+ rw )rq ry rz r\ r1 r1 r2 parse_dict_headerX s (
r} c C s\ | rX| d | d kr dkrXn n4| dd } |rD| dd dkrX| dd d dS | S )
zUnquotes a header value. (Reversal of :func:`quote_header_value`).
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
:rtype: str
r r^ ru r Nr> z\\\z\")r, )rq Z is_filenamer1 r1 r2 rw { s
$ rw c C s i }| D ]}|j ||j< q|S )zReturns a key/value dictionary from a CookieJar.
:param cj: CookieJar object to extract cookies from.
:rtype: dict
)rq r\ )cj cookie_dictZcookier1 r1 r2 dict_from_cookiejar s r c C s
t || S )zReturns a CookieJar from a key/value dictionary.
:param cj: CookieJar to insert cookies into.
:param cookie_dict: Dict of key/values to insert into CookieJar.
:rtype: CookieJar
r )r r r1 r1 r2 add_dict_to_cookiejar s r c C sT t dt tjdtjd}tjdtjd}td}|| || || S )zlReturns encodings from given content string.
:param content: bytestring to extract encodings from.
zIn requests 3.0, get_encodings_from_content will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)z!])flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])rE rF DeprecationWarningr- compiler/ findall)contentZ
charset_reZ pragma_reZxml_rer1 r1 r2 get_encodings_from_content s
r c
C s | d}|d |dd }}i }d}|D ]`}| }|r0|d }}|d} | dkr|d| |}|| d d |}||| < q0||fS ) zReturns content type and parameters from given header
:param header: string
:return: tuple containing content type and dictionary of
parameters
r r r Nz"' Tr| r^ )r+ stripfindlower)
headertokens content_typeparamsZ params_dictZitems_to_stripparamkeyrq Zindex_of_equalsr1 r1 r2 _parse_content_type_header s
r c C sD | d}|sdS t|\}}d|kr4|d dS d|kr@dS dS )z}Returns encodings from given HTTP Header Dict.
:param headers: dictionary to extract encoding from.
:rtype: str
z content-typeNcharsetz'"textz
ISO-8859-1)getr r )headersr r r1 r1 r2 get_encoding_from_headers s
r c c sj |j dkr | D ]
}|V qdS t|j dd}| D ]}||}|r4|V q4|jddd}|rf|V dS )zStream decodes a iterator.Nr, errors T)final)encodingcodecsgetincrementaldecoderrV )iteratorrrz decoderchunkrvr1 r1 r2 stream_decode_response_unicode s
r c c sH d}|dks|dkr t | }|t | k rD| ||| V ||7 }q dS )z Iterate over slices of a string.r N)r9 )stringZ slice_lengthposr1 r1 r2 iter_slices s r c C s~ t dt g }t| j}|rLzt| j|W S tk
rJ || Y nX zt| j|ddW S t k
rx | j Y S X dS )zReturns the requested content back in unicode.
:param r: Response object to get unicode content from.
Tried:
1. charset from content-type
2. fall back and replace all unicode characters
:rtype: str
zIn requests 3.0, get_unicode_from_response will be removed. For more information, please see the discussion on issue #2266. (This warning should only appear once.)r, r N)
rE rF r r r r r UnicodeErrorrx TypeError)r Ztried_encodingsr r1 r1 r2 get_unicode_from_response s
r zBABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~c C s | d}tdt|D ]}|| dd }t|dkr| rztt|d}W n tk
rr td| Y nX |tkr||| dd ||< qd|| ||< qd|| ||< qd |S ) zUn-escape any percent-escape sequences in a URI that are unreserved
characters. This leaves all reserved, illegal and non-ASCII bytes encoded.
:rtype: str
%r r r> z%Invalid percent-escape sequence: '%s'N )
r+ ranger9 isalnumchrr) ro r UNRESERVED_SETre )uripartsihcr1 r1 r2 unquote_unreserved5 s
r c C s@ d}d}zt t| |dW S tk
r: t | |d Y S X dS )zRe-quote the given URI.
This function passes the given URI through an unquote/quote cycle to
ensure that it is fully and consistently quoted.
:rtype: str
z!#$%&'()*+,/:;=?@[]~z!#$&'()*+,/:;=?@[]~)ZsafeN)r r r )r Zsafe_with_percentZsafe_without_percentr1 r1 r2 requote_uriM s r c C sl t dt| d }|d\}}t dttt|d }t dt|d |@ }||@ ||@ kS )zThis function allows you to check if an IP belongs to a network subnet
Example: returns True if ip = 192.168.1.1 and net = 192.168.1.0/24
returns False if ip = 192.168.1.1 and net = 192.168.100.0/24
:rtype: bool
z=Lr rd )structunpacksocket inet_atonr+ dotted_netmaskr) )ZipZnetZipaddrZnetaddrbitsZnetmaskZnetworkr1 r1 r2 address_in_networkc s
r c C s& ddd| > d A }t td|S )zConverts mask from /xx format to xxx.xxx.xxx.xxx
Example: if mask is 24 function returns 255.255.255.0
:rtype: str
l r z>I)r Z inet_ntoar pack)maskr r1 r1 r2 r r s r c C s, zt | W n t jk
r& Y dS X dS )z
:rtype: bool
FT)r r error)Z string_ipr1 r1 r2 is_ipv4_address} s
r c C s | ddkrzt| dd }W n tk
r: Y dS X |dk sL|dkrPdS zt| dd W q tjk
r Y dS X ndS dS )zV
Very simple check of the cidr format in no_proxy variable.
:rtype: bool
rd r Fr r T)countr) r+ ro r r r )Zstring_networkr r1 r1 r2
is_valid_cidr s
r c c sT |dk }|r"t j| }|t j| < z
dV W 5 |rN|dkrDt j| = n
|t j| < X dS )zSet the environment variable 'env_name' to 'value'
Save previous value, yield, and then restore the previous value stored in
the environment variable 'env_name'.
If 'value' is None, do nothingN)rA environr )Zenv_namerq Z
value_changed old_valuer1 r1 r2 set_environ s
r c
C s* dd }|}|dkr |d}t | }|jdkr2dS |rdd |dd d
D }t|jr|D ]0}t|rt|j|r dS q`|j|kr` dS q`nB|j}|jr|d |j7 }|D ] }|j |s| |r dS qt
d|6 zt |j}W n t t
jfk
r d }Y nX W 5 Q R X |r&dS d S )
zL
Returns whether we should bypass proxies or not.
:rtype: bool
c S s t j| pt j| S N)rA r r upper)kr1 r1 r2 r z'should_bypass_proxies..Nno_proxyTc s s | ]}|r|V qd S r r1 ).0r0 r1 r1 r2 s z(should_bypass_proxies.. r ,z:{}F)r hostnamer, r+ r r r ZportrR endswithr r r r Zgaierror) rY r Z get_proxyZ no_proxy_argparsedZproxy_ipZhost_with_portr0 Zbypassr1 r1 r2 should_bypass_proxies s@
r c C s t | |dri S t S dS )zA
Return a dict of environment proxies.
:rtype: dict
)r N)r r )rY r r1 r1 r2 get_environ_proxies s r c C st |pi }t | }|jdkr.||j|dS |jd |j |jd|j dg}d}|D ]}||krV|| } qpqV|S )zSelect a proxy for the url, if applicable.
:param url: The url being for the request
:param proxies: A dictionary of schemes or schemes and hosts to proxy URLs
Nallz://zall://)r r r scheme)rY ZproxiesZurlpartsZ
proxy_keysproxyZ proxy_keyr1 r1 r2 select_proxy s
r python-requestsc C s d| t f S )zO
Return a string representing the default user agent.
:rtype: str
z%s/%sr )r\ r1 r1 r2 default_user_agent s r c C s t t dddddS )z9
:rtype: requests.structures.CaseInsensitiveDict
z, )ZgzipZdeflatez*/*z
keep-alive)z
User-AgentzAccept-EncodingZAcceptZ
Connection)r r re r1 r1 r1 r2 default_headers" s r c
C s g }d}| |} | s|S td| D ]}z|dd\}}W n tk
r\ |d }}Y nX d| di}|dD ]F}z|d\}} W n tk
r Y qY nX | ||| |< qv|| q&|S ) zReturn a list of parsed link headers proxies.
i.e. Link: ; rel=front; type="image/jpeg",; rel=back;type="image/jpeg"
:rtype: list
z '"z, * '"r| )r r- r+ ro rx ) rq ZlinksZ
replace_charsvalrY r linkr r r1 r1 r2 parse_header_links. s&
r rN r> c C s | dd }|t jt jfkr dS |dd t jkr6dS |dd t jt jfkrRdS |t}|dkrhd S |dkr|ddd tkrd
S |d dd tkrd S |dkr|dd t krd
S |d d t krdS dS )z
:rtype: str
N zutf-32r z utf-8-sigr> zutf-16r zutf-8z utf-16-ber z utf-16-lez utf-32-bez utf-32-le)
r BOM_UTF32_LE BOM_UTF32_BEBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BEr _null_null2_null3)dataZsampleZ nullcountr1 r1 r2 guess_json_utfY s*
r c C s8 t | |\}}}}}}|s$|| }}t||||||fS )zGiven a URL that may or may not have a scheme, prepend the given scheme.
Does not replace a present scheme with the one provided as an argument.
:rtype: str
)r r )rY Z
new_schemer rW rP r queryfragmentr1 r1 r2 prepend_scheme_if_neededy s
r c C sB t | }zt|jt|jf}W n ttfk
r< d}Y nX |S )z{Given a url with authentication components, extract them into a tuple of
username,password.
:rtype: (str,str)
)r r )r r
ZusernameZpasswordrX r )rY r Zauthr1 r1 r2 get_auth_from_url s
r s ^\S[^\r\n]*$|^$z^\S[^\r\n]*$|^$c C sf | \}}t |trt}nt}z||s4td| W n* tk
r` td||t|f Y nX dS )zVerifies that header value is a string which doesn't contain
leading whitespace or return characters. This prevents unintended
header injection.
:param header: tuple, in the format (name, value).
z7Invalid return character or leading space in header: %sz>Value for header {%s: %s} must be of type str or bytes, not %sN)rU r
_CLEAN_HEADER_REGEX_BYTE_CLEAN_HEADER_REGEX_STRr. r r type)r r\ rq Zpatr1 r1 r2 check_header_validity s
r c C sF t | \}}}}}}|s"|| }}|ddd }t|||||dfS )zW
Given a url remove the fragment and the authentication part.
:rtype: str
@r r^ r )r rsplitr )rY r rW rP r r r r1 r1 r2
urldefragauth s
r c C s` t | jdd}|dk rTt| jtrTz|| j W q\ ttfk
rP tdY q\X ntddS )zfMove file pointer back to its recorded starting position
so it can be read again on redirect.
r= Nz;An error occurred when rewinding request body for redirect.z+Unable to rewind request body for redirect.)r` ZbodyrU Z_body_positionr rG r* r )Zprepared_requestZ body_seekr1 r1 r2 rewind_body s r )F)F)N)r )a__doc__r
contextlibr? rA r- r r sysrg rE rf r r r Z_internal_utilsr compatr rv r r r
r r r
r r r r r r r r r Zcookiesr Z
structuresr
exceptionsr r r r rO whereZDEFAULT_CA_BUNDLE_PATHZ
DEFAULT_PORTSplatformr3 r7 rJ r[ rc rl rr rt r{ r} rw r r r r r r r r frozensetr r r r r r r contextmanagerr r r r r r r encoder r r r r r r r r r r r r1 r1 r1 r2 s D
' =
3 #
%
=
&