ReN @ s d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l m Z d d l m
Z
d d l m Z d d l m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z d d l m Z m Z d d l m Z d d
l! m" Z" e f Z# dS Z$ e
j% Z& d
d Z' d d Z( d d Z) d d Z* d d Z+ d d Z, d d Z- d d Z. d d d Z/ d d! Z0 d" d# Z1 d$ d% Z2 d& d' Z3 d( d) Z4 d* d+ Z5 d, d- Z6 e7 d. d/ Z8 d0 d1 Z9 d2 d3 Z: d4 d5 Z; d6 d7 Z< d8 d9 Z= d: d; Z> d< d= Z? d> d? Z@ d@ dA dB ZA dC dD ZB dE dF ZC dG jD dH ZE eE dI ZF eE dJ ZG dK dL ZH dM dN ZI dO dP ZJ dH dQ dR ZK d S)Tz
requests.utils
~~~~~~~~~~~~~~
This module provides utility functions that are used within Requests
that are also useful for external consumption.
N ) __version__)certs)parse_http_list) quoteurlparsebytesstr OrderedDictunquoteis_py2 builtin_str
getproxies proxy_bypass
urlunparse)RequestsCookieJarcookiejar_from_dict)CaseInsensitiveDict)
InvalidURL.netrc_netrcc C s" t | d r | j } n | S)z/Returns an internal sequence dictionary update.items)hasattrr )d r 5/tmp/pip-ztkk0jow-build/pip/_vendor/requests/utils.pydict_to_sequence' s r c C s t | d r t | St | d r/ | j St | d ry y | j } Wn t j k
re Yqy Xt j | j Sn t | d r t | j Sd S)N__len__lenfilenogetvalue) r r r ioUnsupportedOperationosfstatst_sizer )or r r r super_len0 s
r' c
C s8 yd d l m } m } d } xb t D]Z } y t j j d j | } Wn t k
rc d SYn Xt j j | r& | } Pq& q& W| d k r d St | } | j
j d d } yG | | j | } | r | d r d n d } | | | d f SWn | t
f k
rYn XWn t t f k
r3Yn Xd S)z;Returns the Requests tuple auth for a given url from netrc.r )netrcNetrcParseErrorNz~/{0}:r )r( r) NETRC_FILESr# path
expanduserformatKeyErrorexistsr netlocsplitauthenticatorsIOError ImportErrorAttributeError)
urlr( r)
netrc_pathflocrihostr login_ir r r get_netrc_authD s0
r? c C sL t | d d } | rH | d d k rH | d d k rH t j j | Sd S)z0Tries to guess the filename of the given object.nameNr )getattrr# r- basename)objr@ r r r guess_filenamer s &rG c C sD | d k r d St | t t t t f r: t d n t | S)a Take an object and test to see if it can be represented as a
dictionary. Unless it can not be represented as such, return an
OrderedDict, e.g.,
::
>>> from_key_val_list([('key', 'val')])
OrderedDict([('key', 'val')])
>>> from_key_val_list('string')
ValueError: need more than 1 value to unpack
>>> from_key_val_list({'key': 'val'})
OrderedDict([('key', 'val')])
Nz+cannot encode objects that are not 2-tuples)
isinstancer r boolint
ValueErrorr
)valuer r r from_key_val_listy s
rM c C se | d k r d St | t t t t f r: t d n t | t j r[ | j } n t | S)az Take an object and test to see if it can be represented as a
dictionary. If it can be, return a list of tuples, e.g.,
::
>>> to_key_val_list([('key', 'val')])
[('key', 'val')]
>>> to_key_val_list({'key': 'val'})
[('key', 'val')]
>>> to_key_val_list('string')
ValueError: cannot encode objects that are not 2-tuples.
Nz+cannot encode objects that are not 2-tuples)
rH r r rI rJ rK collectionsMappingr list)rL r r r to_key_val_list s
rQ c C sw g } xj t | D]\ } | d d | d d k oD d k n rb t | d d } n | j | q W| S)a Parse lists as described by RFC 2068 Section 2.
In particular, parse comma-separated lists where the elements of
the list may include quoted-strings. A quoted-string could
contain a comma. A non-quoted string could have quotes in the
middle. Quotes are removed automatically after parsing.
It basically works like :func:`parse_set_header` just that items
may appear multiple times and case sensitivity is preserved.
The return value is a standard :class:`list`:
>>> parse_list_header('token, "quoted value"')
['token', 'quoted value']
To create a header from the :class:`list` again, use the
:func:`dump_header` function.
:param value: a string with a list header.
:return: :class:`list`
Nr "rC rC )_parse_list_headerunquote_header_valueappend)rL resultitemr r r parse_list_header s 0rX c C s i } x t | D] } d | k r5 d | | >> d = parse_dict_header('foo="is a fish", bar="as well"')
>>> type(d) is dict
True
>>> sorted(d.items())
[('bar', 'as well'), ('foo', 'is a fish')]
If there is no value for a key it will be `None`:
>>> parse_dict_header('key_without_value')
{'key_without_value': None}
To create a header from the :class:`dict` again, use the
:func:`dump_header` function.
:param value: a string with a dict header.
:return: :class:`dict`
=Nr rR rC rC )rS r3 rT )rL rV rW r@ r r r parse_dict_header s
0rZ Fc C sz | rv | d | d k o% d k n rv | d d
} | sW | d d d k rv | j d d j d d Sn | S) zUnquotes a header value. (Reversal of :func:`quote_header_value`).
This does not use the real unquoting but what browsers are actually
using for quoting.
:param value: the header value to unquote.
r r rR Nr+ z\\\z\"rC rC )replace)rL is_filenamer r r rT s
* rT c C s+ i } x | D] } | j | | j ]flagsz+]z$^<\?xml.*?encoding=["\']*(.+?)["\'>])recompileIfindall)content
charset_re pragma_rexml_rer r r get_encodings_from_content s
rn c C s_ | j d } | s d St j | \ } } d | k rK | d j d Sd | k r[ d Sd S)zmReturns encodings from given HTTP Header Dict.
:param headers: dictionary to extract encoding from.
z content-typeNcharsetz'"textz
ISO-8859-1)getcgi parse_headerstrip)headers content_typeparamsr r r get_encoding_from_headers, s rx c c s | j d k r) x | D] } | Vq Wd St j | j d d } x+ | D]# } | j | } | rK | VqK qK W| j d d d } | r | Vn d S)zStream decodes a iterator.Nerrorsr\ finalT)encodingcodecsgetincrementaldecoderdecode)iteratorrrW decoderchunkrvr r r stream_decode_response_unicode@ s
r c c s@ d } x3 | t | k r; | | | | V| | 7} q Wd S)z Iterate over slices of a string.r N)r )string slice_lengthposr r r iter_slicesR s r c C s g } t | j } | rT y t | j | SWqT t k
rP | j | YqT Xn y t | j | d d SWn t k
r | j SYn Xd S)a Returns the requested content back in unicode.
:param r: Response object to get unicode content from.
Tried:
1. charset from content-type
2. every encodings from ````
3. fall back and replace all unicode characters
ry r\ N)rx ru r rj UnicodeErrorrU TypeError)r tried_encodingsr| r r r get_unicode_from_responseZ s
r Z4ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzz0123456789-._~c
C s | j d } x t d t | D] } | | d d } t | d k r | j r y t t | d } Wn" t k
r t d | Yn X| t k r | | | d d | | d A} t j t j d | S)zt
Converts mask from /xx format to xxx.xxx.xxx.xxx
Example: if mask is 24 function returns 255.255.255.0
l r z>I)r inet_ntoar pack)maskr r r r r s r c C s1 y t j | Wn t j k
r, d SYn Xd S)NFT)r r error) string_ipr r r is_ipv4_address s
r c C s | j d d k r y t | j d d } Wn t k
rJ d SYn X| d k sc | d k rg d Sy t j | j d d Wq t j k
r d SYq Xn d Sd S)z9Very simple check of the cidr format in no_proxy variabler r Fr r T)countrJ r3 rK r r r )string_networkr r r r
is_valid_cidr s
r c C s" d d } | d } t | j } | r | j d d j d } | j d d } t | r xs | D]( } t | rn t | | r d Sqn qn Wq x@ | D]5 } | j | s | j d d j | r d Sq Wn y t | } Wn! t t
j f k
rd
} Yn X| r d Sd
S) z:
Returns whether we should bypass proxies or not.
c S s( t j j | p' t j j | j S)N)r# environrq upper)kr r r s z'should_bypass_proxies..no_proxy r ,r* r TF) r r2 r\ r3 r r r endswithr r r gaierror)r8 get_proxyr r2 r proxy_ipr= bypassr r r should_bypass_proxies s*
+ r c C s t | r i St Sd S)z%Return a dict of environment proxies.N)r r )r8 r r r get_environ_proxies s r zpython-requestsc C s8 t j } | d k r' t j } n | d k r d t j j t j j t j j f } t j j d k r d j | t j j g } q n< | d k r t j } n! | d k r t j } n d } y t j
} t j } Wn t k
rd } d } Yn Xd j d
| t
f d
| | f d
| | f g S) z4Return a string representing the default user agent.CPythonPyPyz%s.%s.%sr{ r Jython
IronPythonUnknownr z%s/%s)platformpython_implementationpython_versionsyspypy_version_infomajorminormicro releaselevelr systemreleaser5 r )r@ _implementation_implementation_versionp_system p_releaser r r default_user_agent
s. !
r c C s+ t i t d 6d j d d 6d d 6 S) Nz
User-Agentz, gzipdeflatezAccept-Encodingz*/*Accept)zgzipzdeflate)r r r r r r r default_headers) s
r c C s g } d } x | j d D] } y | j d d \ } } Wn t k
r_ | d } } Yn Xi } | j d | d ; rel=front; type="image/jpeg",; rel=back;type="image/jpeg"
z '"r ;r r z<> '"r8 rY )r3 rK rt rU ) rL links
replace_charsvalr8 rw linkparamkeyr r r parse_header_links1 s"
r asciir+ c C s | d d } | t j t j f k r, d S| d d t j k rI d S| d d t j t j f k ro d S| j t } | d k r d S| d k r | d d d t k r d S| d
d d t k r d Sn | d k r| d d t k r d S| d
d t k rd
Sn d S)N zutf-32r z utf-8-sigr+ zutf-16r zutf-8z utf-16-ber z utf-16-lez utf-32-bez utf-32-le)
r} BOM_UTF32_LEBOM32_BEBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BEr _null_null2_null3)datasample nullcountr r r guess_json_utfY s* " r c C sS t | | \ } } } } } } | s7 | | } } n t | | | | | | f S)zGiven a URL that may or may not have a scheme, prepend the given scheme.
Does not replace a present scheme with the one provided as an argument.)r r )r8
new_schemeschemer2 r- rw queryfragmentr r r prepend_scheme_if_neededv s !r c C sS t | } y" t | j t | j f } Wn t t f k
rN d } Yn X| S)z_Given a url with authentication components, extract them into a tuple of
username,password.r )r r )r r usernamepasswordr7 r )r8 parsedauthr r r get_auth_from_url s " r c C sI d } t | t r | } n' t r6 | j | } n | j | } | S)z
Given a string object, regardless of type, returns a representation of that
string in the native string type, encoding and decoding where necessary.
This assumes ASCII unless told otherwise.
N)rH r
r encoder )r r| outr r r to_native_string s r )r z_netrc)L__doc__rr r} rN r! r# r rf r r r r r r compatr rS r r r r r
r r r
r r r cookiesr r
structuresr
exceptionsr _hush_pyflakesr, whereDEFAULT_CA_BUNDLE_PATHr r' r? rG rM rQ rX rZ rT ra rd rn rx r r r frozensetr r r r r r r r r r r r r r r r r r r r r r r r
sr L . " "
- #