Yf @ sN d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
m Z d d d d d d d
d d d
d d d d d d d d g Z d Z
d Z d Z d Z d Z d Z e j e j j d d e j j j D Z d Z d Z d Z e j d! j Z e j d" j Z e j d# Z! d$ d% d& h Z" d' d( d) Z# Gd* d+ d+ e j$ j% Z& e& d, d- Z' Gd. d d e j( Z) Gd/ d d Z* y d d l+ Z+ Wn e, k
rYn$ XGd0 d1 d1 e* Z- e j. d1 Gd2 d d e/ Z0 Gd3 d d e0 Z1 Gd4 d d e0 Z2 Gd5 d d e0 Z3 Gd6 d d e0 Z4 Gd7 d
d
e0 Z5 Gd8 d d e0 Z6 Gd9 d
d
e0 Z7 Gd: d d e7 Z8 Gd; d d e7 Z9 Gd< d d e7 Z: Gd= d d e0 Z; Gd> d d e0 Z< Gd? d d e= e; Z> e0 Z? d S)@a
HTTP/1.1 client library
HTTPConnection goes through a number of "states", which define when a client
may legally make another request or fetch the response for a particular
request. This diagram details these state transitions:
(null)
|
| HTTPConnection()
v
Idle
|
| putrequest()
v
Request-started
|
| ( putheader() )* endheaders()
v
Request-sent
|\_____________________________
| | getresponse() raises
| response = getresponse() | ConnectionError
v v
Unread-response Idle
[Response-headers-read]
|\____________________
| |
| response.read() | putrequest()
v v
Idle Req-started-unread-response
______/|
/ |
response.read() | | ( putheader() )* endheaders()
v v
Request-started Req-sent-unread-response
|
| response.read()
v
Request-sent
This diagram presents the following rules:
-- a second request may not be started until {response-headers-read}
-- a response [object] cannot be retrieved until {request-sent}
-- there is no differentiation between an unread response body and a
partially read response body
Note: this enforcement is applied by the HTTPConnection class. The
HTTPResponse class does not enforce this state machine, which
implies sophisticated clients may accelerate the request/response
pipeline. Caution should be taken, though: accelerating the states
beyond the above pattern may imply knowledge of the server's
connection-close behavior for certain requests. For example, it
is impossible to tell whether the server will close the connection
UNTIL the response headers have been read; this means that further
requests cannot be placed into the pipeline until it is known that
the server will NOT be closing the connection.
Logical State __state __response
------------- ------- ----------
Idle _CS_IDLE None
Request-started _CS_REQ_STARTED None
Request-sent _CS_REQ_SENT None
Unread-response _CS_IDLE
Req-started-unread-response _CS_REQ_STARTED
Req-sent-unread-response _CS_REQ_SENT
N)urlsplit HTTPResponseHTTPConnection
HTTPException NotConnectedUnknownProtocolUnknownTransferEncodingUnimplementedFileModeIncompleteRead
InvalidURLImproperConnectionStateCannotSendRequestCannotSendHeaderResponseNotReady
BadStatusLine LineTooLongRemoteDisconnectederror responsesP i ZUNKNOWNZIdlezRequest-startedz Request-sentc C s i | ] } | j | q S )phrase).0vr r 0/opt/alt/python35/lib64/python3.5/http/client.py
k s r i i d s [^:\s][^:\r\n]*s \n(?![ \t])|\r(?![ \t\n])z[ - ]ZPATCHZPOSTZPUTdatac C s y | j d SWno t k
r } zO t | j | j | j | j d | j | | j | j | f d WYd d } ~ Xn Xd S)z d? Z S)Cr r Nc C sw | j d | _ | | _ | | _ d | _ | _ t | _ t | _ t | _ t | _
t | _ t | _ t | _
d S)Nrb)makefilerB
debuglevel_methodrC msg_UNKNOWNversionstatusreasonchunked
chunk_leftlength
will_close)r0 sockrG methodurlr r r __init__ s zHTTPResponse.__init__c C si t | j j t d d } t | t k r= t d | j d k r_ t d t | | sq t d y | j
d d \ } } } WnO t k
r y" | j
d d \ } } d } Wn t k
r d } Yn XYn X| j d s| j
t | y4 t | } | d
k s.| d k r:t | Wn t k
r[t | Yn X| | | f S) Nr* z
iso-8859-1z status liner zreply:z-Remote end closed connection without response zHTTP/r i )strrB r; r< r, r rG printreprr split
ValueError
startswith _close_connr int)r0 r2 rK rL rM r r r _read_status s2
zHTTPResponse._read_statusc C s | j d k r d Sx | j \ } } } | t k r8 Pxg | j j t d } t | t k ro t d | j } | s P| j d k r; t
d | q; Wq W| | _ | _ | j | _
| d k r d | _ n' | j d r d | _ n t | t | j | _ | _ | j d k rSx$ | j D] } t
d | d
d q6W| j j d } | r| j d
k rd | _ d | _ n d | _ | j | _ d | _ | j j d } | j j d } | r3| j r3y t | | _ Wn t k
rd | _ Yq<X| j d k r<d | _ n d | _ | t k s| t k sd | k okd k n s| j d k rd | _ | j r| j r| j d k rd | _ d S)Nr* z header liner zheader:HTTP/1.0HTTP/0.9
zHTTP/1. r# ztransfer-encodingrN TFzcontent-lengthr HEAD)ra rb ) rC r` ZCONTINUErB r; r< r, r striprG rY coderL rM rK r] r rD rI getr+ rN rO _check_closerQ rP r_ r\ Z
NO_CONTENTZ NOT_MODIFIEDrH )r0 rK rL rM skiphdrZtr_encrP r r r begin, sf
zHTTPResponse.beginc C s | j j d } | j d k rS | j j d } | rO d | j k rO d Sd S| j j d ri d S| r d | j k r d S| j j d } | r d | j k r d Sd S)NZ
connectionrd closeTFz
keep-alivezproxy-connection)rC rj rK r+ )r0 ZconnZpconnr r r rk | s zHTTPResponse._check_closec C s | j } d | _ | j d S)N)rB ro )r0 rB r r r r^ s zHTTPResponse._close_connc
s, z t j Wd | j r' | j Xd S)N)superro rB r^ )r0 ) __class__r r ro s zHTTPResponse.closec s' t j | j r# | j j d S)N)rp flushrB )r0 )rq r r rr s
zHTTPResponse.flushc C s d S)NTr )r0 r r r readable s zHTTPResponse.readablec C s
| j d k S)z!True if the connection is closed.N)rB )r0 r r r isclosed s zHTTPResponse.isclosedc C s | j d k r d S| j d k r0 | j d S| d k rq t | } | j | } t | d | j S| j r | j S| j d k r | j j
} nA y | j | j } Wn t k
r | j Yn Xd | _ | j | Sd S)Nr9 rg r )
rB rH r^ bytearrayreadinto
memoryviewtobytesrN _readall_chunkedrP read
_safe_readr
)r0 amtbr1 sr r r rz s*
zHTTPResponse.readc C s | j d k r d S| j d k r0 | j d S| j rF | j | S| j d k r t | | j k r t | d | j } | j j | } | r | r | j n1 | j d k r | j | 8_ | j s | j | S)Nr rg ) rB rH r^ rN _readinto_chunkedrP r, rw rv )r0 r} r1 r r r rv s$
zHTTPResponse.readintoc C s | j j t d } t | t k r4 t d | j d } | d k r_ | d | } y t | d SWn t k
r | j Yn Xd S)Nr* z
chunk size ;r ) rB r; r< r, r findr_ r\ r^ )r0 r2 ir r r _read_next_chunk_size s
z"HTTPResponse._read_next_chunk_sizec C sS xL | j j t d } t | t k r7 t d | s> P| d k r Pq Wd S)Nr* z trailer line
r9 )r r r9 )rB r; r< r, r )r0 r2 r r r _read_and_discard_trailer
s z&HTTPResponse._read_and_discard_trailerc C s | j } | s | d k r( | j d y | j } Wn t k
rX t d Yn X| d k r | j | j d } | | _ | S)NrV r9 r )rO r{ r r\ r
r r^ )r0 rO r r r _get_chunk_left s
z HTTPResponse._get_chunk_leftc C s g } yP x< | j } | d k r% P| j | j | d | _ q Wd j | SWn' t k
r t d j | Yn Xd S)Nr r9 )r r/ r{ rO r> r
)r0 valuerO r r r ry 3 s
z HTTPResponse._readall_chunkedc C s d } t | } y x | j } | d k r4 | St | | k rj | j | } | | | _ | | S| d | } | j | } | | d } | | 7} d | _ q WWn. t k
r t t | d | Yn Xd S)Nr )rw r r, _safe_readintorO r
bytes)r0 r} total_bytesmvbrO r1 temp_mvbr r r r A s$
z HTTPResponse._readinto_chunkedc C s| g } xf | d k rn | j j t | t } | sN t d j | | | j | | t | 8} q Wd j | S)aV Read the number of bytes requested, compensating for partial reads.
Normally, we have a blocking socket, but a read() can be interrupted
by a signal (resulting in a partial read).
Note that we cannot distinguish between EOF and an interrupt when zero
bytes have been read. IncompleteRead() will be raised in this
situation.
This function should be used when bytes "should" be present for
reading. If the bytes are truly not available (due to EOF), then the
IncompleteRead exception can be used to detect the problem.
r r9 )rB rz min MAXAMOUNTr
r> r/ r, )r0 r| r~ chunkr r r r{ Y s
zHTTPResponse._safe_readc C s d } t | } x | t | k r t t | k r^ | d t } | j j | } n | j j | } | s t t | d | t | | | d } | | 7} q W| S)z2Same as _safe_read, but for reading into a buffer.r N)rw r, r rB rv r
r )r0 r} r r r r1 r r r r p s %zHTTPResponse._safe_readintor* c C s | j d k s | j d k r" d S| j r8 | j | S| j d k rk | d k sb | | j k rk | j } y | j j | } Wn3 t k
r | d k r | j j d } Yn X| r | r | j n$ | j d k r | j t | 8_ | S)zvRead with at most one underlying system call. If at least one
byte is buffered, return that instead.
Nrg r9 r r i i @ ) rB rH rN _read1_chunkedrP read1r\ r^ r, )r0 r1 resultr r r r s"
*
zHTTPResponse.read1c C sH | j d k s | j d k r" d S| j r8 | j | S| j j | S)Nrg r9 )rB rH rN
_peek_chunkedpeek)r0 r1 r r r r s
zHTTPResponse.peekc s | j d k s | j d k r" d S| j r; t j | S| j d k rn | d k se | | j k rn | j } | j j | } | r | r | j n$ | j d k r | j t | 8_ | S)Nrg r9 r )rB rH rN rp r; rP r^ r, )r0 limitr )rq r r r; s *
zHTTPResponse.readlinec C s | j } | d k s$ | d k r( d Sd | k o? | k n sJ | } | j j | } | j t | 8_ | s t d | S)Nr r9 )r rB r rO r, r
)r0 r1 rO rz r r r r s zHTTPResponse._read1_chunkedc C sS y | j } Wn t k
r( d SYn X| d k r9 d S| j j | d | S)Nr9 )r r
rB r )r0 r1 rO r r r r s
zHTTPResponse._peek_chunkedc C s
| j j S)N)rB fileno)r0 r r r r s zHTTPResponse.filenoc C sd | j d k r t | j j | p- | } t | t sO t | d rS | Sd j | Sd S)N__iter__z, )rC r Zget_all
isinstancerX hasattrr> )r0 r% defaultrC r r r getheader s zHTTPResponse.getheaderc C s+ | j d k r t t | j j S)z&Return list of (header, value) tuples.N)rC r listitems)r0 r r r
getheaders s zHTTPResponse.getheadersc C s | S)Nr )r0 r r r r s zHTTPResponse.__iter__c C s | j S)N)rC )r0 r r r info s zHTTPResponse.infoc C s | j S)N)rT )r0 r r r geturl s zHTTPResponse.geturlc C s | j S)N)rL )r0 r r r getcode s zHTTPResponse.getcoder r )!r4 r5 r6 rU r` rn rk r^ ro rr rs rt rz rv r r r ry r r{ r r r r; r r r r r r r r r r r )rq r r s< ! P
c @ s< e Z d Z d Z d Z e Z e Z d Z d Z
d e j d d d Z
d d d d Z d
d Z d d
Z d d Z d d Z d d Z d d Z d d Z d d d Z d d d d Z d d Z d d Z d! d" Z d d# d$ Z d i d% d& Z d' d( Z d) d* Z d+ d, Z d S)-r rd zHTTP/1.1r* r Nc C s | | _ | | _ d | _ g | _ d | _ t | _ d | _ d | _ d | _ i | _
| j | | \ | _ | _
t j | _ d S)N)timeoutsource_addressrR _buffer_HTTPConnection__response_CS_IDLE_HTTPConnection__staterH _tunnel_host _tunnel_port_tunnel_headers
_get_hostporthostportsocketZcreate_connection_create_connection)r0 r r r r r r r rU s zHTTPConnection.__init__c C sV | j r t d | j | | \ | _ | _ | rE | | _ n
| j j d S)aD Set up host and port for HTTP CONNECT tunnelling.
In a connection that uses HTTP CONNECT tunneling, the host passed to the
constructor is used as a proxy server that relays all communication to
the endpoint passed to `set_tunnel`. This done by sending an HTTP
CONNECT request to the proxy server when the connection is established.
This method must be called before the HTML connection has been
established.
The headers argument should be a mapping of extra HTTP headers to send
with the CONNECT request.
z.Can't set up tunnel for established connectionN)rR RuntimeErrorr r r r clear)r0 r r rC r r r
set_tunnel
s zHTTPConnection.set_tunnelc C s | d k r | j d } | j d } | | k r y t | | d d } WnV t k
r | | d d d k r | j } n t d | | d d Yn X| d | } n | j } | r | d d k r | d d k r | d d } | | f S)
Nr) ]r* rW znonnumeric port: '%s'r [r r )rfindr_ r\ default_portr )r0 r r r jr r r r " s
# &z HTTPConnection._get_hostportc C s
| | _ d S)N)rG )r0 levelr r r set_debuglevel6 s z HTTPConnection.set_debuglevelc C sh d | j | j f } | j d } | j | xI | j j D]8 \ } } d | | f } | j d } | j | qB W| j d | j | j d | j } | j \ } } }
| t
j j k r | j
t d | |
j f xn | j j t d } t | t k r*t d | s1P| d k r>P| j d k r t d
| j q Wd S)NzCONNECT %s:%d HTTP/1.0
asciiz%s: %s
zlatin-1
rS z Tunnel connection failed: %d %sr* z header line
r9 r zheader:)r r r9 )r r r sendr r response_classrR rH r` http
HTTPStatusZOKro OSErrorrh rB r; r< r, r rG rY r? ) r0 Z connect_strZ
connect_bytesheaderr Z
header_strZ header_bytesresponserK ri messager2 r r r _tunnel9 s2
zHTTPConnection._tunnelc C s] | j | j | j f | j | j | _ | j j t j t j d | j
rY | j d S)z3Connect to the host and port specified in __init__.r* N) r r r r r rR Z
setsockoptr Z IPPROTO_TCPZ TCP_NODELAYr r )r0 r r r connectX s
$ zHTTPConnection.connectc C sY t | _ z& | j } | r. d | _ | j Wd | j } | rT d | _ | j Xd S)z(Close the connection to the HTTP server.N)r r rR ro r )r0 rR r r r r ro a s zHTTPConnection.closec C s | j d k r. | j r% | j n t | j d k rP t d t | d } t | d r| j d k r~ t d d } y
| j } Wn t k
r Yn, Xd | k r d } | j d k r t d
x? | j
| } | s P| r | j d } | j j | q Wd Sy | j j | Wn^ t
k
rt | t j rqx7 | D] } | j j | qTWn t
d t | Yn Xd S)
zSend `data' to the server.
``data`` can be a string object, a bytes object, an array object, a
file-like object that supports a .read() method, or an iterable object.
Nr zsend:i rz zsendIng a read()ableFr} Tz encoding file using iso-8859-1z
iso-8859-1z9data should be a bytes-like object or an iterable, got %r)rR auto_openr r rG rY rZ r modeAttributeErrorrz r Zsendall TypeErrorr collectionsIterabletype)r0 r Z blocksizer r Z datablockdr r r r o sF
zHTTPConnection.sendc C s | j j | d S)zuAdd a line of output to the current request buffer.
Assumes that the line does *not* end with \r\n.
N)r r/ )r0 r~ r r r _output s zHTTPConnection._outputc C s\ | j j d d j | j } | j d d =| j | | d k rX | j | d S)zSend the currently buffered request and clear the buffer.
Appends an extra \r\n to the buffer.
A message_body may be specified, to be appended to the request.
r9 s
N)r9 r9 )r extendr> r )r0 message_bodyrI r r r _send_output s
zHTTPConnection._send_outputFc
C s | j r! | j j r! d | _ | j t k r< t | _ n t | j | | _ | p] d } | j | d | | | j f } | j | j
| | j d k r | sd } | j d r t
| \ } } } } } | r.y | j d } Wn! t k
r| j d } Yn X| j d | n | j rL| j } | j }
n | j } | j }
y | j d } Wn! t k
r| j d } Yn X| j d
d k rd | d
} |
| j k r| j d | n) | j d } | j d d | |
f | s | j d d n d S)a` Send a request to the server.
`method' specifies an HTTP request method, e.g. 'GET'.
`url' specifies the object being requested, e.g. '/index.html'.
`skip_host' if True does not add automatically a 'Host:' header
`skip_accept_encoding' if True does not add automatically an
'Accept-Encoding:' header
N/z%s %s %srd rW r r ZidnaZHostr) r [ ]z%s:%szAccept-EncodingZidentity)r rt r r _CS_REQ_STARTEDr
rH _validate_path
_http_vsn_strr _encode_request _http_vsnr] r r r putheaderr r r r r r r? ) r0 rS rT skip_hostskip_accept_encodingrequestZnetlocZnilZ
netloc_encr r Zhost_encr r r
putrequest sN
zHTTPConnection.putrequestc C s
| j d S)Nr )r )r0 r r r r r 1 s z HTTPConnection._encode_requestc C sC t j | } | r? d j d | j t } t | d S)z Validate a url for putrequest.zJURL can't contain control characters. {url!r} (found at least {matched!r})ZmatchedN)!_contains_disallowed_url_pchar_researchformatgrouplocalsr )r0 rT matchrI r r r r 5 s
z HTTPConnection._validate_pathc G s | j t k r t t | d r6 | j d } t | sU t d | f t | } x t | D] \ } } t | d r | j d | | r )r0 r valuesr Z one_valuer r r r r @ s" zHTTPConnection.putheaderc C s5 | j t k r t | _ n t | j | d S)a Indicate that the last header line has been sent to the server.
This method sends the request to the server. The optional message_body
argument can be used to pass a message body associated with the
request. The message body will be sent in the same packet as the
message headers if it is a string, otherwise it is sent as a separate
packet.
N)r r _CS_REQ_SENTr r )r0 r r r r
endheaders\ s zHTTPConnection.endheadersc C s | j | | | | d S)z&Send a complete request to the server.N)
_send_request)r0 rS rT bodyrC r r r r k s zHTTPConnection.requestc C s d } | j t k } | d k r3 | r3 d } n | d k r y t t | } Wnh t k
r y" t t j | j j } Wn1 t t
f k
r | j d k r t d Yn XYn X| d k r | j
d | d S)N0r z
Cannot stat!!zContent-Length)upper_METHODS_EXPECTING_BODYrX r, r osfstatr st_sizer r rG rY r )r0 r rS ZthelenZmethod_expects_bodyr r r _set_content_lengtho s
" z"HTTPConnection._set_content_lengthc C s t j d d | D } i } d | k r8 d | d s z0HTTPConnection._send_request..r r* r zaccept-encodingr zcontent-lengthr )
dictfromkeysr r r r r rX r' r ) r0 rS rT r rC Z header_namesZskipsrm r r r r r s
z HTTPConnection._send_requestc C s | j r! | j j r! d | _ | j t k s9 | j rH t | j | j d k r{ | j | j | j d | j } n | j | j d | j } y` y | j Wn t
k
r | j Yn Xt | _ | j
r | j n | | _ | SWn | j Yn Xd S)a) Get the response from the server.
If the HTTPConnection is in the correct state, returns an
instance of HTTPResponse or of whatever object is returned by
the response_class variable.
If a request has not been sent or if a previous response has
not be handled, ResponseNotReady is raised. If the HTTP
response indicates that the connection should be closed, then
it will be closed before the response is returned. When the
connection is closed, the underlying socket is closed.
Nr rS )r rt r r r rG r rR rH rn ConnectionErrorro r rQ )r0 r r r r getresponse s,
zHTTPConnection.getresponse) r4 r5 r6 r r r r HTTP_PORTr r rG r _GLOBAL_DEFAULT_TIMEOUTrU r r r r r ro r r r r r r r r r r r r r r r r r s6 0 | c s^ e Z d Z d Z e Z d d d e j d d d d d f d d Z f d d Z S) HTTPSConnectionz(This class allows communication via SSL.Ncontextcheck_hostnamec
s t t | j | | | | | | _ | | _ | d k rI t j } | j t j k } | d k rp | j } | r | r t
d | s | r | j | | | | _ | | _
d S)NzMcheck_hostname needs a SSL context with either CERT_OPTIONAL or CERT_REQUIRED)rp r rU key_file cert_filesslZ _create_default_https_contextZ verify_modeZ CERT_NONEr r\ Zload_cert_chain_context_check_hostname)
r0 r r r r r r r r Z will_verify)rq r r rU s
zHTTPSConnection.__init__c s t j | j r" | j } n | j } | j j | j d | | _ | j j r | j r y t j
| j j | Wn5 t k
r | j j
t j | j j Yn Xd S)z(Connect to a host on a given (SSL) port.server_hostnameN)rp r r r r Z wrap_socketrR r r r Zmatch_hostnameZ getpeercert ExceptionZshutdownr Z SHUT_RDWRro )r0 r )rq r r r s
zHTTPSConnection.connect)
r4 r5 r6 __doc__
HTTPS_PORTr r r rU r r r )rq r r s r c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r s c @ s e Z d Z d d Z d S)r c C s | f | _ | | _ d S)N)argsrK )r0 rK r r r rU s zUnknownProtocol.__init__N)r4 r5 r6 rU r r r r r s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r " s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r % s c @ s7 e Z d Z d d d Z d d Z d d Z d S)r
Nc C s" | f | _ | | _ | | _ d S)N)r partialexpected)r0 r
r r r r rU ) s zIncompleteRead.__init__c C sE | j d k r d | j } n d } d | j j t | j | f S)Nz, %i more expectedrW z%s(%i bytes read%s))r rq r4 r, r
)r0 er r r __repr__- s
zIncompleteRead.__repr__c C s
t | S)N)rZ )r0 r r r __str__4 s zIncompleteRead.__str__)r4 r5 r6 rU r r r r r r r
( s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r 7 s c @ s e Z d Z d S)r
N)r4 r5 r6 r r r r r
: s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r = s c @ s e Z d Z d S)r N)r4 r5 r6 r r r r r @ s c @ s e Z d Z d d Z d S)r c C s+ | s t | } | f | _ | | _ d S)N)rZ r r2 )r0 r2 r r r rU D s zBadStatusLine.__init__N)r4 r5 r6 rU r r r r r C s c @ s e Z d Z d d Z d S)r c C s t j | d t | f d S)Nz&got more than %d bytes when reading %s)r rU r< )r0 Z line_typer r r rU K s zLineTooLong.__init__N)r4 r5 r6 rU r r r r r J s c @ s e Z d Z d d Z d S)r c O s' t j | d t j | | | d S)NrW )r rU ConnectionResetError)r0 poskwr r r rU P s zRemoteDisconnected.__init__N)r4 r5 r6 rU r r r r r O s )@r
Z email.parserr@ Z
email.messager ior rer r Z urllib.parser __all__r r rJ r r r globalsupdater Z __members__r r r r< r= compile fullmatchr r r r r r' r ZMessager( rD BufferedIOBaser r r ImportErrorr r/ r r r r r r r r
r r
r r r r r r r r r r r E sx
0