U
ÀÒ.e_Ê ã @ sD d dl Z d dlmZ d dlZd dlZd dlZd dlZd dlZz d dlZW n e k
rd d dl
ZY nX d dl Z ddl m
Z
ddlmZmZmZmZmZmZmZmZmZmZmZmZm Z m Z m Z ddl m Z m!Z!m"Z" ddl#m$Z$m%Z% ddl&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/ dd l0m1Z1m2Z2 dd
l3m4Z4m5Z5 e 6e7¡Z8e 9d ¡Z:e 9d ej;¡Zd-dd„Z?G dd„ deƒZ@G dd„ deAƒZBG dd„ deBƒZCG dd„ deBƒZDG dd„ deAƒZEG dd „ d eBƒZFG d d „ d eBƒZGG d d „ d eBƒZHG d!d"„ d"eBƒZIG d#d$„ d$eBƒZJeJeHƒ eFd%d&d'd(d)ZKeKjLZLe 9d*¡ZMG d+d,„ d,eAƒZNdS ).é N)ÚBytesIOé )ÚDistlibException)ÚurljoinÚurlparseÚ
urlunparseÚ url2pathnameÚ pathname2urlÚqueueÚquoteÚunescapeÚ string_typesÚ build_openerÚHTTPRedirectHandlerÚ text_typeÚRequestÚ HTTPErrorÚURLError)Ú DistributionÚDistributionPathÚ make_dist)ÚMetadataÚMetadataInvalidError) Úcached_propertyÚparse_credentialsÚ ensure_slashÚsplit_filenameÚget_project_dataÚparse_requirementÚparse_name_and_versionÚ ServerProxyÚnormalize_name)Ú
get_schemeÚUnsupportedVersionError)ÚWheelÚ
is_compatiblez^(\w+)=([a-f0-9]+)z;\s*charset\s*=\s*(.*)\s*$z text/html|application/x(ht)?mlzhttps://pypi.org/pypic C s6 | dkr t } t| dd}z | ¡ W ¢S |dƒƒ X dS )z•
Return all distribution names known by an index.
:param url: The URL of the index.
:return: A list of all known distribution names.
Nç @©ÚtimeoutÚclose)Ú
DEFAULT_INDEXr Ú
list_packages)ÚurlÚclient© r. ú@/usr/lib/python3.8/site-packages/pip/_vendor/distlib/locators.pyÚget_all_distribution_names) s r0 c @ s$ e Zd ZdZdd„ Ze Z ZZdS )ÚRedirectHandlerzE
A class to work around a bug in some Python 3.2.x releases.
c C s‚ d }dD ]}||kr|| } q"q|d kr.d S t |ƒ}|jdkrnt| ¡ |ƒ}t|dƒrf| ||¡ n|||< t | |||||¡S )N)ÚlocationZuriÚ Úreplace_header)r Úschemer Z get_full_urlÚhasattrr4 ÚBaseRedirectHandlerÚhttp_error_302) ÚselfÚreqÚfpÚcodeÚmsgÚheadersZnewurlÚkeyZurlpartsr. r. r/ r8 @ s
ÿz RedirectHandler.http_error_302N)Ú__name__Ú
__module__Ú __qualname__Ú__doc__r8 Zhttp_error_301Zhttp_error_303Zhttp_error_307r. r. r. r/ r1 7 s r1 c @ s¶ e Zd ZdZdZdZdZdZed Zd)dd „Z d
d „ Z
d d
„ Z dd„ Z dd„ Z
dd„ Zee
eƒZdd„ Zdd„ Zdd„ Zdd„ Zd d „ Zd d „ Zd d!„ Zd"d#„ Zd$d%„ Zd*d'd(„ZdS )+ÚLocatorzG
A base class for locators - things that locate distributions.
)z.tar.gzz.tar.bz2z.tarz.zipz.tgzz.tbz)z.eggz.exeú.whl)z.pdfN)rE Údefaultc C s, i | _ || _ttƒ ƒ| _d| _t ¡ | _dS )a^
Initialise an instance.
:param scheme: Because locators look for most recent versions, they
need to know the version scheme to use. This specifies
the current PEP-recommended scheme - use ``'legacy'``
if you need to support existing distributions on PyPI.
N) Ú_cacher5 r r1 ÚopenerÚmatcherr
ÚQueueÚerrors)r9 r5 r. r. r/ Ú__init__f s
zLocator.__init__c C sV g }| j ¡ sRz| j d¡}| |¡ W n | j jk
rD Y qY nX | j ¡ q|S )z8
Return any errors which have occurred.
F)rK ÚemptyÚgetÚappendZEmptyÚ task_done)r9 ÚresultÚer. r. r/ Ú
get_errorsy s
zLocator.get_errorsc C s | ¡ dS )z>
Clear any errors which may have been logged.
N)rS ©r9 r. r. r/ Ú clear_errors‡ s zLocator.clear_errorsc C s | j ¡ d S ©N)rG ÚclearrT r. r. r/ Ú clear_cacheŽ s zLocator.clear_cachec C s | j S rV ©Ú_schemerT r. r. r/ Ú _get_scheme‘ s zLocator._get_schemec C s
|| _ d S rV rY )r9 Úvaluer. r. r/ Ú _set_scheme” s zLocator._set_schemec C s t dƒ‚dS )a=
For a given project, get a dictionary mapping available versions to Distribution
instances.
This should be implemented in subclasses.
If called from a locate() request, self.matcher will be set to a
matcher for the requirement to satisfy, otherwise it will be None.
ú Please implement in the subclassN©ÚNotImplementedError)r9 Únamer. r. r/ Ú _get_project™ s
zLocator._get_projectc C s t dƒ‚dS )úJ
Return all the distribution names known to this locator.
r^ Nr_ rT r. r. r/ Úget_distribution_names¥ s z Locator.get_distribution_namesc C sL | j dkr| |¡}n2|| j kr,| j | }n | ¡ | |¡}|| j |< |S )zÖ
For a given project, get a dictionary mapping available versions to Distribution
instances.
This calls _get_project to do all the work, and just implements a caching layer on top.
N)rG rb rU )r9 ra rQ r. r. r/ Ú get_project« s
zLocator.get_projectc C s^ t |ƒ}t |j¡}d}| d¡}| | j¡}|rBtt|ƒ| jƒ}|j dkd|j
k||||fS )zu
Give an url a score which can be used to choose preferred URLs
for a given project release.
TrE Úhttpszpypi.org) r Ú posixpathÚbasenameÚpathÚendswithÚdownloadable_extensionsr% r$ Ú
wheel_tagsr5 Únetloc)r9 r, Útrh Z
compatibleZis_wheelZis_downloadabler. r. r/ Ú score_url¼ s
ÿzLocator.score_urlc C sR |}|rN| |¡}| |¡}||kr(|}||kr@t d||¡ nt d||¡ |S )a{
Choose one of two URLs where both are candidates for distribution
archives for the same version of a distribution (for example,
.tar.gz vs. zip).
The current implementation favours https:// URLs over http://, archives
from PyPI over those from other locations, wheel compatibility (if a
wheel) and then the archive name.
zNot replacing %r with %rzReplacing %r with %r)ro ÚloggerÚdebug)r9 Úurl1Úurl2rQ Ús1Ús2r. r. r/ Ú
prefer_urlË s
zLocator.prefer_urlc C s
t ||ƒS )zZ
Attempt to split a filename in project name, version and Python version.
)r )r9 ÚfilenameÚ project_namer. r. r/ r á s zLocator.split_filenamec C s dd„ }d}t |ƒ\}}}}} }
|
¡ d¡rz~t |ƒ}t
|| j ƒs°t d
|¡ nX|dkr¾d }n ||j |ƒ}|r|j |j
|jt||||| d fƒd
dd„ |jD ƒ¡dœ}W n0 tk
r: } zt d|¡ W 5 d}~X Y nX nÄ| | j¡sZt d|¡ n¨t |¡ }}| jD ]’}| |¡rn|dt|ƒ … }| ||¡}|s°t d|¡ nH|\}}}|rÌ|||ƒrø|||t||||| d fƒdœ}|rø||d< qqn|r| r|
|d| < |S )a
See if a URL is a candidate for a download URL for a project (the URL
has typically been scraped from an HTML page).
If it is, a dictionary is returned with keys "name", "version",
"filename" and "url"; otherwise, None is returned.
c S s t | ƒt |ƒkS rV )r! )Zname1Zname2r. r. r/ Ú same_projectï s z:Locator.convert_url_to_download_info..same_projectNzegg=z %s: version hint in fragment: %r)NNéÿÿÿÿú/rE zWheel not compatible: %sTr3 z, c S s" g | ]}d t|dd… ƒ¡‘qS )Ú.é N)ÚjoinÚlist)Ú.0Úvr. r. r/ Ú
s z8Locator.convert_url_to_download_info..)ra Úversionrw r, úpython-versionzinvalid path for wheel: %szNot downloadable: %sz No match for project/version: %s)ra rƒ rw r, r„ ú %s_digest)r ÚlowerÚ
startswithrp rq Ú HASHER_HASHÚmatchÚgroupsrj r$ r% rl ra rƒ rw r r~ ÚpyverÚ ExceptionÚwarningrk rg rh Úlenr )r9 r, rx ry rQ r5 rm ri ÚparamsÚqueryÚfragÚmÚalgoÚdigestZorigpathÚwheelZincluderR rw Zextrn ra rƒ r‹ r. r. r/ Ú convert_url_to_download_infoç s€ ÿ
ÿÿú
ÿü
z$Locator.convert_url_to_download_infoc C s2 d}dD ]$}d| }||kr||| f} q.q|S )zØ
Get a digest from a dictionary by looking at keys of the form
'algo_digest'.
Returns a 2-tuple (algo, digest) if found, else None. Currently
looks only for SHA256, then MD5.
N)Zsha256Úmd5r… r. )r9 ÚinforQ r“ r? r. r. r/ Ú _get_digest1 s zLocator._get_digestc C s® | d¡}| d¡}||kr,|| }|j}nt||| jd}|j}| |¡ |_}|d }||d |< |j|d krœ| |j|¡|_|d |t ƒ ¡
|¡ | |_ |||< dS )zñ
Update a result dictionary (the final result from _get_project) with a
dictionary for a specific version, which typically holds information
gleaned from a filename or URL for an archive for the distribution.
ra rƒ ©r5 r, ÚdigestsÚurlsN) ÚpopÚmetadatar r5 r™ r” Ú
source_urlrv Ú
setdefaultÚsetÚaddÚlocator) r9 rQ r˜ ra rƒ ÚdistÚmdr” r, r. r. r/ Ú_update_version_dataA s
z Locator._update_version_dataFc C s¤ d}t |ƒ}|dkr td| ƒ‚t| jƒ}| |j¡ | _}t d|t|ƒj ¡ |
|j ¡}t |ƒdkr2g }|j
} |D ]z}
|
dkr†qxzH| |
¡s¢t d||
¡ n*|s°| |
ƒjs¼| |
¡ nt d|
|j ¡ W qx tk
rð t d||
¡ Y qxX qxt |ƒd krt||jd
}|r2t d |¡ |d } || }|rš|jrH|j|_| d
i ¡ | tƒ ¡|_i } | di ¡}
|jD ] }||
krv|
| | |< qv| |_d| _|S )a
Find the most recent distribution which matches the given
requirement.
:param requirement: A requirement of the form 'foo (1.0)' or perhaps
'foo (>= 1.0, < 2.0, != 1.3)'
:param prereleases: If ``True``, allow pre-release versions
to be located. Otherwise, pre-release versions
are not returned.
:return: A :class:`Distribution` instance, or ``None`` if no such
distribution could be located.
NzNot a valid requirement: %rzmatcher: %s (%s)r} ©rœ r› z%s did not match %rz%skipping pre-release version %s of %szerror matching %s with %rr )r? zsorted list: %srz rœ r› )r r r" r5 rI Ú requirementrp rq Útyper@ re ra rŽ Z
version_classr‰ Z
is_prereleaserO rŒ r Úsortedr? ZextrasrN r¡ Ú
download_urlsr› )r9 r¨ Ú prereleasesrQ Úrr5 rI ÚversionsZslistZvclsÚkrƒ ÚdZsdr, r. r. r/ ÚlocateX sX
ÿ
zLocator.locate)rF )F)r@ rA rB rC Úsource_extensionsÚbinary_extensionsÚexcluded_extensionsrl rk rL rS rU rX r[ r] Úpropertyr5 rb rd re ro rv r r– r™ r¦ r± r. r. r. r/ rD V s.
JrD c s0 e Zd ZdZ‡ fdd„Zdd„ Zdd„ Z‡ ZS )ÚPyPIRPCLocatorz‘
This locator uses XML-RPC to locate distributions. It therefore
cannot be used with simple mirrors (that only mirror file content).
c s* t t| ƒjf |Ž || _t|dd| _dS )z—
Initialise an instance.
:param url: The URL to use for XML-RPC.
:param kwargs: Passed to the superclass constructor.
r& r' N)Úsuperr¶ rL Úbase_urlr r- ©r9 r, Úkwargs©Ú __class__r. r/ rL ™ s zPyPIRPCLocator.__init__c C s t | j ¡ ƒS ©rc )r¡ r- r+ rT r. r. r/ rd ¤ s z%PyPIRPCLocator.get_distribution_namesc C s i i dœ}| j |d¡}|D ]Þ}| j ||¡}| j ||¡}t| jd}|d |_|d |_| d¡|_ | dg ¡|_
| d¡|_ t |ƒ}|r |d } | d
|_
| | ¡|_| |_|||< |D ]:} | d
}
| | ¡} |d |tƒ ¡ |
¡ | |d |
< q¾q |S )
Nr§ Trš ra rƒ ÚlicenseÚkeywordsÚsummaryr r, rœ r› )r- Zpackage_releasesZ release_urlsZ release_datar r5 ra rƒ rN r¾ r¿ rÀ r rŸ r™ r” r£ r r¡ r¢ ) r9 ra rQ r® r rœ Údatarž r¤ r˜ r, r” r. r. r/ rb ª s0
zPyPIRPCLocator._get_project©r@ rA rB rC rL rd rb Ú
__classcell__r. r. r» r/ r¶ ” s r¶ c s0 e Zd ZdZ‡ fdd„Zdd„ Zdd„ Z‡ ZS )ÚPyPIJSONLocatorzw
This locator uses PyPI's JSON interface. It's very limited in functionality
and probably not worth using.
c s t t| ƒjf |Ž t|ƒ| _d S rV )r· rÄ rL r r¸ r¹ r» r. r/ rL É s zPyPIJSONLocator.__init__c C s t dƒ‚dS ©rc z Not available from this locatorNr_ rT r. r. r/ rd Í s z&PyPIJSONLocator.get_distribution_namesc
C s
i i dœ}t | jdt|ƒ ƒ}z¤| j |¡}| ¡ ¡ }t |¡}t | j
d}|d }|d |_ |d |_ |
d¡|_|
dg ¡|_|
d ¡|_t|ƒ}| |_|d
} |||j < |d
D ]T}
|
d }|j |¡ | |
¡|j|< |d
|j tƒ ¡ |¡ | |
¡|d |< qº|d
¡ D ]¤\} } | |j kr4q t | j
d}
|j |
_ | |
_ t|
ƒ}| |_||| < | D ]T}
|
d }|j |¡ | |
¡|j|< |d
| tƒ ¡ |¡ | |
¡|d |< qhq W n@ tk
r } z | j t |ƒ¡ t d|¡ W 5 d }~X Y nX |S )Nr§ z%s/jsonrš r˜ ra rƒ r¾ r¿ rÀ rœ r, r› ZreleaseszJSON fetch failed: %s) r r¸ r rH ÚopenÚreadÚdecodeÚjsonÚloadsr r5 ra rƒ rN r¾ r¿ rÀ r r£ r« r¢ r™ r› r r¡ ÚitemsrŒ rK Úputr rp Ú exception)r9 ra rQ r, ÚresprÁ r° r¥ r¤ rœ r˜ rƒ ZinfosZomdÚodistrR r. r. r/ rb Ó sT
z PyPIJSONLocator._get_projectr r. r. r» r/ rÄ Ä s rÄ c @ s` e Zd ZdZe dejejB ejB ¡Z e dejejB ¡Z
dd„ Z e dej¡Z e
dd„ ƒZd S )
ÚPagez4
This class represents a scraped HTML page.
zÝ
(rel\s*=\s*(?:"(?P[^"]*)"|'(?P[^']*)'|(?P[^>\s
]*))\s+)?
href\s*=\s*(?:"(?P[^"]*)"|'(?P[^']*)'|(?P[^>\s
]*))
(\s+rel\s*=\s*(?:"(?P[^"]*)"|'(?P[^']*)'|(?P[^>\s
]*)))?
z!]+)c C s4 || _ | | _| _| j | j ¡}|r0| d¡| _dS )zk
Initialise an instance with the Unicode page contents and the URL they
came from.
r N)rÁ r¸ r, Ú_baseÚsearchÚgroup)r9 rÁ r, r’ r. r. r/ rL s
z
Page.__init__z[^a-z0-9$&+,/:;=?@.#%_\\|-]c C s¾ dd„ }t ƒ }| j | j¡D ]Š}| d¡}|d pX|d pX|d pX|d pX|d pX|d }|d
pp|d pp|d }t| j|ƒ}t|ƒ}| j d
d„ |¡}|
||f¡ q t |dd„ dd}|S )zâ
Return the URLs of all the links on a page together with information
about their "rel" attribute, for determining which ones to treat as
downloads and which ones to queue for further scraping.
c S s, t | ƒ\}}}}}}t||t|ƒ|||fƒS )zTidy up an URL.)r r r )r, r5 rm ri r r r‘ r. r. r/ Úclean- s ÿzPage.links..cleanr3 Zrel1Zrel2Zrel3Zrel4Zrel5Zrel6rr rs Zurl3c S s dt | d¡ƒ S )Nz%%%2xr )ÚordrÓ )r’ r. r. r/ Ú; ó z Page.links..c S s | d S )Nr r. )rn r. r. r/ rÖ ? r× T)r? Úreverse) r¡ Ú_hrefÚfinditerrÁ Ú groupdictr r¸ r Ú _clean_reÚsubr¢ rª )r9 rÔ rQ r‰ r° Úrelr, r. r. r/ Úlinks& s$
ÿÿÿ z
Page.linksN)r@ rA rB rC ÚreÚcompileÚIÚSÚXrÙ rÑ rL rÜ r rß r. r. r. r/ rÐ s ü rÐ c s˜ e Zd ZdZejdd„ dd„ dœZd ‡ fdd „ Zd
d „ Zd d
„ Z dd„ Z
e de j
¡Zdd„ Zdd„ Zdd„ Zdd„ Zdd„ Ze d¡Zd d „ Z‡ ZS ) ÚSimpleScrapingLocatorzã
A locator which scrapes HTML pages to locate downloads for a distribution.
This runs multiple threads to do the I/O; performance is at least as good
as pip's PackageFinder, which works in an analogous fashion.
c C s t jttƒd ¡ S )N)Zfileobj)ÚgzipZGzipFiler r° rÇ ©Úbr. r. r/ rÖ M r× z SimpleScrapingLocator.c C s | S rV r. rç r. r. r/ rÖ N r× )Zdeflateræ ZnoneNé
c sl t t| ƒjf |Ž t|ƒ| _|| _i | _tƒ | _t
¡ | _ tƒ | _ d| _
|| _t ¡ | _t ¡ | _d| _dS )a¤
Initialise an instance.
:param url: The root URL to use for scraping.
:param timeout: The timeout, in seconds, to be applied to requests.
This defaults to ``None`` (no timeout specified).
:param num_workers: The number of worker threads you want to do I/O,
This defaults to 10.
:param kwargs: Passed to the superclass.
FN)r· rå rL r r¸ r( Ú _page_cacher¡ Ú_seenr
rJ Ú _to_fetchÚ
_bad_hostsÚskip_externalsÚ num_workersÚ threadingÚRLockÚ_lockÚ_gplockÚplatform_check)r9 r, r( rï rº r» r. r/ rL Q s
z SimpleScrapingLocator.__init__c C sF g | _ t| jƒD ]0}tj| jd}| d¡ | ¡ | j |¡ qdS )z¾
Threads are created only when get_project is called, and terminate
before it returns. They are there primarily to parallelise I/O (i.e.
fetching web pages).
)ÚtargetTN) Ú_threadsÚrangerï rð ZThreadÚ_fetchZ setDaemonÚstartrO )r9 Úirn r. r. r/ Ú_prepare_threadsl s
z&SimpleScrapingLocator._prepare_threadsc C s6 | j D ]}| j d¡ q| j D ] }| ¡ q g | _ dS )zu
Tell all the threads to terminate (by sending a sentinel value) and
wait for them to do so.
N)rö rì rÌ r~ )r9 rn r. r. r/ Ú
_wait_threadsy s
z#SimpleScrapingLocator._wait_threadsc C s’ i i dœ}| j x || _|| _t| jdt|ƒ ƒ}| j ¡ | j ¡ | ¡ z&t d|¡ | j
|¡ | j
¡ W 5 |
¡ X | `W 5 Q R X |S )Nr§ z%s/z Queueing %s)ró rQ rx r r¸ r rë rW rê rû rü rp rq rì rÌ r~ )r9 ra rQ r, r. r. r/ rb † s
z"SimpleScrapingLocator._get_projectz<\b(linux_(i\d86|x86_64|arm\w+)|win(32|_amd64)|macosx_?\d+)\bc C s | j |¡S )zD
Does an URL refer to a platform-specific download?
)Úplatform_dependentrÒ )r9 r, r. r. r/ Ú_is_platform_dependent› s z,SimpleScrapingLocator._is_platform_dependentc C sZ | j r| |¡rd}n| || j¡}t d||¡ |rV| j | | j|¡ W 5 Q R X |S )a%
See if an URL is a suitable download for a project.
If it is, register information in the result dictionary (for
_get_project) about the specific version it's for.
Note that the return value isn't actually used other than as a boolean
value.
Nzprocess_download: %s -> %s) rô rþ r– rx rp rq rò r¦ rQ )r9 r, r˜ r. r. r/ Ú_process_download¡ s
z'SimpleScrapingLocator._process_downloadc
C sÄ t |ƒ\}}}}}}| | j| j | j ¡r2d}n|| jrJ| | j¡sJd}nd| | j¡s\d}nR|dkrjd}nD|dkrxd}n6| |¡rˆd}n&| dd¡d } |
¡ dkrªd}nd}t d ||||¡ |S )
z‘
Determine whether a link URL from a referring page and with a
particular "rel" attribute should be queued for scraping.
F)ZhomepageZdownload)Zhttprf Zftpú:r r Z localhostTz#should_queue: %s (%s) from %s -> %s)
r rj r² r³ r´ rî r‡ r¸ rþ Úsplitr† rp rq )
r9 ÚlinkZreferrerrÞ r5 rm ri Ú_rQ Úhostr. r. r/ Ú
_should_queueµ s0 ÿ
ÿz#SimpleScrapingLocator._should_queuec
C sð | j ¡ }zÌz”|rž| |¡}|dkr,W W ¢®q |jD ]j\}}|| jkr2zB| j |¡ | |¡s„| |||¡r„t
d||¡ | j |¡ W q2 t k
rš Y q2X q2W n2 t
k
rÒ } z| j t|ƒ¡ W 5 d}~X Y nX W 5 | j ¡ X |s qìq dS )z×
Get a URL to fetch from the work queue, get the HTML page, examine its
links for download candidates and candidates for further scraping.
This is a handy method to run in a thread.
NzQueueing %s from %s)rì rN rP Úget_pagerß rë r¢ rÿ r rp rq rÌ r rŒ rK r )r9 r, Úpager rÞ rR r. r. r/ rø Ò s,
ÿ & z SimpleScrapingLocator._fetchc C sX t |ƒ\}}}}}}|dkr:tj t|ƒ¡r:tt|ƒdƒ}|| jkr`| j| }t d||¡ nô|
dd¡d }d}|| j krt d||¡ nÄt |d d
id }z¤zât d |¡ | j
j|| jd
} t d|¡ | ¡ }
|
dd¡} t | ¡r„| ¡ } | ¡ }
|
d¡}|r"| j| }||
ƒ}
d}t | ¡}|r@| d¡}z|
|¡}
W n tk
rn |
d¡}
Y nX t |
| ƒ}|| j| < W n¼ t k
rÄ } z |j dkr´t d||¡ W 5 d}~X Y n€ t k
r } z0t d||¡ | j! | j "|¡ W 5 Q R X W 5 d}~X Y n2 t#k
rB } zt d||¡ W 5 d}~X Y nX W 5 || j|< X |S )a
Get the HTML for an URL, possibly from an in-memory cache.
XXX TODO Note: this cache is never actually cleared. It's assumed that
the data won't get stale over the lifetime of a locator instance (not
necessarily true for the default_locator).
Úfilez
index.htmlzReturning %s from cache: %sr r r Nz Skipping %s due to bad host %szAccept-encodingZidentity)r> z Fetching %sr' z
Fetched %sz Content-Typer3 zContent-Encodingzutf-8zlatin-1i” zFetch failed: %s: %s)$r Úosri Úisdirr r r rê rp rq r rí r rH rÆ r( r˜ rN ÚHTML_CONTENT_TYPEr‰ ZgeturlrÇ ÚdecodersÚCHARSETrÒ rÓ rÈ Ú UnicodeErrorrÐ r r< rÍ r rò r¢ rŒ )r9 r, r5 rm ri r rQ r r: rÎ r> Z content_typeZ final_urlrÁ ÚencodingÚdecoderr’ rR r. r. r/ r ó sZ
&$ z SimpleScrapingLocator.get_pagez]*>([^<]+)[\w-]+)\s*\(\s*(==\s*)?(?P[^)]+)\)$c @ sL e Zd ZdZddd„Zdd„ Zdd„ Zd d
„ Zd d „ Zd
d„ Z ddd„Z
dS )ÚDependencyFinderz0
Locate dependencies for distributions.
Nc C s |pt | _t| jjƒ| _dS )zf
Initialise an instance, using the specified locator
to locate distributions.
N)Údefault_locatorr£ r" r5 r) r. r. r/ rL * s
zDependencyFinder.__init__c C sr t d|¡ |j}|| j|< || j||jf< |jD ]:}t|ƒ\}}t d|||¡ | j |t
ƒ ¡ ||f¡ q2dS )z¨
Add a distribution to the finder. This will update internal information
about who provides what.
:param dist: The distribution to add.
zadding distribution %szAdd to provided: %s, %s, %sN) rp rq r? Ú
dists_by_nameÚdistsrƒ Úprovidesr Úprovidedr r¡ r¢ )r9 r¤ ra Úprƒ r. r. r/ Úadd_distribution2 s
z!DependencyFinder.add_distributionc C sx t d|¡ |j}| j|= | j||jf= |jD ]D}t|ƒ\}}t d|||¡ | j| }| ||f¡ |s.| j|= q.dS )z°
Remove a distribution from the finder. This will update internal
information about who provides what.
:param dist: The distribution to remove.
zremoving distribution %sz Remove from provided: %s, %s, %sN)
rp rq r? r0 r1 rƒ r2 r r3 Úremove)r9 r¤ ra r4 rƒ Úsr. r. r/ Úremove_distributionA s
z$DependencyFinder.remove_distributionc C sB z| j |¡}W n, tk
r< | ¡ d }| j |¡}Y nX |S )zÞ
Get a version matcher for a requirement.
:param reqt: The requirement
:type reqt: str
:return: A version matcher (an instance of
:class:`distlib.version.Matcher`).
r )r5 rI r# r )r9 ÚreqtrI ra r. r. r/ Ú get_matcherS s z DependencyFinder.get_matcherc C st | |¡}|j}tƒ }| j}||krp|| D ]B\}}z| |¡}W n tk
rZ d}Y nX |r,| |¡ qpq,|S )zÓ
Find the distributions which can fulfill a requirement.
:param reqt: The requirement.
:type reqt: str
:return: A set of distribution which can fulfill the requirement.
F)r: r? r¡ r3 r‰ r# r¢ ) r9 r9 rI ra rQ r3 rƒ Úproviderr‰ r. r. r/ Úfind_providersc s
z DependencyFinder.find_providersc C sž | j | }tƒ }|D ]$}| |¡}| |j¡s| |¡ q|rZ| d||t|ƒf¡ d}n@| |¡ | j |= |D ]}| j |tƒ ¡ |¡ qp| |¡ d}|S )aŠ
Attempt to replace one provider with another. This is typically used
when resolving dependencies from multiple sources, e.g. A requires
(B >= 1.0) while C requires (B >= 1.1).
For successful replacement, ``provider`` must meet all the requirements
which ``other`` fulfills.
:param provider: The provider we are trying to replace with.
:param other: The provider we're trying to replace.
:param problems: If False is returned, this will contain what
problems prevented replacement. This is currently
a tuple of the literal string 'cantreplace',
``provider``, ``other`` and the set of requirements
that ``provider`` couldn't fulfill.
:return: True if we can replace ``other`` with ``provider``, else
False.
Z cantreplaceFT)
Úreqtsr¡ r: r‰ rƒ r¢ Ú frozensetr8 r r5 ) r9 r; ÚotherÚproblemsZrlistZ unmatchedr7 rI rQ r. r. r/ Útry_to_replace{ s$
ÿ
z DependencyFinder.try_to_replaceFc C sò i | _ i | _i | _i | _t|p g ƒ}d|krH| d¡ |tdddgƒO }t|tƒrh| }}t d|¡ n4| j
j ||d }}|dkrt d| ƒ‚t d |¡ d
|_
tƒ }t|gƒ}t|gƒ}|r¤| ¡ }|j} | | jkræ| |¡ n"| j| }
|
|kr| ||
|¡ |j|jB } |j} tƒ }
|r`||kr`d D ]*}d | }||kr4|
t|d
| ƒO }
q4| | B |
B }|D ].}| |¡}|sFt d|¡ | j
j ||d}|dkrÂ|sÂ| j
j |d
d}|dkrèt d|¡ | d|f¡ n^|j|j }}||f| jkr| |¡ | |¡ || krF||krF| |¡ t d|j¡ |D ]R}|j} | | jkrx| j |tƒ ¡ |¡ n"| j| }
|
|krJ| ||
|¡ qJqpq¼t| j ¡ ƒ}|D ]&}||k|_ |j r¶t d|j¡ q¶t d|¡ ||fS )aŽ
Find a distribution and all distributions it depends on.
:param requirement: The requirement specifying the distribution to
find, or a Distribution instance.
:param meta_extras: A list of meta extras such as :test:, :build: and
so on.
:param prereleases: If ``True``, allow pre-release versions to be
returned - otherwise, don't return prereleases
unless they're all that's available.
Return a set of :class:`Distribution` instances and a set of
problems.
The distributions returned should be such that they have the
:attr:`required` attribute set to ``True`` if they were
from the ``requirement`` passed to ``find()``, and they have the
:attr:`build_time_dependency` attribute set to ``True`` unless they
are post-installation dependencies of the ``requirement``.
The problems should be a tuple consisting of the string
``'unsatisfied'`` and the requirement which couldn't be satisfied
by any distribution known to the locator.
z:*:z:test:z:build:z:dev:zpassed %s as requirement)r¬ NzUnable to locate %rz
located %sT)ZtestZbuildZdevz:%s:z %s_requireszNo providers found for %rzCannot satisfy %rZ unsatisfiedzAdding %s to install_distsz#%s is a build-time dependency only.zfind done for %s) r3 r1 r0 r= r¡ r6 r# r rp rq r£ r± r Z requestedr r? r5 rA Z run_requiresZ
meta_requiresZbuild_requiresÚgetattrr< r¢ rƒ Zname_and_versionr ÚvaluesZbuild_time_dependency)r9 r¨ Z meta_extrasr¬ r¤ rÏ r@ ZtodoZ
install_distsra r? ZireqtsZsreqtsZereqtsr? rR Z all_reqtsr Z providersr; Únr r4 r1 r. r. r/ Úfind£ s’
ÿ
ÿ
ÿ zDependencyFinder.find)N)NF) r@ rA rB rC rL r5 r8 r: r< rA rE r. r. r. r/ r. % s
(r. )N)Oræ Úior rÉ Zloggingr rg rà rð Ú ImportErrorZdummy_threadingr r3 r Úcompatr r r r r r
r r r
r r r7 r r r r Zdatabaser r r rž r r Úutilr r r r r r r r r! rƒ r" r# r• r$ r% Z getLoggerr@ rp rá rˆ râ r
r r* r0 r1 ÚobjectrD r¶ rÄ rÐ rå r r r" r& r/ r± ZNAME_VERSION_REr. r. r. r. r/ Ú s^ D,
@0E: zA& [ÿü