Re @ sB d d l Z d d l Z d d l m Z Gd d d e Z d S) N ) ProbingStatec @ s e Z d Z d Z d d d Z d d Z e d d Z d d
Z e d d Z d
d Z
e d d Z e d d Z
e d d Z d S)
CharSetProbergffffff?Nc C s( d | _ | | _ t j t | _ d S)N)_state lang_filterlogging getLogger__name__logger)selfr r /builddir/build/BUILDROOT/alt-python35-pip-20.2.4-5.el8.x86_64/opt/alt/python35/lib/python3.5/site-packages/pip/_vendor/chardet/charsetprober.py__init__' s zCharSetProber.__init__c C s t j | _ d S)N)r DETECTINGr )r r r r
reset, s zCharSetProber.resetc C s d S)Nr )r r r r
charset_name/ s zCharSetProber.charset_namec C s d S)Nr )r bufr r r
feed3 s zCharSetProber.feedc C s | j S)N)r )r r r r
state6 s zCharSetProber.statec C s d S)Ng r )r r r r
get_confidence: s z CharSetProber.get_confidencec C s t j d d | } | S)Ns ([ -])+ )resub)r r r r
filter_high_byte_only= s z#CharSetProber.filter_high_byte_onlyc C s t } t j d | } xa | D]Y } | j | d d | d d } | j rn | d k rn d } | j | q" W| S)u9
We define three types of bytes:
alphabet: english alphabets [a-zA-Z]
international: international characters [-ÿ]
marker: everything else [^a-zA-Z-ÿ]
The input buffer can be thought to contain a series of words delimited
by markers. This function works to filter all words that contain at
least one international character. All contiguous sequences of markers
are replaced by a single space ascii character.
This filter applies to all scripts which do not use English characters.
s% [a-zA-Z]*[-]+[a-zA-Z]*[^a-zA-Z-]?Nr s r r ) bytearrayr findallextendisalpha)r filteredwordsword last_charr r r
filter_international_wordsB s
z(CharSetProber.filter_international_wordsc C s t } d } d } x t t | D] } | | | d } | d k rW d } n | d k ri d } | d k r( | j r( | | k r | r | j | | | | j d | d } q( W| s | j | | d | S)
a
Returns a copy of ``buf`` that retains only the sequences of English
alphabet and high byte characters that are not between <> characters.
Also retains English alphabet and high byte characters immediately
before occurrences of >.
This filter can be applied to all scripts which contain both English
characters and extended ASCII characters, but is currently only used by
``Latin1Prober``.
Fr r > s