U
ʗRe @ s8 d dl Z d dlZddlmZ edZG dd dZdS ) N ) ProbingStates% [a-zA-Z]*[-]+[a-zA-Z]*[^a-zA-Z-]?c @ sn e Zd ZdZdddZdd Zedd Zd d
Zed d Z d
d Z
e dd Z e dd Z
e dd ZdS )
CharSetProbergffffff?Nc C s d | _ || _tt| _d S N)_state lang_filterlogging getLogger__name__logger)selfr r
/builddir/build/BUILDROOT/alt-python38-pip-22.2.1-2.el8.x86_64/opt/alt/python38/lib/python3.8/site-packages/pip/_vendor/chardet/charsetprober.py__init__+ s zCharSetProber.__init__c C s t j| _d S r )r DETECTINGr r r
r
r reset0 s zCharSetProber.resetc C s d S r r
r r
r
r charset_name3 s zCharSetProber.charset_namec C s t d S r )NotImplementedError)r byte_strr
r
r feed7 s zCharSetProber.feedc C s | j S r )r r r
r
r state: s zCharSetProber.statec C s dS )Ng r
r r
r
r get_confidence> s z CharSetProber.get_confidencec C s t dd| } | S )Ns ([ -])+ )resub)bufr
r
r filter_high_byte_onlyA s z#CharSetProber.filter_high_byte_onlyc C sZ t }t| }|D ]@}||dd |dd }| sJ|dk rJd}|| q|S )u7
We define three types of bytes:
alphabet: english alphabets [a-zA-Z]
international: international characters [-ÿ]
marker: everything else [^a-zA-Z-ÿ]
The input buffer can be thought to contain a series of words delimited
by markers. This function works to filter all words that contain at
least one international character. All contiguous sequences of markers
are replaced by a single space ascii character.
This filter applies to all scripts which do not use English characters.
N r ) bytearrayINTERNATIONAL_WORDS_PATTERNfindallextendisalpha)r filteredwordsword last_charr
r
r filter_international_wordsF s
z(CharSetProber.filter_international_wordsc C s t }d}d}t| d} t| D ]R\}}|dkrB|d }d}q$|dkr$||krr|sr|| || |d d}q$|s|| |d |S )
a[
Returns a copy of ``buf`` that retains only the sequences of English
alphabet and high byte characters that are not between <> characters.
This filter can be applied to all scripts which contain both English
characters and extended ASCII characters, but is currently only used by
``Latin1Prober``.
Fr c >r s