a
€™ReÃ$ ã @ sØ d dl mZmZmZ d dlmZmZmZ d dlm Z m
Z
m Z d dl m
Z
mZmZ d dlmZmZmZ G dd„ deƒZG dd „ d eƒZG d
d „ d eƒZG d d
„ d
eƒZG dd„ deƒZG dd„ deƒZG dd„ deƒZdS )é )ÚEUCTW_CHAR_TO_FREQ_ORDERÚEUCTW_TABLE_SIZEÚ EUCTW_TYPICAL_DISTRIBUTION_RATIO)ÚEUCKR_CHAR_TO_FREQ_ORDERÚEUCKR_TABLE_SIZEÚ EUCKR_TYPICAL_DISTRIBUTION_RATIO)ÚGB2312_CHAR_TO_FREQ_ORDERÚGB2312_TABLE_SIZEÚ!GB2312_TYPICAL_DISTRIBUTION_RATIO)ÚBIG5_CHAR_TO_FREQ_ORDERÚBIG5_TABLE_SIZEÚ BIG5_TYPICAL_DISTRIBUTION_RATIO)ÚJIS_CHAR_TO_FREQ_ORDERÚJIS_TABLE_SIZEÚ JIS_TYPICAL_DISTRIBUTION_RATIOc @ sL e Zd ZdZdZdZdZdd„ Zdd„ Zd d
„ Z d d „ Z
d
d„ Z dd„ Z dS )ÚCharDistributionAnalysisi g®Gáz®ï?g{®Gáz„?é c C s0 d | _ d | _d | _d | _d | _d | _| ¡ d S ©N)Ú_char_to_freq_orderÚ _table_sizeÚtypical_distribution_ratioÚ_doneÚ _total_charsÚ _freq_charsÚreset©Úself© r ú“/builddir/build/BUILDROOT/alt-python39-pip-21.3.1-2.el8.x86_64/opt/alt/python39/lib/python3.9/site-packages/pip/_vendor/chardet/chardistribution.pyÚ__init__. s z!CharDistributionAnalysis.__init__c C s d| _ d| _d| _dS )z reset analyser, clear any stateFé N)r r r r r r r r = s z CharDistributionAnalysis.resetc C sX |dkr| |¡}nd}|dkrT| jd7 _|| jk rTd| j| krT| jd7 _dS )z"feed a character with known lengthé éÿÿÿÿr r i N)Ú get_orderr r r r )r ÚcharÚchar_lenÚorderr r r ÚfeedF s
z CharDistributionAnalysis.feedc C sT | j dks| j| jkr | jS | j | jkrN| j| j | j | j }|| jk rN|S | jS )z(return confidence based on existing datar )r r ÚMINIMUM_DATA_THRESHOLDÚSURE_NOr ÚSURE_YES)r Úrr r r Úget_confidenceT s ÿ
z'CharDistributionAnalysis.get_confidencec C s | j | jkS r )r ÚENOUGH_DATA_THRESHOLDr r r r Úgot_enough_datad s z(CharDistributionAnalysis.got_enough_datac C s dS )Nr" r )r Úbyte_strr r r r# i s z"CharDistributionAnalysis.get_orderN)
Ú__name__Ú
__module__Ú __qualname__r- r* r) r( r r r' r, r. r# r r r r r ( s r c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚEUCTWDistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) Úsuperr3 r r r r r r r r ©Ú __class__r r r r s z"EUCTWDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr éÄ é^ r é¡ r" r ©r r/ Ú
first_charr r r r# x s z#EUCTWDistributionAnalysis.get_order©r0 r1 r2 r r# Ú
__classcell__r r r5 r r3 q s r3 c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚEUCKRDistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) r4 r> r r r r r r r r r5 r r r … s z"EUCKRDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr é° r8 r r9 r" r r: r r r r# ‹ s z#EUCKRDistributionAnalysis.get_orderr< r r r5 r r> „ s r> c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚGB2312DistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) r4 r@ r r r r r r
r r r5 r r r ˜ s z#GB2312DistributionAnalysis.__init__c C s> |d |d }}|dkr6|dkr6d|d | d S dS d S )Nr r r? r9 r8 r" r ©r r/ r; Ú second_charr r r r# ž s z$GB2312DistributionAnalysis.get_orderr< r r r5 r r@ — s r@ c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚBig5DistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) r4 rC r r r r r r
r r r5 r r r « s z!Big5DistributionAnalysis.__init__c C sX |d |d }}|dkrP|dkr:d|d | d d S d|d | d S ndS d S ) Nr r é¤ r9 é é? é@ r" r rA r r r r# ± s z"Big5DistributionAnalysis.get_orderr< r r r5 r rC ª s rC c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚSJISDistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) r4 rH r r r r r r r r r5 r r r Á s z!SJISDistributionAnalysis.__init__c C sr |d |d }}|dkr0|dkr0d|d }n&|dkrR|dkrRd|d d }nd S || d
}|d krnd }|S ) Nr r é éŸ é¼ éà éï é r" rG é r )r r/ r; rB r& r r r r# Ç s z"SJISDistributionAnalysis.get_orderr< r r r5 r rH À s rH c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚEUCJPDistributionAnalysisc s$ t t| ƒ ¡ t| _t| _t| _d S r ) r4 rP r r r r r r r r r5 r r r Ú s z"EUCJPDistributionAnalysis.__init__c C s0 |d }|dkr(d|d |d d S dS d S )Nr é r8 r9 r r" r )r r/ r$ r r r r# à s z#EUCJPDistributionAnalysis.get_orderr< r r r5 r rP Ù s rP N) Ú euctwfreqr r r Ú euckrfreqr r r Ú
gb2312freqr r r
Úbig5freqr r r
Újisfreqr r r Úobjectr r3 r> r@ rC rH rP r r r r Ú s I