U
fh- @ s d dl mZ d dlmZ d dlmZ d dlmZmZm Z m
Z
m Z m Z m
Z
ddlmZ ddlmZmZmZ G dd d ZG d
d d Ze eef Ze
e ZG d d
d
ZdS ) )aliases)sha256)dumps)AnyDictIteratorListOptionalTupleUnion )TOO_BIG_SEQUENCE) iana_nameis_multi_byte_encoding
unicode_rangec @ s e Zd Zd8eeeedee dddZe edddZ
e edd d
Z e ed d d
Z
ed ddZed ddZd ddddZe ed ddZe ee d ddZe ed ddZe ed ddZe ee d d d Ze ed d d Ze ed d d!Ze ed d"d#Ze ed d$d%Ze ed d&d'Ze ed d(d)Z e ed d d*d+Z e ed d,d-Z e ee d d.d/Z e ee d d0d1Z d9eed3d4d5Z!e ed d6d7Z"dS ): CharsetMatchNCoherenceMatches)payloadguessed_encodingmean_mess_ratiohas_sig_or_bom languagesdecoded_payloadc C sF || _ || _|| _|| _|| _d | _g | _d| _d | _d | _ || _
d S )N ) _payload _encoding_mean_mess_ratio
_languages_has_sig_or_bom_unicode_ranges_leavesZ_mean_coherence_ratio_output_payload_output_encoding_string)selfr r r r r r r% L/opt/alt/python38/lib64/python3.8/site-packages/charset_normalizer/models.py__init__ s zCharsetMatch.__init__)otherreturnc C s> t |ts&tdt|jt| j| j|jko<| j|jkS )Nz&__eq__ cannot be invoked on {} and {}.)
isinstancer TypeErrorformatstr __class__encoding fingerprintr$ r( r% r% r& __eq__$ s
zCharsetMatch.__eq__c C s t |tstt| j|j }t| j|j }|dk rJ|dkrJ| j|jkS |dk r|dkrt| jtkrt| j|jk S | j |j kS | j|jk S )zQ
Implemented to make sorted available upon CharsetMatches items.
g{Gz?g{Gz?)
r* r
ValueErrorabschaos coherencelenr r
multi_byte_usage)r$ r( Zchaos_differenceZcoherence_differencer% r% r& __lt__- s
zCharsetMatch.__lt__r) c C s dt t| t | j S )Ng ?)r7 r- rawr$ r% r% r& r8 C s z CharsetMatch.multi_byte_usagec C s" | j d kr t| j| jd| _ | j S )Nstrict)r# r- r r r< r% r% r& __str__G s
zCharsetMatch.__str__c C s d | j| jS )Nz )r, r/ r0 r< r% r% r& __repr__M s zCharsetMatch.__repr__c C s8 t |tr|| kr"td|jd |_| j| d S )Nz;Unable to add instance <{}> as a submatch of a CharsetMatch)r* r r3 r, r. r# r appendr1 r% r% r& add_submatchP s zCharsetMatch.add_submatchc C s | j S N)r r< r% r% r& r/ [ s zCharsetMatch.encodingc C sD g }t D ]2\}}| j|kr*|| q | j|kr || q |S )z
Encoding name are known by many name, using this could help when searching for IBM855 when it's listed as CP855.
)r itemsr/ r@ )r$ Z
also_known_asupr% r% r& encoding_aliases_ s
z CharsetMatch.encoding_aliasesc C s | j S rB r r< r% r% r& boml s zCharsetMatch.bomc C s | j S rB rG r< r% r% r& byte_order_markp s z CharsetMatch.byte_order_markc C s dd | j D S )z
Return the complete list of possible languages found in decoded sequence.
Usually not really useful. Returned list may be empty even if 'language' property return something != 'Unknown'.
c S s g | ] }|d qS )r r% ).0er% r% r&
z s z*CharsetMatch.languages..r r< r% r% r& r t s zCharsetMatch.languagesc C sp | j sbd| jkrdS ddlm}m} t| jr8|| jn|| j}t|dksVd|krZdS |d S | j d d S )z
Most probable language found in decoded sequence. If none were detected or inferred, the property will return
"Unknown".
asciiZEnglishr )encoding_languagesmb_encoding_languagesz Latin BasedUnknown)r could_be_from_charsetZcharset_normalizer.cdrO rP r r/ r7 )r$ rO rP r r% r% r& language| s
zCharsetMatch.languagec C s | j S rB )r r< r% r% r& r5 s zCharsetMatch.chaosc C s | j s
dS | j d d S )Nr r r rM r< r% r% r& r6 s zCharsetMatch.coherencec C s t | jd ddS Nd )ndigits)roundr5 r< r% r% r&
percent_chaos s zCharsetMatch.percent_chaosc C s t | jd ddS rT )rX r6 r< r% r% r& percent_coherence s z CharsetMatch.percent_coherencec C s | j S )z+
Original untouched bytes.
)r r< r% r% r& r; s zCharsetMatch.rawc C s | j S rB )r r< r% r% r& submatch s zCharsetMatch.submatchc C s t | jdkS Nr )r7 r r< r% r% r& has_submatch s zCharsetMatch.has_submatchc C s@ | j d k r| j S dd t| D }ttdd |D | _ | j S )Nc S s g | ] }t |qS r% )r )rJ charr% r% r& rL s z*CharsetMatch.alphabets..c S s h | ] }|r|qS r% r% )rJ rr% r% r& s z)CharsetMatch.alphabets..)r r- sortedlist)r$ Zdetected_rangesr% r% r& alphabets s
zCharsetMatch.alphabetsc C s | j gdd | jD S )z
The complete list of encoding that output the exact SAME str result and therefore could be the originating
encoding.
This list does include the encoding available in property 'encoding'.
c S s g | ]
}|j qS r% )r/ )rJ mr% r% r& rL s z6CharsetMatch.could_be_from_charset..)r r r< r% r% r& rR s z"CharsetMatch.could_be_from_charsetutf_8)r/ r) c C s2 | j dks| j |kr,|| _ t| |d| _| jS )z
Method to get re-encoded bytes payload using given target encoding. Default to UTF-8.
Any errors will be simply ignored by the encoder NOT replaced.
Nreplace)r" r- encoder! )r$ r/ r% r% r& output s zCharsetMatch.outputc C s t | S )zw
Retrieve the unique SHA256 computed using the transformed (re-encoded) payload. Not the original one.
)r rh hexdigestr< r% r% r& r0 s zCharsetMatch.fingerprint)N)re )#__name__
__module__ __qualname__bytesr- floatboolr r' objectr2 r9 propertyr8 r> r? rA r/ r rF rH rI r rS r5 r6 rY rZ r; r[ r] rc rR rh r0 r% r% r% r& r
sb r c @ s e Zd ZdZdeee dddZee dddZ e
e e f ed d
d Z
e dd d
ZedddZedd ddZed dddZed dddZdS )CharsetMatchesz
Container with every CharsetMatch items ordered by default from most probable to the less one.
Act like a list(iterable) but does not implements all related methods.
N)resultsc C s |r t |ng | _d S rB )ra _results)r$ rs r% r% r& r' s zCharsetMatches.__init__r: c c s | j E d H d S rB rt r< r% r% r& __iter__ s zCharsetMatches.__iter__)itemr) c C sN t |tr| j| S t |trFt|d}| jD ]}||jkr.| S q.tdS )z
Retrieve a single item either by its position or encoding name (alias may be used here).
Raise KeyError upon invalid index or encoding not present in results.
FN)r* intrt r- r rR KeyError)r$ rw resultr% r% r& __getitem__ s
zCharsetMatches.__getitem__c C s
t | jS rB r7 rt r< r% r% r& __len__ s zCharsetMatches.__len__c C s t | jdkS r\ r| r< r% r% r& __bool__ s zCharsetMatches.__bool__c C s| t |ts tdt|jt|jtkr`| j D ],}|j
|j
kr2|j |j kr2| | dS q2| j
| t| j | _ dS )z~
Insert a single match. Will be inserted accordingly to preserve sort.
Can be inserted as a submatch.
z-Cannot append instance '{}' to CharsetMatchesN)r* r r3 r, r- r. r7 r; r
rt r0 r5 rA r@ ra )r$ rw matchr% r% r& r@ s
zCharsetMatches.appendr c C s | j s
dS | j d S )zQ
Simply return the first match. Strict equivalent to matches[0].
Nr ru r< r% r% r& best s zCharsetMatches.bestc C s | S )zP
Redundant method, call the method best(). Kept for BC reasons.
)r r< r% r% r& first s zCharsetMatches.first)N)rj rk rl __doc__r r r r' r rv r rx r- r{ r} ro r~ r@ r r r% r% r% r& rr s rr c @ sj e Zd Zeee ee ee eee eeeee ed ddZe e
ee f dddZ edddZ
d S )
CliDetectionResult pathr/ rF alternative_encodingsrS rc r r5 r6 unicode_path is_preferredc C sF || _ |
| _|| _|| _|| _|| _|| _|| _|| _| | _ | | _
d S rB ) r r r/ rF r rS rc r r5 r6 r ) r$ r r/ rF r rS rc r r5 r6 r r r% r% r& r' ) s zCliDetectionResult.__init__r: c C s2 | j | j| j| j| j| j| j| j| j| j | j
d S )Nr r r< r% r% r& __dict__C s zCliDetectionResult.__dict__c C s t | jdddS )NT ) ensure_asciiindent)r r r< r% r% r& to_jsonS s zCliDetectionResult.to_jsonN)rj rk rl r- r r ro rn r' rq r r r r r% r% r% r& r ( s r N)Zencodings.aliasesr hashlibr jsonr typingr r r r r r
r Zconstantr
utilsr r r r rr r- rn ZCoherenceMatchr r r% r% r% r& s $ XC