U
ʗRe3 & @ s8 d Z ddlmZ G dd dZeddddd d
d gd d
gdedddddddgddgdeddddddgddgded d d d d gd!d"gded#d$d d%d&d'gd(d)gded*d+d d%d'gd,d-gded.d/dd0d1gd2d3gded4d5d d%d'gd6gd7ed8d9dd:gd;dd d%d&d'gd?d@gdedAdBddCdDdEgdFdGgdedHdId d%d&d'gdJdKgdedLdMd d%d&d'gdNdOdPgdedQdRddSdTgdUdVgdedWdXdd d gdYdZgded[d\dd d gd]d^gded_d`d d%d&d'gdadbgdedcddddDdEdCgdedfgdedgdhddDdEdCgdidjgdedkdldddddgdmdngdedodpd d%d'gdqgd7edrdsdd d gdtdugdedvdwd d%d&d'gdxdygdedzd{d d d gd|d}gded~ddddddddgddgdeddd d d gddgdedddd d gddgdedddddddgdgdeddddddgddgdedddd:ddgddgdeddddgddgdd ZdS )z
Metadata about languages used by our model training code for our
SingleByteCharSetProbers. Could be used for other things in the future.
This code is based on the language metadata from the uchardet project.
)
ascii_lettersc s* e Zd ZdZd fdd Zdd Z ZS ) Languagea Metadata about a language useful for training models
:ivar name: The human name for the language, in English.
:type name: str
:ivar iso_code: 2-letter ISO 639-1 if possible, 3-letter ISO code otherwise,
or use another catalog as a last resort.
:type iso_code: str
:ivar use_ascii: Whether or not ASCII letters should be included in trained
models.
:type use_ascii: bool
:ivar charsets: The charsets we want to support and create data for.
:type charsets: list of str
:ivar alphabet: The characters in the language's alphabet. If `use_ascii` is
`True`, you only need to add those not in the ASCII set.
:type alphabet: str
:ivar wiki_start_pages: The Wikipedia pages to start from if we're crawling
Wikipedia for training data.
:type wiki_start_pages: list of str
NTc sn t || _|| _|| _|| _| jr<|r6|t7 }qHt}n |sHtd|r^dt t
|nd | _ || _ d S )Nz*Must supply alphabet if use_ascii is False )
super__init__nameiso_code use_asciicharsetsr
ValueErrorjoinsortedsetalphabetwiki_start_pages)selfr r r r
r r __class__ /builddir/build/BUILDROOT/alt-python38-pip-22.2.1-2.el8.x86_64/opt/alt/python38/lib/python3.8/site-packages/pip/_vendor/chardet/metadata/languages.pyr " s
zLanguage.__init__c C s. d dd | j D }| jj d| dS )Nz, c s s* | ]"\}}| d s| d|V qdS )_=N)
startswith).0kvr r r ; s
z$Language.__repr__..())r __dict__itemsr __name__)r Z param_strr r r __repr__: s
zLanguage.__repr__)NNTNNN)r!
__module__ __qualname____doc__r r"
__classcell__r r r r r
s r ArabicarFz
ISO-8859-6z WINDOWS-1256ZCP720ZCP864ub ءآأؤإئابةتثجحخدذرزسشصضطظعغػؼؽؾؿـفقكلمنهوىيًٌٍَُِّu الصفحة_الرئيسية)r r r r
r r
Belarusianbez
ISO-8859-5z WINDOWS-1251IBM866 MacCyrillicu АБВГДЕЁЖЗІЙКЛМНОПРСТУЎФХЦЧШЫЬЭЮЯабвгдеёжзійклмнопрстуўфхцчшыьэюяʼu! Галоўная_старонка BulgarianbgIBM855ux АБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЬЮЯабвгдежзийклмнопрстуфхцчшщъьюяu Начална_страницаCzechczTz
ISO-8859-2z WINDOWS-1250u< áčďéěíňóřšťúůýžÁČĎÉĚÍŇÓŘŠŤÚŮÝŽu Hlavní_stranaDanishdaz
ISO-8859-1z ISO-8859-15z WINDOWS-1252u æøåÆØÅZForsideGermandeu äöüßÄÖÜzWikipedia:HauptseiteGreekelz
ISO-8859-7z WINDOWS-1253u αβγδεζηθικλμνξοπρσςτυφχψωάέήίόύώΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΣΤΥΦΧΨΩΆΈΉΊΌΎΏu Πύλη:ΚύριαEnglishenZ Main_Page)r r r r
r Esperantoeoz
ISO-8859-3uD abcĉdefgĝhĥijĵklmnoprsŝtuŭvzABCĈDEFGĜHĤIJĴKLMNOPRSŜTUŬVZu Vikipedio:ĈefpaĝoSpanishesu ñáéíóúüÑÁÉÍÓÚÜzWikipedia:PortadaEstonianetz
ISO-8859-4z ISO-8859-13z WINDOWS-1257u6 ABDEGHIJKLMNOPRSTUVÕÄÖÜabdeghijklmnoprstuvõäöüZEsilehtFinnishfiu ÅÄÖŠŽåäöšžzWikipedia:EtusivuFrenchfru, œàâçèéîïùûêŒÀÂÇÈÉÎÏÙÛÊu Wikipédia:Accueil_principalu Bœuf (animal)Hebrewhez
ISO-8859-8z WINDOWS-1255u< אבגדהוזחטיךכלםמןנסעףפץצקרשתװױײu עמוד_ראשיCroatianhru@ abcčćdđefghijklmnoprsštuvzžABCČĆDĐEFGHIJKLMNOPRSŠTUVZŽZGlavna_stranica HungarianhuuP abcdefghijklmnoprstuvzáéíóöőúüűABCDEFGHIJKLMNOPRSTUVZÁÉÍÓÖŐÚÜŰu KezdőlapItalianitu ÀÈÉÌÒÓÙàèéìòóùZPagina_principale
LithuanianltuR AĄBCČDEĘĖFGHIĮYJKLMNOPRSŠTUŲŪVZŽaąbcčdeęėfghiįyjklmnoprsštuųūvzžZPagrindinis_puslapisLatvianlvuX AĀBCČDEĒFGĢHIĪJKĶLĻMNŅOPRSŠTUŪVZŽaābcčdeēfgģhiījkķlļmnņoprsštuūvzžu
Sākumlapa
Macedonianmku| АБВГДЃЕЖЗЅИЈКЛЉМНЊОПРСТЌУФХЦЧЏШабвгдѓежзѕијклљмнњопрстќуфхцчџшu Главна_страницаDutchnlZ HoofdpaginaPolishpluR AĄBCĆDEĘFGHIJKLŁMNŃOÓPRSŚTUWYZŹŻaąbcćdeęfghijklłmnńoóprsśtuwyzźżu Wikipedia:Strona_główna
Portugueseptu0 ÁÂÃÀÇÉÊÍÓÔÕÚáâãàçéêíóôõúu Wikipédia:Página_principalRomanianrou ăâîșțĂÂÎȘȚu Pagina_principalăRussianruzKOI8-Ru абвгдеёжзийклмнопрстуфхцчшщъыьэюяАБВГДЕЁЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯu# Заглавная_страницаSlovakskuD áäčďéíĺľňóôŕšťúýžÁÄČĎÉÍĹĽŇÓÔŔŠŤÚÝŽu Hlavná_stránkaSloveneslu8 abcčdefghijklmnoprsštuvzžABCČDEFGHIJKLMNOPRSŠTUVZŽZ Glavna_stranSerbiansrux АБВГДЂЕЖЗИЈКЛЉМНЊОПРСТЋУФХЦЧЏШабвгдђежзијклљмнњопрстћуфхцчџшu Главна_страна)r r r r
r Thaithz ISO-8859-11zTIS-620ZCP874u กขฃคฅฆงจฉชซฌญฎฏฐฑฒณดตถทธนบปผฝพฟภมยรฤลฦวศษสหฬอฮฯะัาำิีึืฺุู฿เแโใไๅๆ็่้๊๋์ํ๎๏๐๑๒๓๔๕๖๗๘๙๚๛u หน้าหลักTurkishtrz
ISO-8859-9z WINDOWS-1254uR abcçdefgğhıijklmnoöprsştuüvyzâîûABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZÂÎÛZ Ana_Sayfa
Vietnameseviz WINDOWS-1258uH aăâbcdđeêghiklmnoôơpqrstuưvxyAĂÂBCDĐEÊGHIKLMNOÔƠPQRSTUƯVXYu Chữ_Quốc_ngữ) r' r) r- r0 r2 r4 r6 r8 r: r< r> r@ rB rD rF rH rJ rL rN rP rR rT rV rX rZ r\ r^ r` rb rd rf N)r% stringr r Z LANGUAGESr r r r s 5