3
SLj;z @ s d Z dZddlZddlZejr"eZddlmZ ddlm Z ddl
m Z dZ e Z
e d Ze jZdd Zd
d Zed>eZed
eZed@eZedeZdd Zdd Zdd Zdd Zdd Zdd Z dd Z ee j eZ ee j eZ ee j eZ!ee j eZ"ee j ee j#Z$ee j ee j#Z%ee j&d Z'ee j(d Z)ee j&d Z*ee j(d Z+e Z,e Z-ee j ee.Z/d!d" Z0d#d$ Z1d%d& Z2d'd( Z3ej4d e j5Z6d)d* Z7d+d, Z8d-d. Z9d/d0 Z:d1d2 Z;d3d4 Zd9d: Z?d;d< Z@e@ ZAdS )Aa Code for decoding protocol buffer primitives.
This code is very similar to encoder.py -- read the docs for that module first.
A "decoder" is a function with the signature:
Decode(buffer, pos, end, message, field_dict)
The arguments are:
buffer: The string containing the encoded message.
pos: The current position in the string.
end: The position in the string where the current message ends. May be
less than len(buffer) if we're reading a sub-message.
message: The message object into which we're parsing.
field_dict: message._fields (avoids a hashtable lookup).
The decoder reads the field and stores it into field_dict, returning the new
buffer position. A decoder for a repeated field may proactively decode all of
the elements of that field, if they appear consecutively.
Note that decoders may throw any of the following:
IndexError: Indicates a truncated message.
struct.error: Unpacking of a fixed-width field failed.
message.DecodeError: Other errors.
Decoders are expected to raise an exception if they are called with pos > end.
This allows callers to be lax about bounds checking: it's fineto read past
"end" as long as you are sure that someone else will notice and throw an
exception later on.
Something up the call stack is expected to catch IndexError and struct.error
and convert them to message.DecodeError.
Decoders are constructed using decoder constructors with the signature:
MakeDecoder(field_number, is_repeated, is_packed, key, new_default)
The arguments are:
field_number: The field number of the field we want to decode.
is_repeated: Is the field a repeated field? (bool)
is_packed: Is the field a packed field? (bool)
key: The key to use when looking up the field within field_dict.
(This is actually the FieldDescriptor but nothing in this
file should depend on that.)
new_default: A function which takes a message object as a parameter and
returns a new instance of the default value for this field.
(This is called for repeated fields and sub-messages, when an
instance does not already exist.)
As with encoders, we define a decoder constructor for every type of field.
Then, for every field of every message class we construct an actual decoder.
That decoder goes into a dict indexed by tag, so when we decode a message
we repeatedly read a tag, look up the corresponding decoder, and invoke it.
z kenton@google.com (Kenton Varda) N)encoder) wire_format)messageg c s fdd}|S )a Return an encoder for a basic varint value (does not include tag).
Decoded values will be bitwise-anded with the given mask before being
returned, e.g. to limit them to 32 bits. The returned decoder does not
take the usual "end" parameter -- the caller is expected to do bounds checking
after the fact (often the caller can defer such checking until later). The
decoder returns a (value, new_pos) pair.
c sn d}d}x`t j| |}||d@ |> O }|d7 }|d@ sN| M }|}||fS |d7 }|dkr
tdq
W d S )Nr @ z$Too many bytes when decoding varint.)six
indexbytes _DecodeError)bufferposresultshiftb)mask result_type /usr/lib/python3.6/decoder.py DecodeVarintu s z$_VarintDecoder..DecodeVarintr )r r r r )r r r _VarintDecoderk s
r c s, d| d > d| > d fdd}|S )z0Like _VarintDecoder() but decodes signed values.r c sz d}d}xlt j| |}||d@ |> O }|d7 }|d@ sZ| M }|A }|}||fS |d7 }|dkr
tdq
W d S )Nr r r r r r z$Too many bytes when decoding varint.)r
r r )r
r r r r )r r signbitr r r s z*_SignedVarintDecoder..DecodeVarintr )bitsr r r )r r r r _SignedVarintDecoder s r r r c C s@ |}x t j| |d@ r |d7 }qW |d7 }t j| || |fS )a Read a tag from the buffer, and return a (tag_bytes, new_pos) tuple.
We return the raw bytes of the tag rather than decoding them. The raw
bytes can then be used to look up the proper decoder. This effectively allows
us to trade some work that would be done in pure-python (decoding a varint)
for work that is done in C (searching for a byte string in a hash table).
In a low-level language it would be much cheaper to decode the varint and
use that, but not in Python.
r r )r
r Z binary_type)r
r startr r r ReadTag s
r c s fdd}|S )zReturn a constructor for a decoder for fields of a particular type.
Args:
wire_type: The field's wire type.
decode_value: A function which decodes an individual value, e.g.
_DecodeVarint()
c sd |r t fdd}|S |rNtj| t fdd}|S fdd}|S d S )Nc s |j }|d kr"|j|}| |\}}||7 }||krHtdx$||k rl | |\}}|j| qJW ||kr|d= td|S )NzTruncated message.r z Packed element was truncated.)get
setdefaultr append)r
r endr
field_dictvalueendpointelement) decode_valuekeylocal_DecodeVarint new_defaultr r DecodePackedField s
zB_SimpleDecoder..SpecificDecoder..DecodePackedFieldc sx |j }|d kr"|j|}xP | |\}}|j| | }| || ks\||kr$||krltd|S q$W d S )NzTruncated message.)r r r! r )r
r r" r r# r$ r& new_pos)r' r( r* tag_bytestag_lenr r DecodeRepeatedField s
zD_SimpleDecoder..SpecificDecoder..DecodeRepeatedFieldc s, | |\|< }||kr(|= t d|S )NzTruncated message.)r )r
r r" r r# )r' r( r r DecodeField s
z<_SimpleDecoder..SpecificDecoder..DecodeField)
_DecodeVarintr TagByteslen) field_number is_repeated is_packedr( r* r+ r/ r0 )r' wire_type)r( r) r* r- r. r SpecificDecoder s z'_SimpleDecoder..SpecificDecoderr )r7 r' r8 r )r' r7 r _SimpleDecoder s /r9 c s fdd}t | |S )zLike SimpleDecoder but additionally invokes modify_value on every value
before storing it. Usually modify_value is ZigZagDecode.
c s | |\}}||fS )Nr )r
r r r, )r' modify_valuer r InnerDecode s z%_ModifiedDecoder..InnerDecode)r9 )r7 r' r: r; r )r' r: r _ModifiedDecoder s r< c s* t j t j fdd}t| |S )zReturn a constructor for a decoder for a fixed-width field.
Args:
wire_type: The field's wire type.
format: The format string to pass to struct.unpack().
c s& | } | || d }||fS )Nr r )r
r r, r )format local_unpack
value_sizer r r; s z'_StructPackDecoder..InnerDecode)structcalcsizeunpackr9 )r7 r= r; r )r= r> r? r _StructPackDecoder s
rC c s t j fdd} ttj| S )zReturns a decoder for a float field.
This code works around a bug in struct.unpack for non-finite 32-bit
floating-point values.
c s |d }| || }|dd dkrl|dd dkrl|dd dkrLt |fS |dd dkrdt|fS t|fS d |d }||fS )
N s r s z r r r; ) s z"_FloatDecoder..InnerDecode)r@ rB r9 r WIRETYPE_FIXED32)r; r )r> r
_FloatDecoder s rM c s t j fdd} ttj| S )zkReturns a decoder for a double field.
This code works around a bug in struct.unpack for not-a-number.
c sb |d }| || }|dd dkrL|dd dkrL|dd dkrLt |fS d|d }||fS ) N r s r s z r r r; K s z#_DoubleDecoder..InnerDecode)r@ rB r9 r WIRETYPE_FIXED64)r; r )r> r _DoubleDecoderC s rR c sp j |r&t fdd}|S |rXtjtjt fdd}|S fdd}|S d S )Nc
s |j }|d kr"|j|}| |\}}||7 }||krHtdxf||k r|}t| |\}}| jkrz|j| qJ|jsg |_tjt j
} |jj| | || f qJW ||kr| jkr|d= n|jd= td|S )NzTruncated message.r z Packed element was truncated.r r ) r r r _DecodeSignedVarint32values_by_numberr! _unknown_fieldsr r2 r WIRETYPE_VARINT)
r
r r" r r# r$ r% value_start_posr& r- ) enum_typer4 r( r) r* r r r+ e s0
z&EnumDecoder..DecodePackedFieldc s |j }|d kr"|j|}xt| |\}}| jkrH|j| n$|jsTg |_|jj| || f | }| || ks||kr$||krtd|S q$W d S )NzTruncated message.)r r rS rT r! rU r )r
r r" r r# r$ r& r, )rX r( r* r- r. r r r/ s
z(EnumDecoder..DecodeRepeatedFieldc sl |}t | |\}}||kr"td| jkr6||< n2|jsBg |_tjtj}|jj|| || f |S )NzTruncated message.) rS r rT rU r r2 r rV r! )r
r r" r r# rW Z
enum_valuer- )rX r4 r( r r r0 s
z EnumDecoder..DecodeField)rX r1 r r2 r rV r3 )r4 r5 r6 r( r* r+ r/ r0 r )rX r4 r( r) r* r- r. r EnumDecodera s rY z } zd| jf |_ W Y d d }~X nX d S )Nzutf-8z%s in field: %s)UnicodeDecodeErrorZ full_namereason)Zbyte_stre)r(
local_unicoder r _ConvertToUnicode s
z(StringDecoder.._ConvertToUnicodec s |j }|d kr"|j|}xd| |\}}|| }||krJtd|j | || | }| || ks||kr$|S q$W d S )NzTruncated string.)r r r r! )r
r r" r r# r$ sizer, )r^ r( r) r* r- r. r r r/ s
z*StringDecoder..DecodeRepeatedFieldc s> | |\}}|| }||kr&t d | || |< |S )NzTruncated string.)r )r
r r" r r# r_ r, )r^ r( r) r r r0 s z"StringDecoder..DecodeFieldN)r1 r
Z text_typer r2 r WIRETYPE_LENGTH_DELIMITEDr3 )r4 r5 r6 r( r* r/ r0 r )r^ r( r) r] r* r- r. r
StringDecoder s ra c sL t |r6tj| tjt fdd}|S fdd}|S dS )z$Returns a decoder for a bytes field.c s |j }|d kr"|j |}x`| |\}}|| }||krJtd|j| || | }| || ks|||kr$|S q$W d S )NzTruncated string.)r r r r! )r
r r" r r# r$ r_ r, )r( r) r* r- r. r r r/ s
z)BytesDecoder..DecodeRepeatedFieldc s: | |\}}|| }||kr&t d| || | < |S )NzTruncated string.)r )r
r r" r r# r_ r, )r( r) r r r0 s z!BytesDecoder..DecodeFieldN)r1 r r2 r r` r3 )r4 r5 r6 r( r* r/ r0 r )r( r) r* r- r. r BytesDecoder s rb c sd t j| tj t |rJt j| tjt fdd}|S fdd}|S dS )z$Returns a decoder for a group field.c s |j }|d kr"|j|}x|j }|d krF|j|}|j j| ||}| }| || ksx||krtd| }| || ks||kr$|S q$W d S )NzMissing group end tag.)r r add_InternalParser )r
r r" r r# r$ r, )
end_tag_bytes end_tag_lenr( r* r- r. r r r/ , s
z)GroupDecoder..DecodeRepeatedFieldc s\ |j }|d kr"|j|}|j| ||}| }| || ksP||krXtd|S )NzMissing group end tag.)r r rd r )r
r r" r r# r$ r, )re rf r( r* r r r0 A s
z!GroupDecoder..DecodeFieldN)r r2 r WIRETYPE_END_GROUPr3 WIRETYPE_START_GROUP)r4 r5 r6 r( r* r/ r0 r )re rf r( r* r- r. r GroupDecoder s ri c sN t |r6tj| tjt fdd}|S fdd}|S dS )z&Returns a decoder for a message field.c s |j }|d kr"|j |}xl| |\}}|| }||krJtd|j j| |||krhtd| }| || ks||kr$|S q$W d S )NzTruncated message.zUnexpected end-group tag.)r r r rc rd )r
r r" r r# r$ r_ r, )r( r) r* r- r. r r r/ Y s
z+MessageDecoder..DecodeRepeatedFieldc sf |j }|d kr"|j |}| |\}}|| }||krHtd|j| |||krbtd|S )NzTruncated message.zUnexpected end-group tag.)r r r rd )r
r r" r r# r$ r_ r, )r( r) r* r r r0 o s
z#MessageDecoder..DecodeFieldN)r1 r r2 r r` r3 )r4 r5 r6 r( r* r/ r0 r )r( r) r* r- r. r MessageDecoderO s rj c sN t jdtjt jdtjt jdtj ttt} fdd}|S )a Returns a decoder for a MessageSet item.
The parameter is the message Descriptor.
The message set message looks like this:
message MessageSet {
repeated group Item = 1 {
required int32 type_id = 2;
required string message = 3;
}
}
rF rE r c
s> |}d}d}d }xx| |\} }| kr8| |\}}q| kr\| |\}
}||
}}q| krhP qt | ||| }|d
krtdqW ||krtd|d krtd|d krtd|jj|} | d k r|j| } | d k r|j| | jj } | j| |||kr:tdn&|j s"g |_ |j j
t | || f |S )
Nr zMissing group end tag.zTruncated message.z MessageSet item missing type_id.z MessageSet item missing message.zUnexpected end-group tag.r r r r r r ) SkipFieldr Z
ExtensionsZ_FindExtensionByNumberr r message_type_concrete_classrd rU r! MESSAGE_SET_ITEM_TAG)
r
r r" r r# Zmessage_set_item_startZtype_idZ
message_startZ message_endr- r_ extensionr$ )item_end_tag_bytesr)
local_ReadTagmessage_tag_bytestype_id_tag_bytesr r
DecodeItem sF
z)MessageSetItemDecoder..DecodeItem) r r2 r rV r` rg r r1 rk )Z
descriptorZlocal_SkipFieldrt r )rp r) rq rr rs r MessageSetItemDecoder s 0ru c sB | t j| jtjtt| j fdd}|S )z"Returns a decoder for a map field.c s j }|j}|d kr*|j|}x| |\}}|| }||krRtd|j |j| |||krttd r||j j|j n |j||j< | }| || ks||kr,|S q,W d S )NzTruncated message.zUnexpected end-group tag.) rm r r r ZClearrd r( Z MergeFromr$ ) r
r r" r r# Zsubmsgr$ r_ r, )is_message_mapr( r) rl r* r- r. r r DecodeMap s$
z MapDecoder..DecodeMap)r r2 Znumberr r` r3 r1 rl )Zfield_descriptorr* rv rw r )rv r( r) rl r* r- r. r
MapDecoder s rx c C sB x$t | ||d d@ r$|d7 }qW |d7 }||kr>td|S )z/Skip a varint value. Returns the new position.r r zTruncated message.)ordr )r
r r" r r r _SkipVarint s rz c C s |d7 }||krt d|S )z0Skip a fixed64 value. Returns the new position.rN zTruncated message.)r )r
r r" r r r _SkipFixed64 s r{ c C s* t | |\}}||7 }||kr&td|S )z9Skip a length-delimited value. Returns the new position.zTruncated message.)r1 r )r
r r" r_ r r r _SkipLengthDelimited s
r| c C s6 x0t | |\}}t| |||}|dkr*|S |}qW dS )z*Skip sub-group. Returns the new position.r Nr )r rk )r
r r" r- r, r r r
_SkipGroup s r} c C s dS )zFSkipping an END_GROUP tag returns -1 to tell the parent loop to break.r r r )r
r r" r r r _EndGroup$ s r~ c C s |d7 }||krt d|S )z0Skip a fixed32 value. Returns the new position.rD zTruncated message.)r )r
r r" r r r _SkipFixed32) s r c C s t ddS )z;Skip function for unknown wire types. Raises an exception.zTag had invalid wire type.N)r )r
r r" r r r _RaiseInvalidWireType1 s r c s, t tttttttg tj fdd} | S )z"Constructs the SkipField function.c s$ t |dd @ } | | ||S )a Skips a field with the specified tag.
|pos| should point to the byte immediately after the tag.
Returns:
The new position (after the tag value), or -1 if the tag is an end-group
tag (in which case the calling loop should break).
r r )ry )r
r r" r- r7 )WIRETYPE_TO_SKIPPER
wiretype_maskr r rk F s z _FieldSkipper..SkipField) rz r{ r| r} r~ r r r Z
TAG_TYPE_MASK)rk r )r r r
_FieldSkipper6 s r l l l l )B__doc__
__author__r@ r
ZPY3intZlongZgoogle.protobuf.internalr r Zgoogle.protobufr rK rJ rI Z DecodeErrorr r r r1 Z_DecodeSignedVarintZ_DecodeVarint32rS r r9 r< rC rM rR rY rV Z Int32DecoderZ Int64DecoderZ
UInt32DecoderZ
UInt64DecoderZ ZigZagDecodeZ
SInt32DecoderZ
SInt64DecoderrL ZFixed32DecoderrQ ZFixed64DecoderZSFixed32DecoderZSFixed64DecoderZ FloatDecoderZ
DoubleDecoderboolZ BoolDecoderra rb ri rj r2 rh rn ru rx rz r{ r| r} r~ r r r rk r r r r O sv
;# O
.%/4J.