File: //proc/self/root/lib/python3.6/site-packages/google/protobuf/__pycache__/text_format.cpython-36.pyc
3
8��e�� � @ s d Z dZddlZddlZddlZejr*eZddlm Z ddl
m Z ddl
m Z ddd d
d gZ
e j� e j� e j� e j� fZejd ej�Zejd
ej�Zee jje jjg�ZedO�Z dZ G dd� de �Z G dd� de �Z G dd� de!�Z"dPdd�Z#dd� Z$dQdd�Z%dRd d �Z&dSd d
�Z'd d � Z(G d d!� d!e!�Z)dTd"d#�Z*dUd$d �Z+dVd%d&�Z,dWd'd(�Z-G d)d*� d*e!�Z.d+d,� Z/d-d.� Z0d/d0� Z1d1d2� Z2G d3d4� d4e!�Z3e3Z4d5d6� Z5d7d8� Z6d9d:� Z7d;d<� Z8d=d>� Z9d?d@� Z:dXdAdB�Z;dYdCdD�Z<dZdEdF�Z=d[dGdH�Z>dIdJ� Z?dKdL� Z@dMdN� ZAdS )\ab Contains routines for printing protocol messages in text format.
Simple usage example:
# Create a proto object and serialize it to a text proto string.
message = my_proto_pb2.MyMessage(foo='bar')
text_proto = text_format.MessageToString(message)
# Parse a text proto string.
message = text_format.Parse(text_proto, my_proto_pb2.MyMessage())
z kenton@google.com (Kenton Varda)� N)�
type_checkers)�
descriptor)�
text_encoding�MessageToString� PrintMessage�
PrintField�PrintFieldValue�Mergez-?inf(?:inity)?f?znanf?�'�"zgoogle.protobuf.Anyc @ s e Zd ZdZdS )�Errorz'Top-level module error for text_format.N)�__name__�
__module__� __qualname__�__doc__� r r �!/usr/lib/python3.6/text_format.pyr I s r c s2 e Zd ZdZd � fdd� Zdd� Zdd� Z� ZS )
�
ParseErrorz3Thrown in case of text parsing or tokenizing error.Nc sr |d k r:|d k r:t |�}|d k r.|dj|�7 }dj||�}|d k rTtt| �j|� ntt| �j� || _|| _d S )Nz:{0}z {0} : {1})�str�format�superr �__init__�_line�_column)�self�message�line�column�loc)� __class__r r r P s zParseError.__init__c C s | j S )N)r )r r r r �GetLine] s zParseError.GetLinec C s | j S )N)r )r r r r � GetColumn` s zParseError.GetColumn)NNN)r
r r r r r r! �
__classcell__r r )r r r M s
r c @ s, e Zd Zdd� Zdd� Zdd� Zdd� Zd S )
�
TextWriterc C s t jrtj� | _n
tj� | _d S )N)�six�PY2�io�BytesIO�_writer�StringIO)r �as_utf8r r r r f s zTextWriter.__init__c C s( t jr t|t j�r |jd�}| jj|�S )Nzutf-8)r$ r% �
isinstance� text_type�encoder( �write)r �valr r r r. l s
zTextWriter.writec C s
| j j� S )N)r( �close)r r r r r0 r s zTextWriter.closec C s
| j j� S )N)r( �getvalue)r r r r r1 u s zTextWriter.getvalueN)r
r r r r. r0 r1 r r r r r# d s r# Fc
C sL t |�}
t|
||||||||| �
} | j| � |
j� } |
j� |rH| j� S | S )a Convert protobuf message to text format.
Floating point values can be formatted compactly with 15 digits of
precision (which is the most that IEEE 754 "double" can guarantee)
using float_format='.15g'. To ensure that converting to text and back to a
proto will result in an identical value, float_format='.17g' should be used.
Args:
message: The protocol buffers message.
as_utf8: Produce text output in UTF8 format.
as_one_line: Don't introduce newlines between fields.
pointy_brackets: If True, use angle brackets instead of curly braces for
nesting.
use_index_order: If True, fields of a proto message will be printed using
the order defined in source code instead of the field number, extensions
will be printed at the end of the message and their relative order is
determined by the extension number. By default, use the field number
order.
float_format: If set, use this to specify floating point number formatting
(per the "Format Specification Mini-Language"); otherwise, str() is used.
use_field_number: If True, print field numbers instead of names.
descriptor_pool: A DescriptorPool used to resolve Any types.
indent: The indent level, in terms of spaces, for pretty print.
message_formatter: A function(message, indent, as_one_line): unicode|None
to custom format selected sub-messages (usually based on message type).
Use to pretty print parts of the protobuf for easier diffing.
Returns:
A string of the text formatted protocol buffer message.
)r# �_Printerr r1 r0 �rstrip)
r r* � as_one_line�pointy_brackets�use_index_order� float_format�use_field_number�descriptor_pool�indent�message_formatter�out�printer�resultr r r r y s (
c C s" | j tjjko | jjo | jj� jS )N)�typer �FieldDescriptor� TYPE_MESSAGE� message_typeZ has_options�
GetOptionsZ map_entry)�fieldr r r � _IsMapEntry� s rE c C s( t ||||||||| |
�
} | j| � d S )N)r2 r ) r r< r: r* r4 r5 r6 r7 r8 r9 r; r= r r r r � s c
C s& t |||||||| �}
|
j| |� dS )z%Print a single field name/value pair.N)r2 r ) rD �valuer< r: r* r4 r5 r6 r7 r; r= r r r r � s
c
C s& t |||||||| �}
|
j| |� dS )z0Print a single field value (not including name).N)r2 r ) rD rF r< r: r* r4 r5 r6 r7 r; r= r r r r � s
c C sd |dkr ddl m} |j� }ddl m} |j� }y|j| �}W n tk
rR dS X |j|�}|� S )a! Returns a protobuf message instance.
Args:
type_name: Fully-qualified protobuf message type name string.
descriptor_pool: DescriptorPool instance.
Returns:
A Message instance of type matching type_name, or None if the a Descriptor
wasn't found matching type_name.
Nr )r9 )�symbol_database)�google.protobufr9 ZDefaultrG ZFindMessageTypeByName�KeyErrorZ GetPrototype)Z type_namer9 Zpool_modrG Zdatabase�message_descriptorrB r r r �_BuildMessageFromTypeName� s
rK c @ sJ e Zd ZdZddd�Zdd� Zd d
� Zd d � Zd
d� Zdd� Z dd� Z
dS )r2 z)Text format printer for protocol message.r FNc C s@ || _ || _|| _|| _|| _|| _|| _|| _| | _|
| _ dS )aZ Initialize the Printer.
Floating point values can be formatted compactly with 15 digits of
precision (which is the most that IEEE 754 "double" can guarantee)
using float_format='.15g'. To ensure that converting to text and back to a
proto will result in an identical value, float_format='.17g' should be used.
Args:
out: To record the text format result.
indent: The indent level for pretty print.
as_utf8: Produce text output in UTF8 format.
as_one_line: Don't introduce newlines between fields.
pointy_brackets: If True, use angle brackets instead of curly braces for
nesting.
use_index_order: If True, print fields of a proto message using the order
defined in source code instead of the field number. By default, use the
field number order.
float_format: If set, use this to specify floating point number formatting
(per the "Format Specification Mini-Language"); otherwise, str() is
used.
use_field_number: If True, print field numbers instead of names.
descriptor_pool: A DescriptorPool used to resolve Any types.
message_formatter: A function(message, indent, as_one_line): unicode|None
to custom format selected sub-messages (usually based on message type).
Use to pretty print parts of the protobuf for easier diffing.
N)
r< r: r* r4 r5 r6 r7 r8 r9 r; ) r r< r: r* r4 r5 r6 r7 r8 r9 r; r r r r s %z_Printer.__init__c C sh t |j� | j�}|r`|j|j� | jjd| jd |jf � | j |� | jj| j
rVdnd� dS dS dS )z5Serializes if message is a google.protobuf.Any field.z%s[%s]� �
TFN) rK ZTypeNamer9 �MergeFromStringrF r< r. r: Ztype_url�_PrintMessageFieldValuer4 )r r Zpacked_messager r r �_TryPrintAsAnyMessage0 s
z _Printer._TryPrintAsAnyMessagec C sV | j || j| j�}|d kr dS | j}|jd| j � |j|� |j| jrLdnd� dS )NFrL rM T)r; r: r4 r< r. )r r Z formattedr< r r r �_TryCustomFormatMessage= s
z _Printer._TryCustomFormatMessagec C s� | j r| j|�rdS |jjtkr.| j|�r.dS |j� }| jrL|jdd� d� x�|D ]z\}}t |�r�xht
|�D ]$}|j � ||| d�}| j ||� qlW qR|j
tjjkr�x&|D ]}| j ||� q�W qR| j ||� qRW dS )zeConvert protobuf message to text format.
Args:
message: The protocol buffers message.
Nc S s | d j r| d jS | d jS )Nr )� is_extension�number�index)�xr r r �<lambda>V s z'_Printer.PrintMessage.<locals>.<lambda>)�key)rW rF )r; rQ �
DESCRIPTOR� full_name�_ANY_FULL_TYPE_NAMErP Z
ListFieldsr6 �sortrE �sorted�
GetEntryClassr �labelr r@ �LABEL_REPEATED)r r ZfieldsrD rF rW Z entry_submsg�elementr r r r H s$
z_Printer.PrintMessagec C s� | j }|jd| j � | jr.|jt|j�� n�|jr�|jd� |jj� j rv|j
t j j
krv|jt j jkrv|j|jj� n |j|j� |jd� n*|j
t j jkr�|j|jj� n |j|j� |jt j jkr�|jd� | j||� | jr�|jd� n
|jd� dS )z%Print a single field name/value pair.rL �[�]z: rM N)r< r. r: r8 r rS rR �containing_typerC Zmessage_set_wire_formatr? r r@ rA r^ ZLABEL_OPTIONALrB rY �
TYPE_GROUP�name�cpp_type�CPPTYPE_MESSAGEr r4 )r rD rF r< r r r r g s*
z_Printer.PrintFieldc C s� | j rd}d}nd}d}| jrF| jjd| � | j|� | jj|� nL| jjd| � | jd7 _| j|� | jd8 _| jjd| j | � d S ) N�<�>�{�}z %s z %s
� rL )r5 r4 r< r. r r: )r rF ZopenbZclosebr r r rO � s
z _Printer._PrintMessageFieldValuec C s< | j }|jtjjkr"| j|� �n|jtjjkrf|jjj |d�}|dk rV|j
|j � n|j
t |�� n�|jtjj
kr�|j
d� t|tj�r�|jd�}n|}|jtjjkr�d}n| j}|j
tj||�� |j
d� nf|jtjjkr�|r�|j
d� n
|j
d� n<|jtk�r*| jdk �r*|j
dj| j|�� n|j
t |�� dS )z�Print a single field value (not including name).
For repeated fields, the value should be a single element.
Args:
field: The descriptor of the field to be printed.
value: The value of the field.
Nr zutf-8F�true�falsez{1:{0}})r< rf r r@ rg rO Z CPPTYPE_ENUM� enum_type�values_by_number�getr. re r ZCPPTYPE_STRINGr+ r$ r, r- r? �
TYPE_BYTESr* r ZCEscapeZ CPPTYPE_BOOL� _FLOAT_TYPESr7 r )r rD rF r< �
enum_valueZ out_valueZ out_as_utf8r r r r � s2
z_Printer.PrintFieldValue) r FFFFNFNN) r
r r r r rP rQ r r rO r r r r r r2 � s
&
!r2 c C s> t | t�s&tjr | jd�} n
| jd�} t| jd�||||d�S )a� Parses a text representation of a protocol message into a message.
NOTE: for historical reasons this function does not clear the input
message. This is different from what the binary msg.ParseFrom(...) does.
Example
a = MyProto()
a.repeated_field.append('test')
b = MyProto()
text_format.Parse(repr(a), b)
text_format.Parse(repr(a), b) # repeated_field contains ["test", "test"]
# Binary version:
b.ParseFromString(a.SerializeToString()) # repeated_field is now "test"
Caller is responsible for clearing the message as needed.
Args:
text: Message text representation.
message: A protocol buffer message to merge into.
allow_unknown_extension: if True, skip over missing extensions and keep
parsing
allow_field_number: if True, both field number and field name are allowed.
descriptor_pool: A DescriptorPool used to resolve Any types.
Returns:
The same message passed as argument.
Raises:
ParseError: On text parsing problems.
zutf-8rM )r9 )r+ r r$ �PY3�decoder- �
ParseLines�split)�textr �allow_unknown_extension�allow_field_numberr9 r r r �Parse� s %
r| c C s> t | t�s&tjr | jd�} n
| jd�} t| jd�||||d�S )a^ Parses a text representation of a protocol message into a message.
Like Parse(), but allows repeated values for a non-repeated field, and uses
the last one.
Args:
text: Message text representation.
message: A protocol buffer message to merge into.
allow_unknown_extension: if True, skip over missing extensions and keep
parsing
allow_field_number: if True, both field number and field name are allowed.
descriptor_pool: A DescriptorPool used to resolve Any types.
Returns:
The same message passed as argument.
Raises:
ParseError: On text parsing problems.
zutf-8rM )r9 )r+ r r$ ru rv r- �
MergeLinesrx )ry r rz r{ r9 r r r r � s
c C s t |||d�}|j| |�S )a Parses a text representation of a protocol message into a message.
Args:
lines: An iterable of lines of a message's text representation.
message: A protocol buffer message to merge into.
allow_unknown_extension: if True, skip over missing extensions and keep
parsing
allow_field_number: if True, both field number and field name are allowed.
descriptor_pool: A DescriptorPool used to resolve Any types.
Returns:
The same message passed as argument.
Raises:
ParseError: On text parsing problems.
)r9 )�_Parserrw )�linesr rz r{ r9 �parserr r r rw s rw c C s t |||d�}|j| |�S )a Parses a text representation of a protocol message into a message.
Args:
lines: An iterable of lines of a message's text representation.
message: A protocol buffer message to merge into.
allow_unknown_extension: if True, skip over missing extensions and keep
parsing
allow_field_number: if True, both field number and field name are allowed.
descriptor_pool: A DescriptorPool used to resolve Any types.
Returns:
The same message passed as argument.
Raises:
ParseError: On text parsing problems.
)r9 )r~ r} )r r rz r{ r9 r� r r r r} 6 s r} c @ sn e Zd ZdZddd�Zdd� Zdd � Zd
d � Zd d
� Zdd� Z dd� Z
dd� Z dd� Z e
dd� �Zdd� ZdS )r~ z(Text format parser for protocol message.FNc C s || _ || _|| _d S )N)rz r{ r9 )r rz r{ r9 r r r r T s z_Parser.__init__c C s& t |t�s|jd�}| j|jd�|�S )zBParses a text representation of a protocol message into a message.zutf-8rM )r+ r rv rw rx )r ry r r r r �ParseFromString\ s
z_Parser.ParseFromStringc C s d| _ | j||� |S )zBParses a text representation of a protocol message into a message.F)�_allow_multiple_scalars�
_ParseOrMerge)r r r r r r rw b s z_Parser.ParseLinesc C s | j |jd�|�S )zBMerges a text representation of a protocol message into a message.rM )Z _MergeLinesrx )r ry r r r r rN h s z_Parser.MergeFromStringc C s d| _ | j||� |S )zBMerges a text representation of a protocol message into a message.T)r� r� )r r r r r r r} l s z_Parser.MergeLinesc C s& t |�}x|j� s | j||� q
W dS )z�Converts a text representation of a protocol message into a message.
Args:
lines: Lines of a message's text representation.
message: A protocol buffer message to merge into.
Raises:
ParseError: On text parsing problems.
N)� Tokenizer�AtEnd� _MergeField)r r r � tokenizerr r r r� r s
z_Parser._ParseOrMergec
C s` |j }|jtkr�|jd�r�| j|�\}}|jd� |jd� |jd�rLd}n|jd� d}t|| j�}|svtd| ��x2|j|�s�|j � r�|j
d |f ��| j ||� qxW |j ||d
� d S |jd��rn|j
� g}x |jd �r�|j|j
� � q�W d j|�}|j�s|j
d
|j ��|jj|�} | �sB| j�r2d } n|j
d| ��n || jk�rb|j
d||jf ��|jd� n�|j� }| j�r�|j� �r�t|dd�}
|jj|
d �} | �r(|j�r(|jj|
�} nh|j j|d �} | �s |j j|j � d �} | �r | j t j j!k�r d } | �r(| j t j j!k�r(| j"j#|k�r(d } | �sB|j
d|j|f ��| �r2| j$ �r�| j%�r�|j&| j%j#�} | d k �r�| | j#k�r�|j
d| j#| | j%j#|jf ��| j't j j(k�r�|jd� | j)} n|jd� | j*} | j+t j j,k�r$|jd��r$|jd��s0x8| ||| � |jd��rP |jd� �q�W n | ||| � n| j�s>t-�t.|� |jd��s\|jd� d S )z�Merges a single protocol message field into a message.
Args:
tokenizer: A tokenizer to parse the field name and values.
message: A protocol message to record the data.
Raises:
ParseError: In case of text parsing problems.
ra rb �:rh ri rj rk z$Type %s not found in descriptor poolzExpected "%s".)�type_url_prefixN�.z+Message type "%s" does not have extensions.z�Extension "%s" not registered. Did you import the _pb2 module which defines it? If you are trying to place the extension in the MessageSet field of another message that is in an Any or MessageSet field, that message's _pb2 module must be imported as wellz1Extension "%s" does not extend message type "%s".Tz*Message type "%s" has no field named "%s".zbField "%s" is specified along with field "%s", another member of oneof "%s" for message type "%s".�,�;)/rX rY rZ �
TryConsume�_ConsumeAnyTypeUrl�ConsumerK r9 r r� �ParseErrorPreviousTokenr� ZPack�ConsumeIdentifier�append�joinZ
is_extendable�
ExtensionsZ_FindExtensionByNamerz rc �ConsumeIdentifierOrNumberr{ �isdigit� ParseIntegerZfields_by_numberrq Z_FindExtensionByNumber�fields_by_name�lowerr? r r@ rd rB re r� Zcontaining_oneofZ
WhichOneofrf rg �_MergeMessageField�_MergeScalarFieldr^ r_ �AssertionError�_SkipFieldContents)
r r� r rJ r� Zpacked_type_nameZexpanded_any_end_tokenZexpanded_any_sub_messagere rD rS Z which_oneofZmergerr r r r� � s�
z_Parser._MergeFieldc C s� |j � g}|jd� |j|j � � |jd� |j|j � � |jd� |j � g}x |jd�rj|j|j � � qPW dj|�dj|�fS )zBConsumes a google.protobuf.Any type URL and returns the type name.r� �/)r� r� r� r� r� )r r� �prefixre r r r r� s
z_Parser._ConsumeAnyTypeUrlc C s� t |�}|jd�rd}n|jd� d}|jtjjkrt|jrJ|j| j � }q�|rbt
||j �j � � }q�t
||j �j � }n||jr�| j
r�|j|�r�|jd|jj|jf ��|j| }n8| j
r�|j|j �r�|jd|jj|j f ��t
||j �}|j� x6|j|��s&|j� �r|jd|f ��| j||� q�W |�r~|jjd j}|tjjk�rjt
||j �|j }|j|j � n|j t
||j �|j< d S )
a" Merges a single scalar field into a message.
Args:
tokenizer: A tokenizer to parse the field value.
message: The message of which field is a member.
field: The descriptor of the field to be merged.
Raises:
ParseError: In case of text parsing problems.
rh ri rj rk z;Message type "%s" should not have multiple "%s" extensions.z7Message type "%s" should not have multiple "%s" fields.zExpected "%s".rF N) rE r� r� r^ r r@ r_ rR r� �add�getattrre r] r� � HasExtensionr� rX rY �HasFieldZ SetInParentr� r� rB r� rf rg rW Z MergeFromrF ) r r� r rD Z is_map_entryZ end_tokenZ sub_messageZ
value_cpptyperF r r r r� sF
z_Parser._MergeMessageFieldc C s | j }t|d�o|jdkS )N�syntax�proto3)rX �hasattrr� )r rJ r r r �_IsProto3SyntaxO s
z_Parser._IsProto3Syntaxc C s | j }d}|jtjjtjjtjjfkr0t|�}n�|jtjjtjj tjj
fkrVt |�}n�|jtjj tjj
fkrvt|�}n�|jtjjtjjfkr�t|�}n�|jtjjtjjfkr�|j� }nr|jtjjkr�|j� }nZ|jtjjkr�|j� }nB|jtjjkr�|j� }n*|jtjjk�r|j |�}nt d|j ��|j tjj k�rf|j �rR|j!| j"|� nt#||j$�j"|� n�| j%|� }|j �r�| j& �r�|�r�|j'|��r�|j(d|j)j*|j*f ��n
||j!|< nF| j& �r�|�r�|j+|j$��r�|j(d|j)j*|j$f ��nt,||j$|� dS )aH Merges a single scalar field into a message.
Args:
tokenizer: A tokenizer to parse the field value.
message: A protocol message to record the data.
field: The descriptor of the field to be merged.
Raises:
ParseError: In case of text parsing problems.
RuntimeError: On runtime errors.
NzUnknown field type %dz;Message type "%s" should not have multiple "%s" extensions.z7Message type "%s" should not have multiple "%s" fields.)-rz r? r r@ Z
TYPE_INT32Z TYPE_SINT32Z
TYPE_SFIXED32�
_ConsumeInt32Z
TYPE_INT64Z TYPE_SINT64Z
TYPE_SFIXED64�
_ConsumeInt64Z TYPE_UINT32Z TYPE_FIXED32�_ConsumeUint32Z TYPE_UINT64Z TYPE_FIXED64�_ConsumeUint64Z
TYPE_FLOATZ TYPE_DOUBLE� ConsumeFloatZ TYPE_BOOL� ConsumeBoolZ TYPE_STRING�
ConsumeStringrr �ConsumeByteStringZ TYPE_ENUM� ConsumeEnum� RuntimeErrorr^ r_ rR r� r� r� re r� r� r� r� rX rY r� �setattr)r r� r rD �_rF Zcan_check_presencer r r r� U s\
z_Parser._MergeScalarField)FFN)r
r r r r r� rw rN r} r� r� r� r� � staticmethodr� r� r r r r r~ Q s
: r~ c C s8 | j d�r,| jd� r,| jd� r,t| � nt| � dS )z}Skips over contents (value or message) of a field.
Args:
tokenizer: A tokenizer to parse the field name and values.
r� rj rh N)r� � LookingAt�_SkipFieldValue�_SkipFieldMessage)r� r r r r� � s
r� c C s^ | j d�r6| j� x| j d�r(| j� qW | jd� n| j� t| � | j d�sZ| j d� dS )z�Skips over a complete field (name and value/message).
Args:
tokenizer: A tokenizer to parse the field name and values.
ra r� rb r� r� N)r� r� r� r� r� )r� r r r �
_SkipField� s
r� c C sR | j d�rd}n| jd� d}x$| jd� rB| jd� rBt| � q W | j|� dS )zfSkips over a field message.
Args:
tokenizer: A tokenizer to parse the field name and values.
rh ri rj rk N)r� r� r� r� )r� Z delimiterr r r r� � s
r� c C sT | j � rx | j � rq
W dS | j� rPt| � rPt| � rP| j� rPtd| j ��dS )z�Skips over a field value.
Args:
tokenizer: A tokenizer to parse the field name and values.
Raises:
ParseError: In case an invalid field value is found.
NzInvalid field value: )�TryConsumeByteString�TryConsumeIdentifier�_TryConsumeInt64�_TryConsumeUint64�TryConsumeFloatr �token)r� r r r r� � s
r� c @ sH e Zd ZdZejd�Zejdej�Zejdej�Z ejdj
ddgdd � e D � ��Z ejd
�Z
ejd �ZdEd
d�Zdd� Zdd� Zdd� Zdd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Z dFd*d+�Z d,d-� Z d.d/� Z d0d1� Z d2d3� Z!d4d5� Z"d6d7� Z#d8d9� Z$d:d;� Z%d<d=� Z&d>d?� Z'd@dA� Z(dBdC� Z)dDS )Gr� z�Protocol buffer text representation tokenizer.
This class handles the lower level string parsing by splitting it into
meaningful tokens.
It was directly ported from the Java protocol buffer API.
z\s+z (\s*#.*$)z (\s|(#.*$))+�|z[a-zA-Z_][0-9a-zA-Z_+-]*z#([0-9+-]|(\.[0-9]))[0-9a-zA-Z_.+-]*c C s g | ]}d j |d��qS )z!{qt}([^{qt}\n\\]|\\.)*({qt}|\\?$))Zqt)r )�.0Zmarkr r r �
<listcomp>� s zTokenizer.<listcomp>z
[^\d\W]\w*z\w+Tc C sl d| _ d| _d| _d | _d| _t|�| _d| _d| _d| _ d| _
|| _ |rP| j pT| j
| _| j� | j� d S )Nr � � T���)Z _positionr r Z _token_startr� �iter�_lines�
_current_line�_previous_line�_previous_column� _more_lines�_skip_comments�_WHITESPACE_OR_COMMENT� _WHITESPACE�_whitespace_pattern�_SkipWhitespace� NextToken)r r Z
skip_commentsr r r r s
zTokenizer.__init__c C s
| j |kS )N)r� )r r� r r r r� s zTokenizer.LookingAtc C s | j S )z^Checks the end of the text was reached.
Returns:
True iff the end was reached.
)r� )r r r r r� s zTokenizer.AtEndc C s` xZt | j�| jkrZyt| j�| _W n tk
rB d| _d| _d S X | jd7 _d| _qW d S )Nr� Fr� r )�lenr� r �nextr� �
StopIterationr� r )r r r r �_PopLine s zTokenizer._PopLinec C sF x@| j � | jj| j| j�}|s"P t|jd��}| j|7 _qW d S )Nr )r� r� �matchr� r r� �group)r r� Zlengthr r r r� , s zTokenizer._SkipWhitespacec C s | j |kr| j� dS dS )z�Tries to consume a given piece of text.
Args:
token: Text to consume.
Returns:
True iff the text was consumed.
TF)r� r� )r r� r r r r� 5 s
zTokenizer.TryConsumec C s | j |�s| jd| ��dS )z�Consumes a piece of text.
Args:
token: Text to consume.
Raises:
ParseError: If the text couldn't be consumed.
zExpected "%s".N)r� r )r r� r r r r� C s
zTokenizer.Consumec C s( | j }| jj|�s | jd��| j� |S )NzExpected comment.)r� �_COMMENTr� r r� )r r> r r r �ConsumeCommentO s
zTokenizer.ConsumeCommentc C s: | j dko| jdk}| j}| j� }| j|ko0| }||fS )zCConsumes a comment, returns a 2-tuple (trailing bool, comment str).r )r r r� r� )r Z just_startedZbefore_parsingZcommentZtrailingr r r � ConsumeCommentOrTrailingCommentV s
z)Tokenizer.ConsumeCommentOrTrailingCommentc C s& y | j � dS tk
r dS X d S )NTF)r� r )r r r r r� f s
z Tokenizer.TryConsumeIdentifierc C s( | j }| jj|�s | jd��| j� |S )z�Consumes protocol message field identifier.
Returns:
Identifier string.
Raises:
ParseError: If an identifier couldn't be consumed.
zExpected identifier.)r� � _IDENTIFIERr� r r� )r r> r r r r� m s
zTokenizer.ConsumeIdentifierc C s& y | j � dS tk
r dS X d S )NTF)r� r )r r r r � TryConsumeIdentifierOrNumber| s
z&Tokenizer.TryConsumeIdentifierOrNumberc C s, | j }| jj|�s | jd| ��| j� |S )z�Consumes protocol message field identifier.
Returns:
Identifier string.
Raises:
ParseError: If an identifier couldn't be consumed.
z&Expected identifier or number, got %s.)r� �_IDENTIFIER_OR_NUMBERr� r r� )r r> r r r r� � s
z#Tokenizer.ConsumeIdentifierOrNumberc C s& y | j � dS tk
r dS X d S )NTF)�ConsumeIntegerr )r r r r �TryConsumeInteger� s
zTokenizer.TryConsumeIntegerFc C sP yt | j|d�}W n0 tk
rB } z| jt|���W Y dd}~X nX | j� |S )z�Consumes an integer number.
Args:
is_long: True if the value should be returned as a long integer.
Returns:
The integer parsed.
Raises:
ParseError: If an integer couldn't be consumed.
)�is_longN)�_ParseAbstractIntegerr� �
ValueErrorr r r� )r r� r> �er r r r� � s zTokenizer.ConsumeIntegerc C s& y | j � dS tk
r dS X d S )NTF)r� r )r r r r r� � s
zTokenizer.TryConsumeFloatc C sL yt | j�}W n0 tk
r> } z| jt|���W Y dd}~X nX | j� |S )z�Consumes an floating point number.
Returns:
The number parsed.
Raises:
ParseError: If a floating point number couldn't be consumed.
N)�
ParseFloatr� r� r r r� )r r> r� r r r r� � s zTokenizer.ConsumeFloatc C sL yt | j�}W n0 tk
r> } z| jt|���W Y dd}~X nX | j� |S )z�Consumes a boolean value.
Returns:
The bool parsed.
Raises:
ParseError: If a boolean value couldn't be consumed.
N)� ParseBoolr� r� r r r� )r r> r� r r r r� � s zTokenizer.ConsumeBoolc C s& y | j � dS tk
r dS X d S )NTF)r� r )r r r r r� � s
z Tokenizer.TryConsumeByteStringc C sF | j � }y tj|d�S tk
r@ } z| j|��W Y dd}~X nX dS )z�Consumes a string value.
Returns:
The string parsed.
Raises:
ParseError: If a string value couldn't be consumed.
zutf-8N)r� r$ r, �UnicodeDecodeError�_StringParseError)r Z the_bytesr� r r r r� � s
zTokenizer.ConsumeStringc C s< | j � g}x&| jr0| jd tkr0|j| j � � q W dj|�S )z�Consumes a byte array value.
Returns:
The array parsed (as a string).
Raises:
ParseError: If a byte array value couldn't be consumed.
r � )�_ConsumeSingleByteStringr� �_QUOTESr� r� )r Zthe_listr r r r� � s
zTokenizer.ConsumeByteStringc C s� | j }t|�dk s |d tkr.| jd|f ��t|�dk sJ|d |d krZ| jd|f ��ytj|dd� �}W n0 tk
r� } z| jt|���W Y dd}~X nX | j� |S ) aO Consume one token of a string literal.
String literals (whether bytes or text) can come in multiple adjacent
tokens which are automatically concatenated, like in C or Python. This
method only consumes one token.
Returns:
The token parsed.
Raises:
ParseError: When the wrong format data is found.
r� r z Expected string but found: %rrl z String missing ending quote: %rNr� r� ) r� r� r� r r Z CUnescaper� r r� )r ry r> r� r r r r� � s z"Tokenizer._ConsumeSingleByteStringc C sN yt || j�}W n0 tk
r@ } z| jt|���W Y d d }~X nX | j� |S )N)� ParseEnumr� r� r r r� )r rD r> r� r r r r� s zTokenizer.ConsumeEnumc C s t || jd | jd �S )z�Creates and *returns* a ParseError for the previously read token.
Args:
message: A message to set for the exception.
Returns:
A ParseError instance.
r� )r r� r� )r r r r r r� s z!Tokenizer.ParseErrorPreviousTokenc C s t || jd | jd �S )z9Creates and *returns* a ParseError for the current token.r� )r r r )r r r r r r % s zTokenizer.ParseErrorc C s | j dt|� �S )NzCouldn't parse string: )r r )r r� r r r r� ) s zTokenizer._StringParseErrorc C s� | j | _| j| _| jt| j�7 _| j� | js<d| _dS | jj | j
| j�}| rn| j rn| j j | j
| j�}|r�|j
d�}|| _n| j
| j | _dS )z Reads the next meaningful token.r� Nr )r r� r r� r� r� r� r� �_TOKENr� r� r� r� r� )r r� r� r r r r� , s
zTokenizer.NextTokenN)T)F)*r
r r r �re�compiler� � MULTILINEr� r� r� r� r� r� r� r r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r� r� r r r r r� � sJ
r� c C s t | ddd�S )z�Consumes a signed 32bit integer number from tokenizer.
Args:
tokenizer: A tokenizer used to parse the number.
Returns:
The integer parsed.
Raises:
ParseError: If a signed 32bit integer couldn't be consumed.
TF)� is_signedr� )�_ConsumeInteger)r� r r r r� F s r� c C s t | ddd�S )z�Consumes an unsigned 32bit integer number from tokenizer.
Args:
tokenizer: A tokenizer used to parse the number.
Returns:
The integer parsed.
Raises:
ParseError: If an unsigned 32bit integer couldn't be consumed.
F)r� r� )r� )r� r r r r� U s r� c C s& y t | � dS tk
r dS X d S )NTF)r� r )r� r r r r� d s
r� c C s t | ddd�S )z�Consumes a signed 32bit integer number from tokenizer.
Args:
tokenizer: A tokenizer used to parse the number.
Returns:
The integer parsed.
Raises:
ParseError: If a signed 32bit integer couldn't be consumed.
T)r� r� )r� )r� r r r r� l s r� c C s& y t | � dS tk
r dS X d S )NTF)r� r )r� r r r r� { s
r� c C s t | ddd�S )z�Consumes an unsigned 64bit integer number from tokenizer.
Args:
tokenizer: A tokenizer used to parse the number.
Returns:
The integer parsed.
Raises:
ParseError: If an unsigned 64bit integer couldn't be consumed.
FT)r� r� )r� )r� r r r r� � s r� c C s, yt | ||d� dS tk
r& dS X d S )N)r� r� TF)r� r )r� r� r� r r r �_TryConsumeInteger� s
r� c C sR yt | j||d�}W n0 tk
rD } z| jt|���W Y dd}~X nX | j� |S )aU Consumes an integer number from tokenizer.
Args:
tokenizer: A tokenizer used to parse the number.
is_signed: True if a signed integer must be parsed.
is_long: True if a long integer must be parsed.
Returns:
The integer parsed.
Raises:
ParseError: If an integer with given characteristics couldn't be consumed.
)r� r� N)r� r� r� r r r� )r� r� r� r> r� r r r r� � s r� c C s2 t | |d�}tdt|� t|� }|j|� |S )a Parses an integer.
Args:
text: The text to parse.
is_signed: True if a signed integer must be parsed.
is_long: True if a long integer must be parsed.
Returns:
The integer value.
Raises:
ValueError: Thrown Iff the text is not a valid integer.
)r� rl )r� �_INTEGER_CHECKERS�intZ
CheckValue)ry r� r� r> Zcheckerr r r r� � s
r� c C sB y |rt | d�S t| d�S W n tk
r< td| ��Y nX dS )a Parses an integer without checking size/signedness.
Args:
text: The text to parse.
is_long: True if the value should be returned as a long integer.
Returns:
The integer value.
Raises:
ValueError: Thrown Iff the text is not a valid integer.
r zCouldn't parse integer: %sN)�longr� r� )ry r� r r r r� � s
r� c C s� yt | �S tk
r� tj| �r@| d dkr6t d�S t d�S nBtj| �rRt d�S yt | jd��S tk
r� td| ��Y nX Y nX dS ) z�Parse a floating point number.
Args:
text: Text to parse.
Returns:
The number parsed.
Raises:
ValueError: If a floating point number couldn't be parsed.
r �-z-inf�inf�nan�fzCouldn't parse float: %sN)�floatr� �_FLOAT_INFINITYr� �
_FLOAT_NANr3 )ry r r r r� � s
r� c C s$ | d
kr dS | dkrd
S t d ��d S )z�Parse a boolean value.
Args:
text: Text to parse.
Returns:
Boolean values parsed
Raises:
ValueError: If text is not a valid boolean.
rm �t�1�TrueTrn r� �0�FalseFzExpected "true" or "false".N)rm r� r� r� )rn r� r� r )r� )ry r r r r� s
r� c C s� | j }yt|d�}W n< tk
rP |jj|d�}|dkrLtd|j|f ��Y nFX t| jd�rn| jjdkrn|S |j j|d�}|dkr�td|j|f ��|j
S )a& Parse an enum value.
The value can be specified by a number (the enum value), or by
a string literal (the enum name).
Args:
field: Enum field descriptor.
value: String value.
Returns:
Enum value number.
Raises:
ValueError: If the enum value could not be parsed.
r Nz%Enum type "%s" has no value named %s.r� r� z+Enum type "%s" has no value with number %d.) ro r� r� Zvalues_by_namerq rY r� �filer� rp rS )rD rF Zenum_descriptorrS rt r r r r� s r� )r
r ) FFFFNFNr N) r FFFFNFNN)r FFFFNN)r FFFFNN)FFN)FFN)FFN)FFN)FF)FF)FF)F)Br �
__author__r& r� r$ ru r� r� Zgoogle.protobuf.internalr rH r r �__all__ZUint32ValueCheckerZInt32ValueCheckerZUint64ValueCheckerZInt64ValueCheckerr� r� �
IGNORECASEr� r� � frozensetr@ Z
CPPTYPE_FLOATZCPPTYPE_DOUBLErs r� rZ � Exceptionr r �objectr# r rE r r r rK r2 r| r rw r} r~ r� r� r� r� r� Z
_Tokenizerr� r� r� r� r� r� r� r� r� r� r� r� r� r r r r �<module>) s�
+
K
/
#
L W