This document discusses Protocol Buffers (Protobuf), an open source language-neutral, platform-neutral, extensible mechanism for serializing structured data. Protobuf data is smaller, faster to serialize and parse, and easier to work with than XML. It supports many languages and is used by companies like Google and Twitter. The document provides examples of Protobuf schema and code to serialize and parse Protobuf messages. It also summarizes additional Protobuf features like RPC services, JSON mapping, unknown fields, and libraries.

1. Protobuf it!
Александр Мокров

2. Serialization
/
Deserialization

3. Python
Go
C++
C#
JavaScript

4. W3C

5. Annotea
CC/PP
Compound Document
Formats
CSS
DOM
HTML
HTTP
InkML
MathML
OWL
PICS
PNG
P3P
RDF
SMIL
SPARQL
Style
SVG
TAG
Timed Text
URI/URL
Voice Browser
WAI
WAI-ARIA
WebCGM
Web Services

6. XML

7. <?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="note">
<xs:complexType>
<xs:sequence>
<xs:element name="to" type="xs:string"/>
<xs:element name="from" type="xs:string"/>
<xs:element name="heading" type="xs:string"/>
<xs:element name="body" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>

8. XML
XPath XSLT XQuery
XInclude
XSDXLink
XML Schema
DTD
XKMS
XML
Encryption
XPointer
XHTML
XForms

9. Google

10. XML but smaller,
faster and simpler

11. Protocol Buffers

12. C++
Java
Python
Objective-C
C#
JavaNano
JavaScript
Ruby
Go
PHP
Dart

13. Twitter

14. Protobuf vs XML
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to
use programmatically

15. message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phone = 4;
}

16. message Person {
string name = 1;
int32 id = 2;
string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
string number = 1;
PhoneType type = 2;
}
repeated PhoneNumber phone = 4;
}

17. protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/person.proto
↓
person_pb2.py
>>> from person_pb2 import Person
Compiling

18. Person = _reflection.GeneratedProtocolMessageType('Person',
(_message.Message,), dict(
PhoneNumber =
_reflection.GeneratedProtocolMessageType('PhoneNumber',
(_message.Message,), dict(
DESCRIPTOR = _PERSON_PHONENUMBER,
__module__ = 'person_pb2'
# @@protoc_insertion_point(class_scope:Person.PhoneNumber)
)),
DESCRIPTOR = _PERSON,
__module__ = 'person_pb2'
# @@protoc_insertion_point(class_scope:Person)
))

19. _PERSON = _descriptor.Descriptor(
name='Person',
full_name='Person',
filename=None,
file=DESCRIPTOR,
containing_type=None,
fields=[
_descriptor.FieldDescriptor(
name='name', full_name='Person.name', index=0,
number=1, type=9, cpp_type=9, label=1,
has_default_value=False, default_value=_b("").decode('utf-8'),
message_type=None, enum_type=None, containing_type=None,
is_extension=False, extension_scope=None, options=None),

20. from person_pb2 import Person
person = Person()
person.id = 1
person.name = "Vasya Pupkin"
person.email = "vasya@example.com"
phone = person.phones.add()
phone.number = "+7 499 270-27-27"
phone.type = Person.HOME

21. >>> person
name: "Vasya Pupkin"
id: 1234
email: "vasya@example.com"
phones {
number: "+7 499 270-27-27"
type: HOME
}
phones {
number: "2702727"
type: WORK
}

22. >>> data = person.SerializeToString()
>>> data
b'nx0cVasya
Pupkinx10xd2tx1ax11vasya@example.com"x14nx10
+7 499 270-27-27x10x01"x0bnx072702727x10x02'
>>> person = Person.FromString(data)
>>> person = Person()
>>> person.ParseFromString(data)

23. Any
import "google/protobuf/any.proto";
message ErrorStatus {
string message = 1;
repeated google.protobuf.Any details = 2;
}

24. OneOf
message SampleMessage {
oneof test_oneof {
string name = 4;
SubMessage sub_message = 9;
}
}

25. Map
map<string, uint32> projects = 5;
>>> person.projects['one'] = 1
>>> person
projects {
key: "one"
value: 1
}
>>> person.projects['one']
1

26. Library
import "google/protobuf/timestamp.proto";
import "google/protobuf/empty.proto";
message MyMessage {
google.protobuf.Timestamp my_field = 1;
}
service Foo {
rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty);
}

27. JSON Mapping
>>> from google.protobuf.json_format import MessageToJson
>>> json_string = MessageToJson(person)
>>> json_string
'{n "name": "Vasya Pupkin",n "email":
"vasya@example.com",n "phones": [n {n "number":
"+7 499 270-27-27",n "type": "HOME"n },n {n
"number": "2702727",n "type": "WORK"n }n ],n "id":
1234n}'

28. >>> from google.protobuf.json_format import Parse
>>> person = Person()
>>> Parse(json_string, person)
>>> person
name: "Vasya Pupkin"
id: 1234
email: "vasya@example.com"
...

29. Unknown Fields
message Foo {
reserved 2, 15, 9 to 11;
reserved "foo", "bar";
}

30. RPC
service SearchService {
rpc Search (SearchRequest) returns (SearchResponse);
}

31. gRPC

32. ● Быстрый
● Компактный
● Позволяет формализовать данные
● Бинарный
● Лаконичный
● Поддерживается всеми распространенными
языками
● Позволяет обеспечить совместимость со

33. Ссылки
https://github.com/google/protobuf
https://developers.google.com/protocol-buffers
http://www.computerworld.com/article/2513433/app-development/twitter-solves-
its-data-formatting-challenge.html

34. Спасибо за внимание!

35. Вопросы?