Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Protobuf it!

235 views

Published on

Protobuf overview. For python and other

Published in: Software
  • Be the first to comment

  • Be the first to like this

Protobuf it!

  1. 1. Protobuf it! Александр Мокров
  2. 2. Serialization / Deserialization
  3. 3. Python Go C++ C# JavaScript
  4. 4. W3C
  5. 5. Annotea CC/PP Compound Document Formats CSS DOM HTML HTTP InkML MathML OWL PICS PNG P3P RDF SMIL SPARQL Style SVG TAG Timed Text URI/URL Voice Browser WAI WAI-ARIA WebCGM Web Services
  6. 6. XML
  7. 7. <?xml version="1.0"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="note"> <xs:complexType> <xs:sequence> <xs:element name="to" type="xs:string"/> <xs:element name="from" type="xs:string"/> <xs:element name="heading" type="xs:string"/> <xs:element name="body" type="xs:string"/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema>
  8. 8. XML XPath XSLT XQuery XInclude XSDXLink XML Schema DTD XKMS XML Encryption XPointer XHTML XForms
  9. 9. Google
  10. 10. XML but smaller, faster and simpler
  11. 11. Protocol Buffers
  12. 12. C++ Java Python Objective-C C# JavaNano JavaScript Ruby Go PHP Dart
  13. 13. Twitter
  14. 14. Protobuf vs XML are simpler are 3 to 10 times smaller are 20 to 100 times faster are less ambiguous generate data access classes that are easier to use programmatically
  15. 15. message Person { required string name = 1; required int32 id = 2; optional string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { required string number = 1; optional PhoneType type = 2 [default = HOME]; } repeated PhoneNumber phone = 4; }
  16. 16. message Person { string name = 1; int32 id = 2; string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { string number = 1; PhoneType type = 2; } repeated PhoneNumber phone = 4; }
  17. 17. protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/person.proto ↓ person_pb2.py >>> from person_pb2 import Person Compiling
  18. 18. Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict( PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict( DESCRIPTOR = _PERSON_PHONENUMBER, __module__ = 'person_pb2' # @@protoc_insertion_point(class_scope:Person.PhoneNumber) )), DESCRIPTOR = _PERSON, __module__ = 'person_pb2' # @@protoc_insertion_point(class_scope:Person) ))
  19. 19. _PERSON = _descriptor.Descriptor( name='Person', full_name='Person', filename=None, file=DESCRIPTOR, containing_type=None, fields=[ _descriptor.FieldDescriptor( name='name', full_name='Person.name', index=0, number=1, type=9, cpp_type=9, label=1, has_default_value=False, default_value=_b("").decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, options=None),
  20. 20. from person_pb2 import Person person = Person() person.id = 1 person.name = "Vasya Pupkin" person.email = "vasya@example.com" phone = person.phones.add() phone.number = "+7 499 270-27-27" phone.type = Person.HOME
  21. 21. >>> person name: "Vasya Pupkin" id: 1234 email: "vasya@example.com" phones { number: "+7 499 270-27-27" type: HOME } phones { number: "2702727" type: WORK }
  22. 22. >>> data = person.SerializeToString() >>> data b'nx0cVasya Pupkinx10xd2tx1ax11vasya@example.com"x14nx10 +7 499 270-27-27x10x01"x0bnx072702727x10x02' >>> person = Person.FromString(data) >>> person = Person() >>> person.ParseFromString(data)
  23. 23. Any import "google/protobuf/any.proto"; message ErrorStatus { string message = 1; repeated google.protobuf.Any details = 2; }
  24. 24. OneOf message SampleMessage { oneof test_oneof { string name = 4; SubMessage sub_message = 9; } }
  25. 25. Map map<string, uint32> projects = 5; >>> person.projects['one'] = 1 >>> person projects { key: "one" value: 1 } >>> person.projects['one'] 1
  26. 26. Library import "google/protobuf/timestamp.proto"; import "google/protobuf/empty.proto"; message MyMessage { google.protobuf.Timestamp my_field = 1; } service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); }
  27. 27. JSON Mapping >>> from google.protobuf.json_format import MessageToJson >>> json_string = MessageToJson(person) >>> json_string '{n "name": "Vasya Pupkin",n "email": "vasya@example.com",n "phones": [n {n "number": "+7 499 270-27-27",n "type": "HOME"n },n {n "number": "2702727",n "type": "WORK"n }n ],n "id": 1234n}'
  28. 28. >>> from google.protobuf.json_format import Parse >>> person = Person() >>> Parse(json_string, person) >>> person name: "Vasya Pupkin" id: 1234 email: "vasya@example.com" ...
  29. 29. Unknown Fields message Foo { reserved 2, 15, 9 to 11; reserved "foo", "bar"; }
  30. 30. RPC service SearchService { rpc Search (SearchRequest) returns (SearchResponse); }
  31. 31. gRPC
  32. 32. ● Быстрый ● Компактный ● Позволяет формализовать данные ● Бинарный ● Лаконичный ● Поддерживается всеми распространенными языками ● Позволяет обеспечить совместимость со
  33. 33. Ссылки https://github.com/google/protobuf https://developers.google.com/protocol-buffers http://www.computerworld.com/article/2513433/app-development/twitter-solves- its-data-formatting-challenge.html
  34. 34. Спасибо за внимание!
  35. 35. Вопросы?

×