Intro to claire-protorpc
@fanyu83
What is claire-protorpc
●
●
●
●
●
●
●

protobuf-based RPC library
non-blocking
event-driving
multi-core ready
modern C++
f...
Tutorial: echo
● In computer telecommunications, echo is the display or
return of sent data at or to the sending end of a
...
Echo Client/Server
Stubs

Services

RpcChannel

RpcServer

HttpClient

HttpServer

HttpConnection
echo.proto
● echo.proto define
message and service.
● Use protoc compile the
echo.proto
○ generate echo.pb.h
and echo.pb.c...
Echo client
● echo.pb.h define Stub
which can call Echo
service
● It need RpcChannel to
communicate remote
server.

int ma...
Echo server
● echo.pb.h define
Abstract interface
EchoService, server side
need implement it.
● EchoService need
register ...
Benefits of claire-protorpc
● claire-protorpc provide:
○
○
○
○
○
○

Define a service only need a function.
Generate stub t...
Concept
●

●
●

●

protobuf provide server declaration but no implementation, it suggest
“provide code generator plugins w...
Service
●

Services themselves are abstract interfaces (implemented either by servers
or as stubs), but they subclass this...
RpcController
● “An RpcController mediates a single method call. The
primary purpose of the controller is to provide a way...
RpcChannel
● “An RpcChannel represents a communication line to a
Service which can be used to call that Service's methods....
RpcServer
● RpcServer used to store all registered service, and
response to all request.
○ It also provide monitor, debug,...
Wire Format
message RpcMessage {
required MessageType type = 1;
required fixed64 id = 2;

4 bytes
4 bytes

meesage length
...
Architecture
protoc-rpc-gen

RpcController

RpcChannel

RpcServer

BuiltInService

gen-assets

HttpConnection

HttpClient
...
Features
●
●
●
●
●
●
●
●
●

Health Detection
LoadBalancer(Random, .etc)
Address Resolver(Dns, .etc)
Compress transport
Fla...
/flags
● claire use gflags manage configures.
● Through /flags, user can view and modify flags.
● Easy for modify flags of...
/flags
Metrics
●

●

claire-protorpc supply powerful metrics type:
○ counter
■ A counter is a value that never decreases. Example...
/couters
/histograms
/form
● /form page show all register services & method on
running server, user can post form to server as post
protobuf me...
/form
/pprof
● claire-protorpc integrate with gperftools, so
it support online profiling.
○ support /pprof/profile, /pprof/heap,...
/pprof
Question

github.com/robbinfan/claire-protorpc
Upcoming SlideShare
Loading in …5
×

Claire protorpc

1,236 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,236
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Claire protorpc

  1. 1. Intro to claire-protorpc @fanyu83
  2. 2. What is claire-protorpc ● ● ● ● ● ● ● protobuf-based RPC library non-blocking event-driving multi-core ready modern C++ for x86-64 Linux BSD license
  3. 3. Tutorial: echo ● In computer telecommunications, echo is the display or return of sent data at or to the sending end of a transmission. ------ from wikipedia ● Use claire-protorpc develop application and services should do: ○ define a .proto file which declared the rpc message structure and methods ○ use generated stub to call echo methods through RpcChannel ○ implement echo services, and register it to RpcServer
  4. 4. Echo Client/Server Stubs Services RpcChannel RpcServer HttpClient HttpServer HttpConnection
  5. 5. echo.proto ● echo.proto define message and service. ● Use protoc compile the echo.proto ○ generate echo.pb.h and echo.pb.cc // protoc --plugin=protoc-rpc-gen --rpc_out . echo.proto package echo; option cc_generic_services = true; message EchoRequest { required string str = 1; } message EchoResponse { required string str = 1; } service EchoService { rpc Echo (EchoRequest) returns (EchoResponse); }
  6. 6. Echo client ● echo.pb.h define Stub which can call Echo service ● It need RpcChannel to communicate remote server. int main(int argc, char* argv[]) { EventLoop loop; InetAddress server_address(argv[1], 8080); RpcChannel channel(&loop); channel.Connect(server_address); echo::EchoService::Stub stub(&channel); RpcControllerPtr controller(new RpcController()); echo::EchoRequest request; request.set_str("0123456789ABCDEF"); stub.Echo(controller, request, boost::bind(&replied, _1, _2)); loop.loop(); }
  7. 7. Echo server ● echo.pb.h define Abstract interface EchoService, server side need implement it. ● EchoService need register to RpcServer. int main(int argc, char* argv[]) { ::google::ParseCommandLineFlags(&argc, &argv, true); InitClaireLogging(argv[0]); EventLoop loop; InetAddress listen_address(8080); echo::EchoServiceImpl impl; RpcServer server(&loop, listen_address); server.set_num_threads(FLAGS_num_threads); server.RegisterService(&impl); server.Start(); loop.loop(); }
  8. 8. Benefits of claire-protorpc ● claire-protorpc provide: ○ ○ ○ ○ ○ ○ Define a service only need a function. Generate stub to call remote service. Automatic message encode/decode. Checksum for each message. Compress data if user set. Very good performance, at least no obvious bottleneck.
  9. 9. Concept ● ● ● ● protobuf provide server declaration but no implementation, it suggest “provide code generator plugins which generate code specific to the particular RPC implementation.” claire-protorpc is one implementation of protobuf-based rpc, and supply itself plugin to generate code for claire-protorpc only. It implement 3 important concept of protobuf rpc: ○ RpcController ○ RpcChannel ○ RpcServer But we do not use Google defined interface directly, instead claire-version!
  10. 10. Service ● Services themselves are abstract interfaces (implemented either by servers or as stubs), but they subclass this base interface. The methods of this interface can be used to call the methods of the Service without knowing its exact type at compile time (analogous to Reflection). ● claire-protorpc use protoc-rpc-gen to generate the rpc service which inherited from Service class(claire-protorpc’s version)
  11. 11. RpcController ● “An RpcController mediates a single method call. The primary purpose of the controller is to provide a way to manipulate settings specific to the RPC implementation and to find out about RPC-level errors.” ● Now it used to declare caller specific option and get error information.
  12. 12. RpcChannel ● “An RpcChannel represents a communication line to a Service which can be used to call that Service's methods. The Service may be running on another machine. Normally, you should not call an RpcChannel directly, but instead construct a stub Service wrapping it.” ● Now it used to communicate to RpcServer, it do message encode/decode, health detection, compress/uncompress, loadbalance, address resolverm, .etc .
  13. 13. RpcServer ● RpcServer used to store all registered service, and response to all request. ○ It also provide monitor, debug, profile, flags management features, user no need write one line code to use these
  14. 14. Wire Format message RpcMessage { required MessageType type = 1; required fixed64 id = 2; 4 bytes 4 bytes meesage length optional string service = 3; checksum optional string method = 4; optional bytes request = 5; RpcMessage optional bytes response = 6; optional ErrorCode error = 7; optional string reason = 8; optional CompressType compress_type = 9; }
  15. 15. Architecture protoc-rpc-gen RpcController RpcChannel RpcServer BuiltInService gen-assets HttpConnection HttpClient HttpServer Resolver LoadBalancer TcpConnection TcpClient TcpServer Inspector ProtobufIO Logging EventLoop Metrics String claire-protorpc claire-netty claire-common System Symbolizer Thread Time File gtest protobuf profiler ctemplate boost external c-ares tcmalloc snappy rapidjson gflags
  16. 16. Features ● ● ● ● ● ● ● ● ● Health Detection LoadBalancer(Random, .etc) Address Resolver(Dns, .etc) Compress transport Flags view/modify Internal statistics view/collection Online profile Methods form for test/debug All built-on Http
  17. 17. /flags ● claire use gflags manage configures. ● Through /flags, user can view and modify flags. ● Easy for modify flags of a lot of machines through /flags post method.
  18. 18. /flags
  19. 19. Metrics ● ● claire-protorpc supply powerful metrics type: ○ counter ■ A counter is a value that never decreases. Examples might be "sent bytes" or "receied bytes". You just increment the counter each time a countable event happens, and graphing utilities usually graph the deltas over time. ○ histogram ■ A metric is tracked via distribution, and is usually used for timings. claire-protorpc support graphing pages & json interface together
  20. 20. /couters
  21. 21. /histograms
  22. 22. /form ● /form page show all register services & method on running server, user can post form to server as post protobuf message ● Easy for test and debug
  23. 23. /form
  24. 24. /pprof ● claire-protorpc integrate with gperftools, so it support online profiling. ○ support /pprof/profile, /pprof/heap, /pprof/growth ○ also support profile without binary file, through /pprof/symbol & /pprof cmdlind ○ will support /pprof/contension later
  25. 25. /pprof
  26. 26. Question github.com/robbinfan/claire-protorpc

×