Download to read offline
Uploaded byKenneth Sørensen
Salesforce Developer User Group, Oslo, Norway - Salesforce PubSub API and gRPC.pptx
The document covers an introduction to gRPC and the Salesforce Pub/Sub API, highlighting their capabilities and benefits, particularly in real-time data processing and event-driven architectures. It details the advantages of HTTP/2 in gRPC, the integration of Apache Avro for efficient data serialization, and provides an overview of a Java quick start for implementing the Salesforce Pub/Sub API. Overall, it emphasizes the importance of these technologies in building scalable, high-performance distributed applications.
Salesforce Developer User Group, Oslo, Norway - Salesforce PubSub API and gRPC.pptx
- 1. Developer User Group,Oslo, Norway SF. Pub/Sub API and gRPC Kenneth Sørensen kenneth.sorensen@trailblazercgl.com @kensor81
- 2. Kenneth Sørensen Tech LeadSaaS Development @ NAV I am a Salesforce Certified Application Architect with over a decade of IT experience, specializing in Salesforce development and integration. Currently leading NAV's Salesforce Platform Team – Team Platforce. I also contribute to the Salesforce Developer User Group in Oslo, focusing on practical implementations and community learning. Passionate about cloud innovation, I aim to design scalable solutions that improve internal processes and customer experiences.
- 3. Introduction to gRPC and SalesforcePub/Sub API
- 4. What is gRPC? KeyPoints: ● gRPC is a modern, high-performance Remote Procedure Call (RPC) framework created by Google. ● It uses Protocol Buffers (protobuf) to serialize data, making it efficient in terms of speed and bandwidth usage. ● Supports multiple languages and platforms, making it highly versatile.
- 5. HTTP/2 in gRPC:Enhancing Real-Time Communication What is HTTP/2? ● A major upgrade over HTTP/1.1, designed for faster, more efficient communication. ● Key improvements: multiplexing, binary protocol, header compression, server push, and stream prioritization.
- 6. HTTP/2 in gRPC:Enhancing Real-Time Communication How HTTP/2 Enhances gRPC: ● Multiplexing: Multiple RPC calls over a single connection without delays. ● Binary Protocol: Reduces overhead and speeds up message parsing. ● Bi-Directional Streaming: Enables real-time, continuous communication between client and server. ● Low-Latency Communication: Ideal for real-time applications like microservices and live streaming.
- 7. HTTP/2 in gRPC:Enhancing Real-Time Communication Benefits for gRPC: ● Concurrency: Multiple streams on a single connection improve scalability. ● Real-Time Data: Enables fast, bidirectional communication for real-time interactions. ● Efficiency: Reduces resource usage and increases speed for distributed systems.
- 8. Salesforce Pub/Sub APIOverview Key Capabilities: ● Salesforce Pub/Sub API facilitates the publishing and subscribing to platform events, ensuring real-time data distribution across Salesforce and other services. ● Supports scalable, high-volume event processing with millions of events per second. ● Built using gRPC for highly efficient communication.
- 9. Apache Avro inSalesforce Pub/Sub API What is Apache Avro? ● A schema-based serialization system used for efficient data exchange in distributed systems.
- 10. Apache Avro inSalesforce Pub/Sub API Why Avro? ● Schema-Based Serialization: Uses schemas to ensure consistent structure across different platforms. ● Compact Data Encoding: Encodes data in binary format, reducing message size and increasing performance. ● Schema Evolution: Supports backward compatibility, allowing changes to event structures without breaking older versions. ● Interoperability: Language-agnostic, making it ideal for communication between systems in different programming languages.
- 11. Apache Avro inSalesforce Pub/Sub API Avro in Salesforce Pub/Sub API: ● Used to serialize platform event data into an efficient binary format. ● Ensures real-time, high-performance transmission of events, allowing Salesforce to manage large volumes of data. ● Provides flexibility through schema evolution, ensuring systems remain compatible as event definitions change.
- 12. Benefits of IntegratinggRPC with SF Pub/Sub API Key Benefits: ● High Throughput & Low Latency: Using gRPC’s efficient serialization, Salesforce Pub/Sub API can handle real-time, large-scale data processing. ● Cross-Platform Compatibility: gRPC’s support for multiple languages allows integration with backend systems across various environments. ● Real-Time Synchronization: Bidirectional streaming in gRPC complements Salesforce’s event-driven architecture.
- 13. Exploring gRPC: ProjectSetup Key Files: ● package.json: Manages project dependencies. ● todo.proto: Defines the structure of the gRPC service. ● server.js: Implements the gRPC server that handles to-do operations. ● client.js: Implements the client that sends requests to the server .
- 14. Define the ProtoFile (todo.proto) Key Components: ● Service Definition: Defines the methods (createTodo, readTodos, readTodosStream) that will be exposed by the gRPC service. ● Message Definition: Defines the structure of TodoItem that will be exchanged between the client and server.
- 15. gRPC Server Implementation(server.js) Key Code: ● createTodo: Adds a new to-do item. ● readTodos: Returns all to-do items. ● readTodosStream: Streams to-do items one by one to the client (server).
- 16. gRPC Client Implementation(client.js) Key Code: ● Create a to-do item: ● Stream to-do items:
- 17. Salesforce Pub/Sub API:Java Quick Start Overview: ● Define Platform Events: Define events like Order_Event__e in Salesforce to publish and subscribe to them. ● Build Java Client: Use gRPC to implement a Java client that subscribes to platform events. ● Use PublishStream and Subscribe RPC Methods: Publish platform events and receive them in real-time.
- 18. Running the JavaPub/Sub API Example Steps: 1. Install Java dependencies and build the project using Maven. 2. Run the server to handle event subscriptions and publish events. 3. Execute the client to publish and subscribe to events in real time.
- 19. Conclusion Key Points: ● gRPCis a robust framework for building high-performance, real-time distributed applications. ● Salesforce Pub/Sub API leverages gRPC for efficient, scalable event- driven architectures and real-time event streaming. ● Apache Avro plays a crucial role in the Salesforce Pub/Sub API, enabling compact, efficient data serialization with support for schema evolution. ● Together, these technologies provide powerful tools for building scalable and interoperable systems, especially in large enterprises with complex data flows.
- 20. Some useful links ●gRPC Node example: https://youtu.be/Yw4rkaTc0f8?si=-fH94MYakerKDCGV&t=1185 ● SF Pub/Sub API example: https://github.com/forcedotcom/pub-sub-api ● SF Pub Sub API: https://developer.salesforce.com/docs/platform/pub-sub-api/overview ● Apache Avro: https://avro.apache.org/ ● gRPC: https://grpc.io/ ● Trailhead - Pub/Sub API Basics: https://trailhead.salesforce.com/content/learn/modules/pub-sub-api-basi cs
Editor's Notes
- #3 In this session, we will explore two powerful technologies: gRPC and Salesforce Pub/Sub API. We’ll start by understanding what gRPC is and how it facilitates efficient, low-latency communication in distributed systems. Then, we’ll dive into Salesforce Pub/Sub API—Salesforce’s event-driven architecture for handling platform events and real-time data exchange.
- #4 Explain that gRPC enables a client to directly invoke methods on a server running on a different machine as if it were a local call. Highlight the use of Protocol Buffers for compact data serialization, which reduces transmission sizes compared to JSON or XML. Emphasize its key features such as support for streaming, authentication, and cross-platform communication(gRPC)(gRPC). When I mention gRPC's key features such as support for streaming, authentication, and cross-platform communication, I’m referring to specific technical advantages that gRPC offers, making it a powerful tool for building distributed systems: Streaming: gRPC supports four types of RPC (Remote Procedure Calls): Unary RPC: The client sends a single request to the server and gets a single response. Server Streaming RPC: The client sends a request, and the server responds with a stream of responses, which can be processed as they arrive. Client Streaming RPC: The client sends a stream of requests, and the server processes them and sends back a single response. Bidirectional Streaming RPC: Both client and server can send a stream of messages to each other independently, which allows for complex real-time interactions like chat applications or real-time data feeds(gRPC)(gRPC). Authentication: gRPC integrates authentication, making it possible to secure communication between the client and server. It supports token-based authentication (like OAuth2) or mutual TLS (mTLS) for end-to-end encryption and verification of both client and server identities(gRPC). Cross-Platform Communication: gRPC is designed to work across various platforms and languages (like Java, Go, Python, C++, and Node.js). This makes it ideal for microservices architectures where different services may be written in different languages. It automatically generates client and server code for supported languages, making it easier to connect services across different systems(gRPC). These features make gRPC highly flexible and suitable for modern, cloud-based, and distributed applications.
- #5 Start by explaining that HTTP/2 is a significant improvement over HTTP/1.1, which was limited by its sequential request-response model. HTTP/2 introduces features like multiplexing, header compression, and server push that dramatically improve efficiency and speed.
- #6 Multiplexing: Mention that HTTP/2 allows multiple streams to exist simultaneously over the same TCP connection. This eliminates the bottleneck in HTTP/1.1, where only one request could be handled per connection at a time. In gRPC, this means multiple RPC calls can be processed at once, improving throughput. Binary Protocol: HTTP/2 uses a binary protocol for communication, unlike HTTP/1.1’s text-based approach. This makes data transmission more efficient and parsing faster. In gRPC, where performance is crucial, the binary nature of HTTP/2 ensures that message serialization and deserialization happen quickly, reducing latency. Bi-Directional Streaming: Explain that HTTP/2 supports bi-directional communication, allowing the client and server to send streams of data to each other simultaneously. In gRPC, this feature powers streaming RPCs, where both the client and server can continuously exchange messages in real time, without waiting for each other to finish. Low-Latency Communication: gRPC is designed for low-latency communication, and HTTP/2 plays a key role in achieving this. Features like header compression and stream prioritization further reduce delays in message transmission, making gRPC ideal for applications where speed is critical—such as microservices, live data feeds, or real-time collaboration tools.
- #7 Concurrency and Efficiency: Finally, highlight how gRPC leverages HTTP/2’s ability to handle multiple streams efficiently, reducing the need for opening and managing multiple connections. This is particularly beneficial for distributed systems, where gRPC services can handle a large number of simultaneous clients without the overhead of managing numerous TCP connections.
- #8 Mention that Salesforce Pub/Sub API is designed for event-driven architecture, where services communicate through platform events. Events can trigger workflows, push notifications, or sync data across various systems. (Salesforce Developers)(Salesforce Developers).
- #10 Explain that Apache Avro ensures compact, efficient, and schema-driven serialization of event data in Salesforce Pub/Sub API. Mention how Avro helps maintain schema evolution, which is important when platform events evolve over time but need to maintain backward compatibility(Salesforce Developers)(Salesforce Developers). Schema-Based Serialization: Avro uses a schema to define the structure of the data. This ensures that the data being serialized is structured consistently, making it easier to enforce standards when sending data between systems. In Salesforce Pub/Sub API, the platform defines events using Avro schemas, which allows the API to serialize the event data into a compact binary format and ensure that the messages are compatible across different systems(Salesforce Developers)(Salesforce Developers). Efficient Data Encoding: Avro provides a very compact serialization format by using binary encoding, which reduces the message size compared to JSON or XML. This is particularly useful in Salesforce Pub/Sub API for transmitting large amounts of event data efficiently over the network, reducing latency and improving performance. Schema Evolution: One of the strong features of Avro is its support for schema evolution. This allows you to add, remove, or modify fields in the schema without breaking compatibility with old data, which is crucial for long-lived applications where event structures may change over time. Salesforce Pub/Sub API benefits from this flexibility as event definitions evolve(Salesforce Developers). Interoperability: Since Avro is language-neutral, it allows different applications written in different languages to communicate seamlessly. In the context of Salesforce Pub/Sub API, this is particularly valuable because external systems (written in Java, Python, etc.) can easily decode Avro-encoded messages and integrate with Salesforce.
- #11 Explain that Apache Avro is crucial for making Salesforce Pub/Sub API both compact and efficient in transmitting large amounts of event data. Mention that Avro’s schema evolution feature allows Salesforce users to modify event definitions over time without breaking existing systems. Highlight how its language neutrality ensures seamless data exchange across systems, even when they are written in different programming languages(Salesforce Developers)(Salesforce Developers).
- #12 Describe how combining Salesforce Pub/Sub API with gRPC allows for seamless data synchronization and processing in real-time environments. Highlight how this setup can benefit large enterprises with distributed systems and microservice architectures. (gRPC)(gRPC)
- #13 Provide an overview of the project setup and key files. Highlight the dependencies defined in package.json and introduce todo.proto as the file that defines the service and messages for the gRPC communication
- #14 Explain the role of .proto files in gRPC communication. This file defines the methods and the data structures used by both client and server to communicate. Mention that this example showcases a simple to-do list where the client can create to-do items and retrieve them via RPC methods.
- #15 Discuss how the server is initialized, and the services are added using addService. Explain the role of each method: createTodo (adding items), readTodos (returning a list), and readTodosStream (streaming data to the client).
- #16 Explain how the client sends requests to the server using createTodo and receives a list of to-do items through streaming with readTodosStream. Demonstrate how gRPC supports real-time communication using client-server streams.
- #17 Walk through how to define platform events in Salesforce and build a Java client using gRPC. Explain how the PublishStream method is used to send events and how the client can subscribe to and handle incoming events in real-time(Salesforce Developers)(Salesforce Developers).
- #18 Walk through the steps to build the Java client for Salesforce Pub/Sub API. Highlight how easy it is to subscribe to platform events and handle them as they are published.
- #19 Recap that gRPC enables low-latency communication and is well-suited for cross-platform and multi-language environments. Highlight how Salesforce Pub/Sub API benefits from gRPC to handle high-volume, real-time event streams with efficient serialization provided by Apache Avro. Mention that the Java Quick Start demonstrated the real-world application of Salesforce Pub/Sub API for handling platform events, showing how these technologies come together to create reliable, scalable systems. Emphasize that these tools make it easier to manage event-driven architectures, and are especially valuable in large organizations that need flexibility, scalability, and real-time processing.