Presented by Warren Vella | Solutions Engineer, Melbourne | Confluent

1. The Playful Bond Between
REST And Data Streams
Warren Vella | Solutions Engineer, Melbourne | Confluent

2. About me.

3. “Hey, I am a microservice,
come and talk to me.”
REST API

4. “I need to share my data with
an external application.”
REST API

5. “I need to modernise my
database with a facade.”
REST API

6. API ≠ HTTP-based
RESTful APIs

7. Operational Databases Analytic Systems
Azure
Synapse
How do we integrate these systems together?
SaaS Applications
Applications
Microservices Monoliths
LEGACY INTEGRATION
REST APIs only work here

8. REST APIs
8
1. …have proliferated as a way to integrate
microservices.
2.…are often the default when exchanging
data between organisations.
3.…don’t meet all our integration needs.

9. Lets go on a journey…
The Stream Bean Cafe ☕

10. Early 2000s

11. Intra-Process Communication

12. 2011ish

13. Microservices.
Trend away from monolithic applications.

14. Distributed application architecture
POS Payment Gateway
DataWarehouse
Realtime Analytics
CRM

15. How many lines to
integrate 4 systems?

16. n2-n

17. App 2
App 3
App 4
App 1
42-4
= 12 lines

18. How many lines to
integrate 2,000 systems?

19. n2-n

20. 20002-2000
= 3,998,000 lines

22. The microservices death star…

23. 1. Physical
2. Data Link
3. Network
4. Transport
5. Session
6. Presentation
7. Application
REST API
POS DataWarehouse

24. Synchronous
request-response
REST APIS are
addictive.
Like an addiction, they give us a false sense of
comfort of being always-available.
The more we build, the more we want but we often
overlook the exponential costs associated with
building failure mechanisms for never ending point-
to-point integrations.
We end up relapsing to REST APIs.

25. Payment Gateway
Request
Response
POS
DataWarehouse
Realtime Analytics
CRM
Asynchronous
Synchronous

26. REST APIs
26
1. …have proliferated as a way to integrate
microservices.
2.…are often the default when exchanging
data between organisations.
3.…don’t meet all our integration needs.
But we have ended up with a death star.
But are not suited for high-throughput.
So we have lots of different integration tools.

27. Let’s meet

28. Govern
Process
Connect
Store
Kafka decouples the producers and consumers
Data Topic

29. Side note…
Why is Kafka not a HTTP protocol like
REST?

30. Old World New World
Event Processor Stream Processor
REST API

31. REST API
• Great for Synchronous Responses
• Tight-coupling
• No delivery guarantees
• Receiving systems can be
overwhelmed
• Emphasis on Requests/Responses
• No Replay
• True Decoupling
• Stronger delivery guarantees. Kafka is
a buffer.
• Receiving systems process at a rate
they can handle.
• Emphasis on events.
• Rebuild state by replay.

32. Payment Gateway
Data Topic
Can REST and Kafka co-exist?
DataWarehouse
Realtime Analytics
CRM
Inside Boundary
POS
Microservices
Outside Boundary
Request / Response

33. 1. Physical
2. Data Link
3. Network
4. Transport
5. Session
6. Presentation
7. Application
POS Realtime Analytics

34. Confluent Stream Sharing

35. Connect with Confluent

36. Operational Databases SaaS Applications Analytic Systems
Azure
Synaps
e
| DATA STREAMING PLATFORM
Today, a fourth estate is emerging that provides
a new foundation for your data strategy

37. “Hey, I am a microservice,
come and talk to me.”
REST API

38. “I need to share my data with
an external application.”
REST API

39. “I need to modernise my
database with a facade.”

40. The future of integration.

41. Key Takeaways
41
1. ✅ Reduce the amount of lines between
systems with Kafka.
2.✅ REST APIs are great for Request-
Response but also consider Kafka as a
pattern for exchanging data between
organisations.
3.✅ A Data Streaming Platform integrates
both application and data estates.

42. Let’s connect.
linkedin.com/in/warrenvella
github.com/wvella