A CaseStudy of a Global Cruise Company; how they migrate from legacy monolith to battle-ready micro-services and provides seamless guest experience leveraging Kafka Streams.

1. Seamless Guest Experience with Kafka Streams
Ram IndukuriHimani Arora Suresh Mulukaladu

2. About Knoldus
Technology consulting firm with focus on digital transformation
Functional. Reactive. Cloud Native

3. We are global cruise vacation company that controls and operates four global
brands. Royal Caribbean International, Azamara club, Celebrity cruises and
Silversea Cruises.We are also 50% joint venture owner of German brand TUI
Cruises and 49% owner of the spanish brand Pullmantur Cruceros.
About Royal Caribbean International

4. Royal Caribbean International - Facts
● World’s second largest global cruise line
● Represents more than 20% of global cruisers
● 9.5 billion in revenues (2018)
● Serving more than 5 Million guests every year around the globe
● 60 ships in service and another 14 ships on the way
● Oasis class(World’s largest) ships carry close to 6000 guests and 2000
crew

5. Guest Journey - touchpoints
Discover Book
Cruise
Excursions Check In Board Ship Cruise Take
Excursions
Debark Create
Memories
● Research vacation options
● Book the cruise & air reservations
● Plan and Book activities on the cruise
● Complete online check-in and get boarding pass
● Pre cruise activities
● Get to the embarkation port
● Check-in to the cruise and board
● Get to your state room
● Book dinner reservations and activities
● Enjoy the cruise and activities
● Excursions at ports of call
● Debarkation
● Post cruise activities
● Fly back
● Get home

6. Excalibur

7. Digital Vision
Discover Book Cruise Pre Cruise
Planning
Check In Board Ship Cruise Take
Excursions
Debark Create
Memories
Sensor
Wearable
TV
API Layer
Device Layer
One App

8. Architectural Challenges
1
How to Standardize Footprints
on all ships and shore ?
3
How do guests have seamless
experience on ship(s) and
shore if we rely on batch
process ?
2
How do we build on what we
have today ?
Same infrastructure on
ship/shore
API and Microservices Event driven Architecture with real time
synchronization

9. Ship-Shore Highway - Event enabling Fleet
System of
Record
System of
Record
Ship Shore Highway
Event enablement, transmission, processing & Analytics across fleet

10. SoR Ship-Shore Pattern
API Gateway
Micro Service
Data Store
System of
Record
Data Store
System of
Record
Micro Service

11. Change data Capture - Using Kafka Connect
Web
Call Center
Travel Agent
SQL
Mirror
Kafka
Connect
Kafka
Shore
Reservation
SOR
AWS
Data Transformer
Kafka Ship

12. Broadcasting events - Using Streams
Web
Call Center
Travel Agent
IIDR
Business Events &
Change Data
Reservation
SOR
AWS
Kafka
Streams
Kafka

13. Generating Business Events by Combining Topics
Booking ID(pk)
Guest ID
Guest ID(pk)
Name
Address
State Store
State Store
JOIN
Guest ID
Booking ID
Name
Address

14. Generating Business Events by Combining Topics
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message */,
JoinWindows.of(Duration.ofHours(10));

15. Guest Bookings Stream
Guest Details Stream
Transparent Handling of Late Arrival Records
SLIDING
WINDOW
JoinWindows.of(Duration.ofHours(10))
Late
Arriving
Record
Event 1Event 2
Event 1Event 2
Event 12
Event 13
Event 13
Event 14
Event 14
Event 15

16. Transparent Handling of Late Arrival Records
Guest Bookings Stream
Guest Details Stream
SLIDING
WINDOW
Grace Period set to:
JoinWindows.of(Duration.ofHours(10)).grace(Duration.ofHours(24)))
Event 1Event 2
Event 1Event 2
Event 12
Event 13
Event 13
Event 14
Event 14
Event 15

17. Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message */,
JoinWindows.of(Duration.ofHours(10)
.grace(Duration.ofHours(24))
);

18. Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purchase ID
Intermediate
State Store

19. Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purchase ID
Intermedia
te Store
State Store
JOIN
Guest ID
Booking ID
Purchase ID
Name
Address

20. Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, PurchaseDetails> guestPurchases = builder.stream(guestPurchasesTopic);
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message with booking id and
guest details */,
JoinWindows.of(Duration.ofHours(10)
.join(guestPurchases,
(guestBookingDetail,guestPurchase) -> /* business event message with guest
details, booking and purchase details*/,
JoinWindows.of(Duration.ofHours(10)));

21. Ensuring Streaming reliability - Graphite/Grafana
Kafka
Stream
App
JMX
Graphite Grafana
● Consumer Lag to detect surges
● Idle time threshold to predict Thread health
● Bytes Received in last window indicating Kafka broker issues
Lack of activity indicate either
broker failure or app failure

22. Ensuring Data Quality - Apache Spark
Apache Spark Apache Spark
Minio incremental
Mirroring
1
2
3
4 5

23. Next Steps - Interactive Queries
Instance 1
Instance 2
Instance 3
GUEST API
Local Kafka
State Store
Local Kafka
State Store
Local Kafka
State Store
host1:4047
host2:5047
host3:4460
Exposing RPC endpoint and
information about its local
state store
Kafka Cluster

24. Summary
● Real time guest information synchronization across ship and
shore was possible with kafka, connect and replicators
● Kafka streams and IIDR helped to reduce the latency in
processing the CDC
● Isolate critical connectors into their own connect cluster to
reduce risk
● Order guarantee is easier to achieve with streams than
connectors.
● Invest in domain driven design
● Include legacy dependencies in critical path for planning
feature delivery.

25. More Information
Ram Indukuri ram@knoldus.com
Himani Arora himani@knoldus.com
Suresh Mulukuladu smulukuladu@rccl.com
For more information
Blog: https://blog.knoldus.com/join-semantics-kafka-streams/
Git Samples: https://github.com/knoldus/