The cruise industry has unique set of challenges when it comes to deploying products that directly interact with guests across ships. No matter what the challenges are, Applications that run on ships, always need to provide same functionality as on shore. To name few, Each ship is almost a separate datacenter and there are tens of ships. Second, the ships compute capacity is much lower than what an AWS data center can provide and hence any thing that is deployed on ship need to be lean.Third, the bandwidth, though improving, is still a big constraint in transferring data between ship and shore. Fourth, The company’s legacy systems power the core booking functionality and that information is still vital to every feature or application that happens across enterprise including ships. Hence traditionally cruise industry shied away from real time integration and mostly resorts to a ‘rollover’ process, where in the data gets transferred at the beginning or end of the cruise. But this can’t be the solution with digital. Mobiles need to allow stream of changes real time, as guests go from ship to shore to ship or ship to ship or shore to ship. Our customer, a leading cruise company instituted a team to solve these synchronization issues and also enable event driven architecture. This presentation will address the evolution, idea and implementation of data synchronization and enabling event driven organization using Kafka and streams technologies at its core.

1. Seamless Guest Experience with Kafka Streams
Ram IndukuriHimani Arora Suresh Mulukaladu

2. About Knoldus
Technology consulting firm with focus on digital transformation
Functional. Reactive. Cloud Native

3. We are global cruise vacation company that controls and operates four global
brands. Royal Caribbean International, Azamara club, Celebrity cruises and
Silversea Cruises.We are also 50% joint venture owner of German brand TUI
Cruises and 49% owner of the spanish brand Pullmantur Cruceros.
About Royal Caribbean International

4. Royal Caribbean International - Facts
● World’s second largest global cruise line
● Represents more than 20% of global cruisers
● 9.5 billion in revenues (2018)
● Serving more than 5 Million guests every year around the globe
● 60 ships in service and another 14 ships on the way
● Oasis class(World’s largest) ships carry close to 6000 guests and 2000
crew

5. Guest Journey - touchpoints
Discover Book
Cruise
Excursion
s
Check In Board
Ship
Cruise Take
Excursion
s
Debark Create
Memories
● Research vacation options
● Book the cruise & air reservations
● Plan and Book activities on the cruise
● Complete online check-in and get boarding pass
● Pre cruise activities
● Get to the embarkation port
● Check-in to the cruise and board
● Get to your state room
● Book dinner reservations and activities
● Enjoy the cruise and activities
● Excursions at ports of call
● Debarkation
● Post cruise activities
● Fly back
● Get home

6. Excalibur

7. Digital Vision
Discover Book Cruise Pre Cruise
Planning
Check In Board Ship Cruise Take
Excursions
Debark Create
Memories
Sensor
Wearable
TV
API Layer
Device Layer
One App

8. Architectural Challenges
1
How to Standardize Footprints
on all ships and shore ?
3
How do guests have seamless
experience on ship(s) and
shore if we rely on batch
process ?
2
How do we build on what we
have today ?
Same infrastructure on
ship/shore
API and Microservices Event driven Architecture with real time
synchronization

9. Ship-Shore Highway - Event enabling Fleet
M
icroservice
sData
stores
IoT Devices
System of
Record
M
icroservice
s Data
stores
IoT Devices
System of
Record
Ship Shore Highway
Event enablement, transmission, processing & Analytics across fleet

10. SoR Ship-Shore Pattern
API Gateway
Micro Service
Data Store
System of
Record
Data Store
System of
Record
Micro Service

11. Change data Capture - Using Kafka Connect
Web
Call Center
Travel Agent
SQL
Mirror
Kafka
Connect
ConfluentKafka
Kafka
Shore
Reservation
SOR
AWS
Data Transformer
Replicator
Kafka Ship

12. Broadcasting events - Using Streams
Web
Call Center
Travel Agent
IIDR
Confluent Kafka
Business Events &
Change Data
Reservation
SOR
AWS
Kafka
Streams Replicator
Kafka

13. Generating Business Events by Combining Topics
Booking ID(pk)
Guest ID
Guest ID(pk)
Name
Address
State Store
State Store
JOIN
Guest ID
Booking ID
Name
Address

14. Generating Business Events by Combining Topics
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message */,
JoinWindows.of(Duration.ofHours(10));

15. Guest Bookings Stream
Guest Details Stream
Transparent Handling of Late Arrival Records
SLIDING
WINDOW
JoinWindows.of(Duration.ofHours(10))
Late
Arriving
Record
Event 1Event 2
Event 1Event 2
Event 12
Event 13
Event 13
Event 14
Event 14
Event 15

16. Transparent Handling of Late Arrival Records
Guest Bookings Stream
Guest Details Stream
SLIDING
WINDOW
Grace Period set to:
JoinWindows.of(Duration.ofHours(10)).grace(Duration.ofHours(24)))
Event 1Event 2
Event 1Event 2
Event 12
Event 13
Event 13
Event 14
Event 14
Event 15

17. Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message */,
JoinWindows.of(Duration.ofHours(10)
.grace(Duration.ofHours(24))
);

18. Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purchase ID
Intermediate
State Store

19. Consecutive Joins on Multiple Topics
Booking ID
Guest ID
Guest ID
Name
Address
State Store
State Store
JOIN
Guest ID
Purchase ID
Intermedi
ate Store
State Store
JOIN
Guest ID
Booking ID
Purchase ID
Name
Address

20. Transparent Handling of Late Arrival Records
StreamsBuilder builder = new StreamsBuilder();
KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic)
.selectKey((key, value) -> value.getGuestId());
KStream<String, PurchaseDetails> guestPurchases = builder.stream(guestPurchasesTopic);
.selectKey((key, value) -> value.getGuestId());
KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic);
KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings
.join(guestDetails,
(guestBooking, guestDetail) -> /* business event message with booking id and
guest details */,
JoinWindows.of(Duration.ofHours(10)
.join(guestPurchases,
(guestBookingDetail,guestPurchase) -> /* business event message with guest
details, booking and purchase details*/,
JoinWindows.of(Duration.ofHours(10)));

21. Ensuring Streaming reliability - Graphite/Grafana
Kafka
Stream
App
JMX
Graphite Grafana
● Consumer Lag to detect surges
● Idle time threshold to predict Thread health
● Bytes Received in last window indicating Kafka broker issues
Lack of activity indicate either
broker failure or app failure

22. Ensuring Data Quality - Apache Spark
Apache Spark Apache Spark
Minio
incremental
Mirroring
1
2
3
4 5

23. Next Steps - Interactive Queries
Instance 1
Instance 2
Instance 3
GUEST API
Local Kafka
State Store
Local Kafka
State Store
Local Kafka
State Store
host1:4047
host2:5047
host3:4460
Exposing RPC endpoint and
information about its local
state store
Kafka Cluster

24. Summary
● Real time guest information synchronization across ship and
shore was possible with kafka, connect and replicators
● Kafka streams and IIDR helped to reduce the latency in
processing the CDC
● Isolate critical connectors into their own connect cluster to
reduce risk
● Order guarantee is easier to achieve with streams than
connectors.
● Invest in domain driven design
● Include legacy dependencies in critical path for planning
feature delivery.

25. More Information
Ram Indukuri ram@knoldus.com
Himani Arora himani@knoldus.com
Suresh Mulukuladu smulukuladu@rccl.com
For more information
Blog: https://blog.knoldus.com/join-semantics-kafka-streams/
Git Samples: https://github.com/knoldus/