This document discusses Unity's migration of their data infrastructure to Google Cloud Platform (GCP) using Confluent Cloud and Kafka. The key points are: 1) Unity migrated their infrastructure to GCP to consolidate systems and take advantage of new product opportunities, managing a large complex migration. 2) They used Confluent Cloud on GCP along with tools like Pub/Sub, Dataflow, and BigQuery to ingest, process, and analyze streaming data in their systems. 3) The migration was successful in transferring petabytes of data between clouds without disruption, allowing Unity to take advantage of GCP's managed streaming services alongside Confluent tools.

1. Try FREE
$50 Free
each month
3 Months
from signup
confluent.io/cloud
Available in
GCP Marketplace

2. Simplified Hybrid Cloud Migration
with
Confluent and Google Cloud
Unity’s Multi-Cloud Migration Story

3. Presenters
Josh Treichel Oguz Kayral Kir Titievsky
3
Leads Partner
Solutions Engineering
at Confluent to help
enable partners on
Kafka and the
Confluent platform.
He’s excited to work
closely with GCP and
help grow the
partnership.
Is the engineering
manager at Unity
Technologies. He
leads the centralized
data team that
supports the Unity
development
platform and the
monetization
network.
Is the product
manager of Google
Cloud Pub/Sub and a
fan of all things
streaming and Kafka.
He has had the
privilege of working
with Confluent to
bring Confluent Cloud
to GCP users.

4. Agenda
4
01
Unity Story - Why move to
GCP?
03
Kafka, Confluent and
Confluent Cloud
02
GCP and Partners

5. About Unity
5
• Millions of Creators, Billions of Gamers
• Half of Top 1000 Mobile Games
• Products include Analytics, Monetization, Crash and Performance
Reporting, Asset Store, Collaborate, Cloud Build
• Beyond Game Development:
• Automotive, Transportation & Manufacturing
• Film, Animation & Cinematics
• Architecture, Engineering & Construction

6. Unity and GCP Partnership
• Massive Cloud Migration consolidating Unity infrastructure across
business units
• Managing a Large and Complex GCP Migration (Cloud Next '19)
• New product development opportunities
• Open Match: Open Match: Flexible and extensible matchmaking
for games
• Complete re-architecture of Unity Data Platform
• Decision to continue investing in Kafka
6

7. Unity and GCP Partnership
7

8. Unity and GCP Partnership
8
Confluent Platform Cloud Dataflow BigQuery

9. Unity Kafka Traffic
9

10. The Migration
• Go slow to go fast. Time spent on coming up with failure cases
during planning phase is much better than seeing failures once the
migration starts.
• 3 engineers with help from Confluent
• Replicator - Petabytes of data transferred between clouds without
disruption. No outages or degraded service caused by the migration.
• Once we were on GCP. It opened up a whole new world of stream
processing with managed services.
• Kafka Connect and “it’s always the network”
10

12. Why Confluent users come to GCP
12
Ingest & distribute Connect & Transform
Interact
Warehouse
Apache
Beam
Pub/Sub
Dataflow
Dataproc
BigQuery
Cloud
Storage
IoT Core
AI APIs
Google
Data Studio
Tensorflow
Sheets
Transfer
Services
Data Catalog
Data Fusion
Composer

13. Mixing GCP & Confluent tools
13
Pub/Sub
Kafka
KStreams/KSQ
L
Kafka Connect
Dataflow
Storage
App
Storage
App

14. What about GCP makes mixing easy?
• Self-serve
• Account team collaboration
• Dataflow: Serverless connectivity,
analytics
14

16. Apache Kafka is a Distributed Event
Streaming Platform
Process streams of events In real time, as they occur
110101
010111
001101
100010
Publish and subscribe to
streams of events
Similar to a message queue or
enterprise messaging system
110101
010111
001101
100010
Store streams of events In a fault tolerant way
110101
010111
001101
100010
16

17. Confluent Platform builds on Apache Kafka
Apache Kafka
Schema Registry
Data Compatibility
Kafka Connect
GCS Sink
KSQL
Stream processing
BigQuery
Cloud
Storage
Dataflow
Your Apps
KStreams
BigQuery
Sink

18. It can get complicated….
Kafka Brokers
Broker +
Rebalancer
Schema Registry
Leader Follower
Kafka Connect
GCS Sink
KSQL
ksqlDB
Server
ksqlDB
Server
BigQuery
Cloud
Storage
Dataflow
Broker +
Rebalancer
Broker +
Rebalancer
Broker +
Rebalancer
Your Apps
KStreams
GCS SinkGCS Sink
GCS SinkGCS SinkBigQuery
Sink
ZooKeeper Nodes
ZK ZK ZK ZK ZK

19. Dataflow
BigQuery
Cloud
Storage
KSQL
real time filtering,
curation,
aggregation
Schema Registry
Confluent Cloud | Apache Kafka re-engineered for cloud
Part of Confluent Platform
Kafka Connect
...
BigQuery
Sink
GCS
Sink
Auto Data
Balancing
Tiered
Storage*
Your Apps
KStreams
Apache Kafka
Elastic
Scaling

20. Two Ways to Deploy
2020
SELF-MANAGED SOFTWARE
Confluent Platform
The Enterprise Distribution of Apache Kafka
Deploy on any platform, on-prem or cloud
VM
FULLY-MANAGED SOFTWARE
Confluent Cloud
Apache Kafka Re-Engineered for the Cloud
Available on the leading public clouds
Option to purchase via GCP marketplace
Both: Subscription products where price scales with usage

21. On-premise or Other
Cloud
What does hybrid Kafka architecture look like on GCP?
Web
IoT
Mobile
Data
Store
Dataflow
BigQuery
Cloud
Storage
1
Trigger &
Send
2
Ingest &
Prepare
3
Transform &
Enrich
4 Store & Analyze 5
Share &
Activate
Data
Studio
Cloud
Functions
AI
Platform
Bigtable
Events
Confluent Replicator
ksqlDB

22. Try FREE
$50 Free
each month
3 Months
from signup
confluent.io/cloud
Available in
GCP Marketplace