"A complex system that works is invariably found to have evolved from a simple system that worked." (John Gall) - The evolvability of a software system is in many cases its most important architectural quality attribute. And an essential tactic to achieve this attribute is the localization of change by reusing domain boundaries as system and team boundaries. This talk discusses the application of this principle to big data systems and describes mobile.de's journey from a monolithic data architecture to a domain-driven data platform.

1. Domain-Driven
Data Platforms
Florian Stefan eBay Classifieds

2. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

3. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

4. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

5. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

6. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

7. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

8. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

9. 15.780.000
Unique
Visitors
*
horizontal
buyer
innovation
seller
external
35.680.000 Unique Visitors**
mobile.de in June 2020
ebay Kleinanzeigen in Juni 2020
*
**
vertical

10. where do we
come from?
where do we
want to be?
how do we
get there?

11. where do we
come from?
where do we
want to be?
how do we
get there?

12. where do we
come from?
where do we
want to be?
how do we
get there?

13. where do we
come from?
where do we
want to be?
how do we
get there?

14. where do we
come from?
where do we
want to be?
how do we
get there?

15. where do we
come from?
where do we
want to be?
how do we
get there?

16. functions
business units
Product
&
Technology
Analytics
⬤
⬤
⬤
⬤
⬤
⬤
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤

17. functions
business units
Product
&
Technology
Analytics
⬤
⬤
⬤
⬤
⬤
⬤
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤

18. data department
functions
business units
Product
&
Technology
Analytics
⬤
⬤
⬤
⬤
⬤
⬤
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤

19. data department
functions
business units
Product
&
Technology
Analytics
⬤
⬤
⬤
⬤
⬤
⬤
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤

20. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

21. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

22. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

23. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

24. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

25. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

26. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

27. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

28. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
batch
processors
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
workflow
management

29. where do we
come from?
where do we
want to be?
how do we
get there?

30. where do we
come from?
where do we
want to be?
how do we
get there?

31. abstractness
stability

32. useless
abstractness
stability

33. useless
expensive
abstractness
stability

34. useless
expensive
abstractness
stability
fast

35. useless
expensive
abstractness
stability
fast
efficient

36. useless
expensive
abstractness
stability
fast
efficient
design
for
extension
accept
m
odifi
cation

37. useless
expensive
abstractness
stability
data
platform
design
for
extension
accept
m
odifi
cation
fast

38. useless
expensive
abstractness
stability
bounded
context
data
platform
design
for
extension
accept
m
odifi
cation

39. useless
expensive
abstractness
stability
bounded
context
data
platform
design
for
extension
accept
m
odifi
cation
bounded
context

40. Ivory Tower Technical Reality Legacy System

41. Ivory Tower Technical Reality Legacy System

42. Ivory Tower Technical Reality Legacy System

43. Ivory Tower Technical Reality Legacy System

44. useless
expensive
abstractness
stability
bounded
context
data
platform
design
for
extension
accept
m
odifi
cation

45. useless
expensive
abstractness
stability
bounded
context
data
platform
design
for
extension
accept
m
odifi
cation

46. where do we
come from?
where do we
want to be?
how do we
get there?

47. where do we
come from?
where do we
want to be?
how do we
get there?

48. data department
functions
business units
Product
&
Technology
Analytics
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤
⬤
⬤
⬤
⬤
⬤
⬤

49. functions
business units
Product
&
Technology
Analytics
Seller Buyer ⬤ ⬤ ⬤ ⬤ ⬤ ⬤
⬤
⬤
⬤
⬤
⬤
⬤

50. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

51. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management

52. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
self-service
batch
processors
workflow
management

53. gateway
business
systems
data ingestion
business
intelligence
platform
services
realtime
processors
data
governance
distributed
log
distributed
storage
container
orchestration
distributed
computing
data
warehouses
microservices
https://suchen.mobile.de/fahrzeuge/details.html?id=123
batch
processors
workflow
management
self-service

54. Libraries Plugins
Inner Source Templates Configurators

55. Libraries Plugins
Inner Source Templates Configurators

56. Libraries Plugins
Inner Source Templates Configurators

57. Libraries Plugins
Inner Source Templates Configurators

58. Libraries Plugins
Inner Source Templates Configurators

59. Libraries Plugins
Inner Source Templates Configurators

60. where are we?

61. where are we?

62. bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
producer
bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
processor
data
producer
data
platform
bounded
context(s)
data
governance
data
ingestion
business
intelligence
data
processor
data
producer

63. bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
producer
bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
processor
data
producer
data
platform
bounded
context(s)
data
governance
data
ingestion
business
intelligence
data
processor
data
producer

64. bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
producer
bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
processor
data
producer
data
platform
bounded
context(s)
data
governance
data
ingestion
business
intelligence
data
processor
data
producer

65. bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
producer
bounded
context(s)
data
platform
data
governance
data
ingestion
business
intelligence
data
processor
data
producer
data
platform
bounded
context(s)
data
governance
data
ingestion
business
intelligence
data
processor
data
producer

66. subdomain-bounded-context
┗ build.sbt
┗ Jenkinsfile
┗ webservice
┗ marathon
┗ src/main/scala
┗ src/main/resources/gdpr/schema.json
┗ job
┗ airflow
┗ src/main/scala
┗ domain
┗ src/main/scala

67. subdomain-bounded-context
┗ build.sbt
┗ Jenkinsfile
┗ webservice
┗ marathon
┗ src/main/scala
┗ src/main/resources/gdpr/schema.json
┗ job
┗ airflow
┗ src/main/scala
┗ domain
┗ src/main/scala

68. “A good developer is like a werewolf:
afraid of silver bullets.”
Jochen Mader

69. Thank you!

70. Questions?