Both digital and traditional businesses are constantly evolving, and the need to move fast is a pervasive reality. Delivering what customers want and need goes beyond the creation of delivery channels. In fact, it relies on the company’s ability to produce, consume, organise, understand, curate, and distribute data.
In this presentation, Dan Aragao and Simon Hope provide a glimpse of the journey ThoughtWorks and REA are currently undergoing to create a truly data-centric, cutting-edge digital business.
5. • In the beginning…
• Organising our Data
• Implementation approaches
• Hipster Batches
• Reactify
• Bring Your Own Data
• Finding the Data
• What we have learned so far
THIS IS WHAT THE STORY IS ABOUT
6. SORRY… IT’S OK TO LEAVE NOW
• Nope, we didn’t create a new Hadoop
• No hardcore Data Science
• There are some implementation details
• REA embraced the Cloud. AWS everywhere
• Under construction
30. ORGANISING OUR DATA
Increasingly, content is being
distributed through search
and social platforms...
There’s less visiting
of publishers as destinations.
Jeff Weiner, CEO, Linkedin
40. Good things come in small
packages services
THE HIPSTER BATCH
???
Hipster Batch
41. Hipster Batch
THE HIPSTER BATCH
• Small and short lived
• Decoupled via flat
files via S3
• Single purpose
• Idempotent
• Polyglot
• Minimal runtime
dependencies
• Discoverable
68. • Exposes current state only
• Stream of change notifications
• Hypertext Application Language - HAL
• Clear entity types
• Linking over embedding
• Cacheable and discoverable
REST API
REACTIFY REST API
107. WHAT WE HAVE LEARNED SO FAR
• Consumers create the last-mile data as needed
• We must work with external, independent
delivery channels
• Push quality back to source/producer systems
• Data belongs to the entire organisation,
not to a single team
108. I’ll give you my
Data Warehouse
when you can pry it
from my cold dead hands.