Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1QN63P7.
Ben Stopford looks at the implications of mixing toolsets from the stream processing world into real-time business applications: how to effectively handle infinite streams, how to leverage a high throughput, persistent Log and deploy dynamic, fault tolerant, and streaming services. Filmed at qconlondon.com.
Ben Stopford is a specialist in data technologies. He’s worked in Finance, at Thoughtworks and is now at Confluent. His experience cover a variety of fields including online trading, retail, High Performance Computing and building the central data platform for a large investment bank.
2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/microservices-streaming
3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com
4. There are many good reasons for
building service-based systems
• Loose Coupling
• Bounded Contexts
• Autonomy
• Ease of scaling
• Composability
5. But when we do,
we’re building a distributed system
70. What is stream processing
engine?
Data
Index
Query
Engine
Query
Engine
vs
Database
Finite, well defined source
Stream Processor
Infinite, poorly defined source
89. More Complex Use Cases
Trades Valuations
Books Customers
General
Ledger
90. trades books
risk results
ex-
rates
Practical mechanism for managing data
intensive, loosely coupled services
• Stateful streams live
inside the Log
• Data extracted quickly!
• Fast, local joins, over
large datasets
• HA pre-caching
• Manage intermediary
state
• Just a simple library
(over Kafka)
91. There is much more to
stream processing
it is grounded in the world of big-data analytics