Events: Distributed programming made easy
•Download as PPTX, PDF•
1 like•239 views
F#unctional Toronto Users Group presentation. Broadcast and handle the updates instead of asking for information, get scalability availability and fault tolerance in return.
Recommended
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to Events: Distributed programming made easy
Similar to Events: Distributed programming made easy (20)
Recently uploaded
Recently uploaded (20)
Events: Distributed programming made easy
- 1. EVENTS: DISTRIBUTED PROGRAMMING MADE EASY STORM + F#
- 2. ABOUT ME Eugene Tolmachev Principal Architect @ Prolucid Developing software for over 20 years Open source contributor: • FsStorm maintainer • Bitlore author • FsUnit • Dapper • MassTransit • Nowin Storm + F#
- 3. SCALE OUT & AVAILABILITY Storm + F# UI Backend DB A = Aui Abe Adb
- 4. DISTRIBUTED COMPUTING DRPC/Actors • CORBA/DCOM • Web-services • AKKA • Orleans Data freshness ESP/CEP • Storm • Spark Batch processing • Custom ETL • Hadoop • mbrace • Spark Storm + F#
- 5. ASK VS TELL Storm + F# Model Price Risk Date getRisk(date) risk date getModel(date) model Stack: … getPrice getRisk getModel … • Easy to start with • Composed at design-time • Fault handling: on the caller • Challenging to cache • Fragile integration
- 6. EVENTS (FIRE-N-FORGET) Implemented via Store-and-forward messaging Represent a summary of a change Self-contained* Time-relevant, not time-sensitive Uniquely identifiable • Tracing/correlation • Idempotency Storm + F#
- 7. ASK VS TELL Storm + F# • Upfront setup • Can compose at runtime • Fault handling: Compensations • Caching – unnecessary • Can evolve independently • Embarrassingly parallel Price Risk DateModel modelUpdated(m,date) riskUpdated(r,m,date) newDay(date) newDay(date) calculate Model calculate Risk calculate Price generate Date Multiple Stacks:
- 8. MICRO-SERVICES Services encapsulate • Business-logic • Can evolve independently as long as the surface API remains the same Services control the context • Persistence • Security • Performance Micro-services + Do one thing only, do it well - Introduce latency - Deployment/management overhead Storm + F#
- 9. THERMOSTAT ALERT Storm + F# Sensor is up Sensor value Settings updated Write down raw data Evaluate against the threshold Send notification Read all sensors Write down the settings Write down the result
- 10. APACHE STORM Runs on JRE Storm • Spouts: throttled • Streams: default and named • Tuples: anchored or not • Bolts: input – on demand/output – at will • Grouping (affinity): all, shuffle, fields, direct Storm multilang protocol: • JSON over stdin/stdout • Components: a process per instance Storm CLI, to start: • storm dev-zookeeper (runtime coordination) • storm nimbus (management/api) • storm supervisor (runs components) • storm ui (optionally, http://localhost:8080) Storm + F#
- 11. THERMOSTAT TOPOLOGY Storm + F# sensorStartedEvents [1] sensorUpdateEvents [1] settingsUpdateEvents [1] sensorWriter [2] sensorViewProjection [2] sensorViewUpdatedNotifier [1] settingsUpdatedNotifier [1] settingsWriter [1] sensorViewWriter [2] allSensorsReader [1]
- 12. FSSTORM Runtime • Implements multilang protocol • Lets you write components in F# • Additional implementations for unit-testing DSL: • Define topology in F# • Validated Scripts • Submit your topology to nimbus Storm + F#
Editor's Notes
- Availability – scalability implied Scale out means distribution
- Events Stream Processing – sweet spot Next – oversimplified and mostly wrong example from financial domain
- Data – typically normalized Model – valid till then next one;
- Events – summary of change, de-normalized data, Next – hosting