Download as PDF, PPTX
Uploaded byTill Rohrmann
Fault Tolerance and Job Recovery in Apache Flink @ FlinkForward 2015
The document discusses fault tolerance and job recovery mechanisms in Apache Flink, highlighting the importance of implementing such features to prevent data loss and system downtime. It details concepts like checkpoints, barriers, and the different fault tolerance guarantees provided by Flink, enabling high availability and effective job processing. The overall message emphasizes that Flink ensures reliable data processing with low latency and high throughput through its advanced recovery strategies.
More Related Content
Similar to Fault Tolerance and Job Recovery in Apache Flink @ FlinkForward 2015
Fault Tolerance and Job Recovery in Apache Flink @ FlinkForward 2015
- 1. Fault Tolerance andJob Recovery in Apache Flink™ Till Rohrmann trohrmann@apache.org @stsffap
- 2.
- 3. Better be safethan sorry § Failures will happen § EMC estimated $1.7 billion costs due to data loss and system downtime § Recovery will save you time and costs § Switch between algorithms § Live upgrade of your system 3
- 4.
- 5. Fault tolerance guarantees § At most once • No guarantees at all § At least once • For many applications sufficient § Exactly once § Flink provides all guarantees 5
- 6. Checkpoints § Consistent snapshotsof distributed data stream and operator state 6
- 7. Barriers § Markers forcheckpoints § Injected in the data flow 7
- 8. 8 § Alignment formulti-input operators
- 9. Operator State § Statelessoperators § System state § User defined state 9 ds.filter(_ != 0) ds.keyBy(0).window(TumblingTimeWindows.of(5, TimeUnit.SECONDS)) public class CounterSum implements RichReduceFunction<Long> { private OperatorState<Long> counter; @Override public Long reduce(Long v1, Long v2) throws Exception { counter.update(counter.value() + 1); return v1 + v2; } @Override public void open(Configuration config) { counter = getRuntimeContext().getOperatorState(“counter”, 0L, false); } }
- 10.
- 11.
- 12.
- 13.
- 14. Advantages § Separation ofapp logic from recovery • Checkpointing interval is just a config parameter § High throughput • Controllable checkpointing overhead § Low impact on latency 14
- 15.
- 16.
- 17.
- 18.
- 19.
- 20. Persisting jobs 20 JobManager Client TaskManagers Apache Zookeeper™ 1. Submit job 2. Persist execuAon graph
- 21. Persisting jobs 21 JobManager Client TaskManagers Apache Zookeeper™ 1. Submit job 2. Persist execuAon graph 3. Write handle to ZooKeeper
- 22. Persisting jobs 22 JobManager Client TaskManagers Apache Zookeeper™ 1. Submit job 2. Persist execuAon graph 3. Write handle to ZooKeeper 4. Deploy tasks
- 23.
- 24. Handling checkpoints 24 JobManager Client TaskManagers Apache Zookeeper™ 1. Take snapshots 2. Persist snapshots 3. Send handles to JM
- 25. Handling checkpoints 25 JobManager Client TaskManagers Apache Zookeeper™ 1. Take snapshots 2. Persist snapshots 3. Send handles to JM 4. Create global checkpoint
- 26. Handling checkpoints 26 JobManager Client TaskManagers Apache Zookeeper™ 1. Take snapshots 2. Persist snapshots 3. Send handles to JM 4. Create global checkpoint 5. Persist global checkpoint
- 27. Handling checkpoints 27 JobManager Client TaskManagers Apache Zookeeper™ 1. Take snapshots 2. Persist snapshots 3. Send handles to JM 4. Create global checkpoint 5. Persist global checkpoint 6. Write handle to ZooKeeper
- 28.
- 29.
- 30.
- 31. TL;DL § Job recoverymechanism with low latency and high throughput § Exactly one processing semantics § No single point of failure è Flink will always keep processing your data 31
- 32.