Kafka Streams relies on state restoration for maintaining standby tasks as failure recovery mechanism as well as for restoring the state after rebalance scenarios. When you are scaling up or down your application instances, it is necessary to know the current state of the restoration process for each active and standby task in order to prevent a long restoration process as much as possible. During this presentation, you will get an understanding of how KIP-869 provides valuable information about the current active task restoration after a rebalance and KIP-988 opens a window to the continuous process of standby restoration. When you encounter a situation in which you need to choose whether or not to scale up or down your application instances, both KIPs will be an invaluable ally for you.

1. A Window Into Your Kafka Streams Tasks
Eduwer Camacaro

2. A Window into your Kafka
Stream Tasks
Eduwer Camacaro, Software Engineer
@eduwercamacaro

3. LittleHorse Server

4. LittleHorse Server

5. LittleHorse Server

6. LittleHorse Server

7. Keeping Key Value Stores Healthy

8. Keeping Key Value Stores Healthy

9. Active Tasks

10. Standby Tasks

11. Take a look at what’s happening with
your tasks behind the scenes.

12. Standby Visibility

13. Standby Visibility

14. Standby Visibility

15. State Restore visibility

16. State Restore Visibility

17. State Restore Visibility

18. Questions we can answer
with these callbacks

19. Littlehorse in the
Cloud
3> instances 145 tasks
1 standby replica 4> warmups replicas

20. Standby Tasks per Instance

21. Total Standby Lag per Store

22. How long does it take to restore an active task?

23. Thank you
Eduwer Camacaro
eduwerc@littlehorse.io
@eduwercamacaro