Federated learning enables us to build machine learning models using data collected by edge devices like smartphones and IoT devices, without moving data off the device. This minimizes concerns about privacy, data regulation, bandwidth, and storage, while providing similar results as centralized models. Examples include predictive text on cell phones, a person’s engagement with their own photos, and machine learning in the browser applied to corporate text archives such as a team Slack or Google Drive, and ML on low-powered field devices in energy, agriculture and logistics. The principles of data minimization established by the GDPR and the prevalence of smart sensors makes these use cases more common, and the advantages of federated learning more compelling. In this talk we’ll cover the algorithmic solutions and the product opportunities. This talk was presented by Alice Albrecht (Research Engineer, Cloudera) at NYAI #26 on Tues, 11/28 at Capital One Labs. nyai.co/nyai-26

1. FAST FORWARD LABS
FEDERATED LEARNING: Machine learning on edge devices
Alice Albrecht
NYAI November, 2018

2. 2
CLOUDERA FAST FORWARD LABS
ADVISING &
RESEARCH
ML APPLICATION
DEVELOPMENT
ML STRATEGY
ENGAGEMENT
Evaluates and, if feasible, delivers a
single ML application (model, code,
docs) using your data on Cloudera
architecture
Delivers a data strategy document with
a focus on ML, by identifying and
prioritizing ML use cases, and
recommending process design
Ongoing ML expert advising plus
access to CFFL research reports and
prototypes.
Expert guidance to fast track value and scale

3. 3
CLOUDERA FAST FORWARD LABS
Research reports

4. 4
CLOUDERA FAST FORWARD LABS
Federated Learning

5. 5 © Cloudera, Inc. All rights reserved.
• Privacy needed
• Bandwidth and power consumption are concerns
• High cost of data transfer
WHEN TO APPLY FEDERATED LEARNING

6. 6 © Cloudera, Inc. All rights reserved.
A SPECIAL CASE OF DISTRIBUTED MACHINE LEARNING

7. 7 © Cloudera, Inc. All rights reserved.
A SPECIAL CASE OF DISTRIBUTED MACHINE LEARNING

8. 8 © Cloudera, Inc. All rights reserved.
TEXT COMPLETION UNDER PRIVACY CONSTRAINTS

9. 9 © Cloudera, Inc. All rights reserved.
• https://canopy.cr/blog/
• https://snips.ai/
• https://www.silklabs.com/ (recently acquired by Apple)
• https://owkin.com/
COMPANIES USING FEDERATED LEARNING

10. 10 © Cloudera, Inc. All rights reserved.
• When more data won’t improve your model (construct a learning curve!)
• When additional data is uncorrelated
• Performance is already at ceiling
WHEN NOT TO APPLY FEDERATED LEARNING

11. 11 © Cloudera, Inc. All rights reserved.
WHAT TO CONSIDER BEFORE YOU BEGIN
• Benchmark performance first
• Nodes must be devices that can train a model
• Determine how you’ll be transferring the models

12. 12

13. 13

14. 14

15. 15

16. 16

17. 17

18. 18

19. 19

20. 20 © Cloudera, Inc. All rights reserved.
FEDERATED AVERAGING
• Element-wise averaging of the coefficients
• Works for many different algorithms
see Communication-Efficient Learning of Deep Networks from Decentralized Data by McMahan et al. 2016

21. 2
1

22. 2
2

23. 2
3

24. 24 © Cloudera, Inc. All rights reserved.
OPTIMIZE WHEN TO MAINTAIN
• Maintaining at the wrong time is expensive
• Reluctant to share data with suppliers
• Getting data from remote locations is hard

25. 25 © Cloudera, Inc. All rights reserved.
STRATEGIES
• Corrective
• Failure has already occured
• Preventative
• Maintain on a schedule
• Predictive
• Smartly guess when a failure will occur

26. https://turbofan.fastforwardlabs.com/

27. 2
7

28. 28 © Cloudera, Inc. All rights reserved.
LEARN MORE
• https://vision.cloudera.com/an-introduction-to-federated-learning/
• https://blog.fastforwardlabs.com/2018/11/14/federated-learning.html
• https://www.cloudera.com/more/events/webinars/federated_learning.html
• https://ai.googleblog.com/2017/04/federated-learning-collaborative.html

29. THANK YOU