Federated Learning

© Cloudera, Inc. All rights reserved.
FEDERATED LEARNING
Chris J Wallace • Data Scientist • Cloudera Fast Forward Labs
@_cjwallace

Available to
Fast Forward Labs
clients
Play at turbofan.fastforwardlabs.com

© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved.
TODAY
● WHY CARE?
● FEDERATED AVERAGING
● PROTOTYPE
● CHALLENGES
● TOOLS

WHY CARE ABOUT FEDERATED LEARNING?
7

1
0

1
1

REQUIREMENTS FOR FEDERATED LEARNING
● Performance improves with more data.
● Models can be meaningfully combined.
● Nodes can train models, not only predict.
amount of data
performance

1
3

FEDERATED AVERAGING
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan et al. 2016
1
4

A network of nodes shares models rather than training data with the server

The server has an untrained model

It sends a copy of that model to the nodes

The nodes now also have the untrained model

The nodes have data on which to train their model

Each node trains the model to fit the data they have

Each node sends a copy of its trained model back to the
server

The server combines these models by taking an average
We repeat the whole process many times.

The server now has a model that captures
the patterns in the training data on all the nodes
But at no point did the nodes share their training data
which increases privacy and saves on bandwidth.

FEDERATED AVERAGING CAN HANDLE...
● Non-IID data
○ Training data on each node can be idiosyncratic.
● Unbalanced data
○ Unequal amount of data on each node.
● Massively distributed data
○ Can have many more devices than training examples per node.
● Limited communication
○ Cannot guarantee availability of nodes. Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan et al. 2016

TURBOFAN TYCOON
25
turbofan.fastforwardlabs.com

Corrective Preventative Predictive
PREDICTIVE MAINTENANCE

turbofan.fastforwardlabs.com
CMAPPS data set
https://ti.arc.nasa.gov/tech/dash/groups/pcoe/prognostic-data-repository/

CHALLENGES
28

© Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. 29
Power consumption Dropped connections Stragglers
SYSTEMS ISSUES

PRIVACY
Adversary can’t inspect data, but can inspect model.

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning, Hitaj et al. (2017)
PRIVACY

TOOLS
32

OpenMined
https://www.openmined.org/
“OpenMined is an open-source community focused on
researching, developing, and promoting tools for secure,
privacy-preserving, value-aligned artificial intelligence.”
● More than federated learning.
● PySyft is a library for privacy preserving deep learning.
● Grid is a peer-to-peer platform for decentralized data
science.

TensorFlow Federated
https://www.tensorflow.org/federated
● Federated Learning API
○ Wrap TensorFlow models in included FL
implementations.
○ High level, with attention paid to separating the
concerns of models, communication, and so on.
● Federated Core API
○ Low level interfaces for building novel FL algorithms.
Local simulation runtime only right now.
New, but promising.

SUMMARY

Federated Learning is machine learning on
decentralized data

● Privacy is needed (FL not the whole solution)
● Bandwidth or power consumption are concerns
● High cost of data transfer
● Your model improves with more data
YOU MIGHT HAVE A USE CASE IF …
37

● Predictive maintenance/industrial IOT
● Smartphones
● Healthcare (wearables, drug discovery, prognostics, etc.)
● Enterprise/corporate IT (chat, issue trackers, email, etc.)
EXAMPLES
38

Cloudera Fast Forward Labs
• An introduction to Federated Learning (Cloudera VISION blog, business audience)
• Federated learning: distributed machine learning with data locality and privacy (FFL blog, more technical)
• Turbofan Tycoon (working prototype, see FFL blog post for some details)
Other blog posts
• Collaborative Machine Learning without Centralized Training Data (Google research blog)
• Federated Learning for Firefox (Firefox on florian.github.io)
• Federated Learning for wake word detection (snips.ai on medium.com)
Papers
• Communication-Efficient Learning of Deep Networks from Decentralized Data by McMahan et al. (Google, 2016)
• Practical Secure Aggregation for Privacy-Preserving Machine Learning by Bonawitz et al. (Google, 2017)
• Federated Multi-Task Learning by Smith et al. (2017)
• A generic framework for privacy preserving deep learning by Ryffel et al. (2018, and see also github.com/OpenMined/PySyft)
• Federated Learning for Mobile Keyboard Prediction by Hard et al. (Google, 2018)
39

THANK YOU
cffl@cloudera.com
@_cjwallace

Federated Learning

More Related Content

What's hot

Similar to Federated Learning

More from DataWorks Summit

Recently uploaded

Federated Learning