Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Federated Learning: ML with Privacy on the Edge 11.15.18

1,001 views

Published on

Join Cloudera Fast Forward Labs Research Engineer, Mike Lee Williams, to hear about their latest research report and prototype on Federated Learning. Learn more about what it is, when it’s applicable, how it works, and the current landscape of tools and libraries.

Published in: Technology
  • Be the first to comment

Federated Learning: ML with Privacy on the Edge 11.15.18

  1. 1. © Cloudera, Inc. All rights reserved. FEDERATED LEARNING Mike Lee Williams (Cloudera) Virginia Smith (CMU), Eric Tramel (Owkin), and Andrew Trask (OpenMined)
  2. 2. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. THE FEDERATED LEARNING SETTING 4
  3. 3. © Cloudera, Inc. All rights reserved. 5
  4. 4. © Cloudera, Inc. All rights reserved. 6
  5. 5. © Cloudera, Inc. All rights reserved. 7
  6. 6. © Cloudera, Inc. All rights reserved. 8
  7. 7. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. FEDERATED AVERAGING Communication-Efficient Learning of Deep Networks from Decentralized Data by McMahan et al. 2016 9
  8. 8. © Cloudera, Inc. All rights reserved. 10
  9. 9. © Cloudera, Inc. All rights reserved. 11
  10. 10. © Cloudera, Inc. All rights reserved. 12
  11. 11. © Cloudera, Inc. All rights reserved. 13
  12. 12. © Cloudera, Inc. All rights reserved. 14
  13. 13. © Cloudera, Inc. All rights reserved. 15
  14. 14. © Cloudera, Inc. All rights reserved. 16
  15. 15. © Cloudera, Inc. All rights reserved. 17
  16. 16. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. TURBOFAN TYCOON turbofan.fastforwardlabs.com 18
  17. 17. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. CHALLENGES 19
  18. 18. © Cloudera, Inc. All rights reserved. Power consumption Dropped connections Stragglers
  19. 19. © Cloudera, Inc. All rights reserved. 21
  20. 20. © Cloudera, Inc. All rights reserved. 22 Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning by Hitaj et al. (2017)
  21. 21. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. USE CASES • Predictive maintenance/industrial IOT • Smartphones • Healthcare (wearables, drug discovery, prognostics, etc.) • Browsers • Retail video analytics • Enterprise/corporate IT (chat, issue trackers, email, etc.) 23
  22. 22. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. Q&A 24
  23. 23. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. ANDREW TRASK OPENMINED 25 What is the OpenMined project? What are secure aggregation and differential privacy? Could you tell us about PySyft? And what is the open source landscape like for federated learning? What other tools are out there to build upon?
  24. 24. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. ERIC TRAMEL OWKIN 26 What are the use cases for federated learning in healthcare? What healthcare challenges is Owkin working on? Can you tell us a little bit about how what you do differs algorithmically from the simple version described earlier? And what problems are you still working on?
  25. 25. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. VIRGINIA SMITH CMU 27 What does it mean to personalize federated learning? What are the use cases for this, and how does it work at a high level? Does federated learning work, i.e. do we end up with the same trained model as we’d have got it we moved the training data to our datacenter? What open problems are you interested in?
  26. 26. © Cloudera, Inc. All rights reserved.© Cloudera, Inc. All rights reserved. Cloudera Fast Forward Labs • An introduction to Federated Learning (Cloudera VISION blog, business audience) • Federated learning: distributed machine learning with data locality and privacy (FFL blog, more technical) • Turbofan Tycoon (working prototype, see FFL blog post for some details) Other blog posts • Collaborative Machine Learning without Centralized Training Data (Google research blog) • Federated Learning for Firefox (Firefox on florian.github.io) • Federated Learning for wake word detection (snips.ai on medium.com) Papers • Communication-Efficient Learning of Deep Networks from Decentralized Data by McMahan et al. (Google, 2016) • Practical Secure Aggregation for Privacy-Preserving Machine Learning by Bonawitz et al. (Google, 2017) • Federated Multi-Task Learning by Smith et al. (2017) • A generic framework for privacy preserving deep learning by Ryffel et al. (2018, and see also github.com/OpenMined/PySyft) • Federated Learning for Mobile Keyboard Prediction by Hard et al. (Google, 2018) 28

×