Downloaded 16 times
Uploaded byAnup kumar
Federated Machine Learning Framework
Federated learning allows training of machine learning models across decentralized data by using a centralized aggregator. The document discusses IBM's approach to federated learning including its Python framework, supported machine learning models and libraries, communication methods, and security features. It provides an overview of the basic federated learning process and architecture with local models trained at each party and aggregated into a shared model.
More Related Content
Similar to Federated Machine Learning Framework
![[OREILLY AI LONDON 2019] Anomaly Detection in Smart Buildings using Federated...](https://cdn.slidesharecdn.com/ss_thumbnails/oreillyailondon2019-250320154727-d2c2115c-thumbnail.jpg?width=640&height=640&fit=bounds)
Federated Machine Learning Framework
- 1. Federated Learning Hifaz Hassanand Velmurugan Ramadasan IBM Research Singapore
- 2. Agenda 2 © Copyright IBMCorporation 2020 Federated Learning concepts and architecture overview IBM Federated Learning approach Live interactive Federated Learning demo Q&A
- 3. Enterprise Data is Fragmented Moving data across silos is costly, risky, and slow Duplication is expensive AI applications must be built and run across different clouds Data privacy regulations prohibit cross—border data transfers 3 3 © Copyright IBM Corporation 2020
- 4. What is Federate d Learning ? © CopyrightIBM Corporation 2020 Provides an approach to train machine learning models in which data cannot be centralized for a training process. Distributed machine learning process in which multiple participants (parties) collaborate to train machine learning model without sharing data Interaction between parties via learning protocol coordinated by aggregator.
- 5. Edge and Embedded •Predictive Maintenance at the Edge • Process Optimization Consortia • Healthcare – Patient Analytics, Computer Vision • Financial services – Fraud Detection, Money Laundering Mobile Applications • Location Driven Predictions • Auto Completion Enterprises Need Federated Learning Federated Learning can be applied in a wide variety of use cases across hybrid cloud, multi-cloud, mobile and edge devices. 5 © Copyright IBM Corporation 2020
- 6. Basic Federated Learning 6 … Aggregator(A) D1 D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) © Copyright IBM Corporation 2020
- 7. Basic Federated Learning 7 … Aggregator(A) Q D1 Q Q D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) 1. Aggregator queries each parties about information necessary for learning a predictive model. (e.g. Weights, Gradients, Samples Counts). © Copyright IBM Corporation 2020
- 8. Basic Federated Learning 8 … Aggregator(A) Q D1 Q Q D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) 1. Aggregator queries each parties about information necessary for learning a predictive model. (e.g. Weights, Gradients, Samples Counts). 2. Given the query (Q), each party computes a reply (R) based on their own local data (D). R1=Q(D1) R2=Q(D2) RN=Q(DN) © Copyright IBM Corporation 2020
- 9. Basic Federated Learning 9 … Aggregator(A) R1 Q D1 R2 Q RN Q D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) 1. Aggregator queries each parties about information necessary for learning a predictive model. (e.g. Weights, Gradients, Samples Counts). 2. Given the query (Q), each party computes a reply (R) based on their own local data (D). R1=Q(D1) R2=Q(D2) RN=Q(DN) 3. Each party then sends its computed reply (R) back to the aggregator, where the results are then fused together as a single model (M). Model (M) F(R1, R2, …, RN) © Copyright IBM Corporation 2020
- 10. Basic Federated Learning 10 … Aggregator(A) R1 Q D1 R2 Q RN Q D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) 1. Aggregator queries each parties about information necessary for learning a predictive model. (e.g. Weights, Gradients, Samples Counts). 2. Given the query (Q), each party computes a reply (R) based on their own local data (D). R1=Q(D1) R2=Q(D2) RN=Q(DN) 3. Each party then sends its computed reply (R) back to the aggregator, where the results are then fused together as a single model (M). Model (M) F(R1, R2, …, RN) Key Point: Raw data from each party is never shared, it remains where it is stored. © Copyright IBM Corporation 2020
- 11. Definition of anFL Job Aggregator Side Fusion Algorithm Local Training Data Pre-Processing For each party …
- 12. IBM FL Approach 12 ©Copyright IBM Corporation 2020 Python based framework which provides fabric to implement FL applications Supports wide variety of machine learning models like tensorflow, pytorch, sci-kit learn and more Provides communication layer with many in-built connectors like flask, web sockets and RabbitMQ Supports wide variety of fusion algorithms and other security features like crypto, differential privacy and more Available as IBM FL CE library for developer community and IBM Cloud service for enterprise deployment (Beta Release)
- 13. IBM FL Approach … Aggregator(A) D1 D2 DN Party 1 (P1) Party 2 (P2) Party N (PN) © Copyright IBM Corporation 2020 Party Client Library Party Client Library Party Client Library Aggregator Fusion Algo Local Model Local Model
- 14. IBM Federated LearningFeatures Fusion Algos • FedAvg McMahan et al. • FedAvgPlus • Cordinate Median • PFNMYurochkin et al. • Krum • Zeno and more Supported ML Libraries • Sklearn • Keras • Tensorflow • Pytorch • RLlib Supported Models • Neural networks • XGBoost • Linear/Logistic regressors/classifiers • Decision Tree ID3 • K-means • Reinforcement Learning Connectivity • Web sockets • gRPC • Flask • Rabbit MQ FL Process Mgmt • Early termination • Quorum support Data Handlers • MNIST • FEMNIST • Adult • Diabetes • CIFAR-10 and more Others • Crypto • Experiment Manager • Differential Privacy • Fairness * Features only available in IBM Cloud version
- 15. Configurable Federated LearningStacks 15 15 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator Stack Party Stack © Copyright IBM Corporation 2020
- 16. Configurable Communication 16 16 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator StackParty Stack © Copyright IBM Corporation 2020 Communications for deployment scenario (https/Flask, RabbitMQ, Web- socket, …)
- 17. Supports Learning ProtocolInnovation 17 17 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator Stack Party Stack © Copyright IBM Corporation 2020 Different learning protocols
- 18. Abstracts from MLLibrary used 18 18 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator Stack Party Stack © Copyright IBM Corporation 2020 Wraps ML libra- ries such as Keras, PyTorch, RLLib, SKLearn Classifiers, ..
- 19. New FL Algosare matching pairs of Fusion and Training Handlers 19 19 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator Stack Party Stack © Copyright IBM Corporation 2020 Aggregator side FL code … Party side FL code …
- 20. Data Handlers: ReadData Party-Specifically 20 20 FLConnection ProtoHandler FusionHandler FLModel Aggregator FLConnection PartyProtoHandler TrainingHandler FLModel Party DataHandler Aggregator Stack Party Stack © Copyright IBM Corporation 2020
- 21. 21 © Copyright IBMCorporation 2020 Steps to train a neural network in IBM Federated learning 1. Specify any Keras, Pytorch, or TensorFlow model 2. Configure the federation, select the fusion algorithm AggregatorConfig File Snippet Party Config File Snippet 3. Train In all this process, parties keep the training data to themselves.
- 22. 22 © Copyright IBMCorporation 2020 IBMFL Job FLow
- 23. IBM FL CommunityEdition 23 Github: https://github.com/IBM/federated- learning-lib Getting Started :- 1. examples 2. Experiment Manager notebook Web site: https://ibmfl.mybluemix.net Slack Channel : https://ibm-fl.slack.com/ Distributes as WHL file, not open source
- 24. 24 © Copyright IBMCorporation 2020 Live Federated Learning Demo Train TensorFlow model using MNIST dataset
- 25. Thank you © CopyrightIBM Corporation 2020. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM’s current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and ibm.com are trademarks of IBM Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available at Copyright and trademark information. IBM Research AI/ © 2020 IBM Corporation 25 IBM FL Web Site https://ibmfl.mybluemix.net https://arxiv.org/abs/2007.10987 IBM FL White Paper Github https://github.com/IBM/federated-learning-lib
Editor's Notes
- #2 https://event.on24.com/wcc/r/2282646/3912A6162DF3F3014A7D7CEC244A8E08
- #6 Here are a subset of use cases and industries where Federated Learning is already being used. There are many use cases for Financial Services, Manufacturing, Healthcare, and Mobile Applications. Something common across the use cases and industries are that they require Strict data privacy rules, both for regulatory and competitive reasons ---- Traditionally, banks use rule-based and manual efforts to detect fraud and risk, which is prone to human error Risky small and micro enterprise loans are an important rising indicator of bank success, without credit risk identification Save time from banks using manual and rule-based human identification of fraud and increase accuracy which saves money and better customer experience Webank united several banks and trained anti money laundering models jointly LINK High, given IBM’s expertise in the area of Fraud analysis Predictive Maintenance & Quality (Manufacturing) Largest amount of data from sensors/IOT devices on individual machines Data cannot be gathered fast enough in one place to do analysis Unscheduled machine break down is a top challenge that can derail the business Achieve high asset utilization and savings in operational costs Model trained on many factories’ data is more robust in predicting failures ByteLake, an AI consultancy based in Poland, recently released a POC in concert with Lenovo for predictive maintenance LINK High, as IBM offers a PMQ solution that can be applied Patient Analytics (Healthcare) Cannot freely share or pool patient data due to policies like GDPR, HIPAA, CCPA etc. Need for more complex analysis data sets, such as medical images or from medical sensors Data is also a valuable proprietary resource for pharma/healthcare organizations Delivers exceptional performance in deep learning while keeping patient data secure and private Using pre-trained models and transfer learning techniques, NVIDIA AI assists radiologists in labeling, reducing the time for complex 3D studies from hours to minutes LINK High, as IBM offers strong capabilities in medical imaging analysis (Watson) and IOT Motivation to adopt FL Solution Benefits Competitive Use cases Ability for IBM to Execute this use case