Federated Machine Learning (FedML) is a distributed machine learning approach which enables training on decentralised data. A server coordinates a network of nodes, each of which has local, private training data. The nodes contribute to the construction of a global model by training on local data , and the server combines non-sensitive node model contributions into the global model. Federated learning addresses fundamental problems of centralized AI such as privacy, ownership, and locality of data. It extends, even disrupts, the centralized AI paradigm in which better algorithms always comes at the cost of collecting more and more sensitive data.
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Dataconomy Media
"Machine learning algorithms require significant amounts of training data which has been centralized on one machine or in a datacenter so far. For numerous applications, such need of collecting data can be extremely privacy-invasive. Recent advancements in AI research approach this issue by a new paradigm of training AI models, i.e., Federated Learning.
In federated learning, edge devices (phones, computers, cars etc.) collaboratively learn a shared AI model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud. From personal data perspective, this paradigm enables a way of training a model on the device without directly inspecting users’ data on a server. This talk will pinpoint several examples of AI applications benefiting from federated learning and the likely future of privacy-aware systems."
Slide about working of federated learning and the introduction of machine learning and how user privacy is preserved in future machine learning approach.
A sharing talk in Hsinchu Coders.
The materials (i.e. images) are from their respective owners:
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
Federated Learning: ML with Privacy on the Edge 11.15.18Cloudera, Inc.
Join Cloudera Fast Forward Labs Research Engineer, Mike Lee Williams, to hear about their latest research report and prototype on Federated Learning. Learn more about what it is, when it’s applicable, how it works, and the current landscape of tools and libraries.
Federated Learning makes it possible to build machine learning systems without direct access to training data. The data remains in its original location, which helps to ensure privacy, reduces network communication costs, and taps edge device computing resources. The principles of data minimization established by the GDPR, and the growing prevalence of smart sensors make the advantages of federated learning more compelling. Federated learning is a great fit for smartphones, industrial and consumer IoT, healthcare and other privacy-sensitive use cases, and industrial sensor applications.
We’ll present the Fast Forward Labs team’s research on this topic and the accompanying prototype application, “Turbofan Tycoon”: a simplified working example of federated learning applied to a predictive maintenance problem. In this demo scenario, customers of an industrial turbofan manufacturer are not willing to share the details of how their components failed with the manufacturer, but want the manufacturer to provide them with a strategy to maintain the part. Federated learning allows us to satisfy the customer's privacy concerns while providing them with a model that leads to fewer costly failures and less maintenance downtime.
We’ll discuss the advantages and tradeoffs of taking the federated approach. We’ll assess the state of tooling for federated learning, circumstances in which you might want to consider applying it, and the challenges you’d face along the way.
Speaker
Chris Wallace
Data Scientist
Cloudera
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Dataconomy Media
"Machine learning algorithms require significant amounts of training data which has been centralized on one machine or in a datacenter so far. For numerous applications, such need of collecting data can be extremely privacy-invasive. Recent advancements in AI research approach this issue by a new paradigm of training AI models, i.e., Federated Learning.
In federated learning, edge devices (phones, computers, cars etc.) collaboratively learn a shared AI model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud. From personal data perspective, this paradigm enables a way of training a model on the device without directly inspecting users’ data on a server. This talk will pinpoint several examples of AI applications benefiting from federated learning and the likely future of privacy-aware systems."
Slide about working of federated learning and the introduction of machine learning and how user privacy is preserved in future machine learning approach.
A sharing talk in Hsinchu Coders.
The materials (i.e. images) are from their respective owners:
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
Federated Learning: ML with Privacy on the Edge 11.15.18Cloudera, Inc.
Join Cloudera Fast Forward Labs Research Engineer, Mike Lee Williams, to hear about their latest research report and prototype on Federated Learning. Learn more about what it is, when it’s applicable, how it works, and the current landscape of tools and libraries.
Federated Learning makes it possible to build machine learning systems without direct access to training data. The data remains in its original location, which helps to ensure privacy, reduces network communication costs, and taps edge device computing resources. The principles of data minimization established by the GDPR, and the growing prevalence of smart sensors make the advantages of federated learning more compelling. Federated learning is a great fit for smartphones, industrial and consumer IoT, healthcare and other privacy-sensitive use cases, and industrial sensor applications.
We’ll present the Fast Forward Labs team’s research on this topic and the accompanying prototype application, “Turbofan Tycoon”: a simplified working example of federated learning applied to a predictive maintenance problem. In this demo scenario, customers of an industrial turbofan manufacturer are not willing to share the details of how their components failed with the manufacturer, but want the manufacturer to provide them with a strategy to maintain the part. Federated learning allows us to satisfy the customer's privacy concerns while providing them with a model that leads to fewer costly failures and less maintenance downtime.
We’ll discuss the advantages and tradeoffs of taking the federated approach. We’ll assess the state of tooling for federated learning, circumstances in which you might want to consider applying it, and the challenges you’d face along the way.
Speaker
Chris Wallace
Data Scientist
Cloudera
byteLAKE and Lenovo presenting Federated Learning at MWC 2019byteLAKE
byteLAKE and Lenovo presenting Federated Learning for IoT live on stage at #MWC19
• real time machine learning
• data stays on edge, only models travel beyond
• leverage on all local AI models across IoT distributed infrastructure
More at: https://www.bytelake.com/en/federated-learning/ and www.byteLAKE.com/en/MWC19
A Privacy Framework for Hierarchical Federated LearningDebmalya Biswas
Federated Learning (FL) enables heterogeneous entities to collaboratively develop an optimized (global) model by sharing data and models in a privacy preserving fashion. We consider a Hierarchical Federated Learning (HFL) environment with data ownership split among the entities representing the edge nodes. Each node can train models on the data they own, as well as request access to data and model(s) owned by their descendant nodes-to optimize their models, perform transfer learning on new data, and develop an ensemble model. Unfortunately, a practical realization of HFL is challenging today due to issues with data/model lineage tracking and providing subsequent privacy guarantees. In this paper, we propose a conceptual framework for HFL by capturing the data/model attributes at each node, including their privacy exposure. The framework enables scenarios where a node output may expose certain attributes of its underlying data, as well as identifying models in the hierarchy that need to be updated once a user whose data was used in their training has opted-out. By designing the computations appropriately and limiting the exposure by the nodes, we show that different levels of privacy can be guaranteed.
Can we use data to train Machine Learning models, perform statistical analysis, yet without putting private data on risk? There are tools and techniques such as Federated Learning, Differential Privacy or Homomorphic Encryption enabling safer work on the data.
Here we describe federated learning based traffic flow prediction system. In federated learning we solve the problem of data security and also provide collaborative learning. model parameter are shared here ,not data
Security and Privacy of Machine LearningPriyanka Aash
Machine learning is a powerful new tool that can be used for security applications (for example, to detect malware) but machine learning itself introduces many new attack surfaces. For example, attackers can control the output of machine learning models by manipulating their inputs or training data. In this session, I give an overview of the emerging field of machine learning security and privacy.
Learning Objectives:
1: Learn about vulnerabilities of machine learning.
2: Explore existing defense techniques (differential privacy).
3: Understand opportunities to join research effort to make new defenses.
(Source: RSA Conference USA 2018)
Explainable AI makes the algorithms to be transparent where they interpret, visualize, explain and integrate for fair, secure and trustworthy AI applications.
Poisoning attacks on Federated Learning based IoT Intrusion Detection SystemSai Kiran Kadam
Attacks on federated learning model are discussed as a part of my research to build a model that overcomes the diverse security issues and vulnerabilities in the cloud in the process of building a unified machine learning model that can benefit multi-user/ multi-companies to work together.
Currently hundreds of tools are promising to make artificial intelligence accessible to the masses. Tools like DataRobot, H20 Driverless AI, Amazon SageMaker or Microsoft Azure Machine Learning Studio.
These tools promise to accelerate the time-to-value of data science projects by simplifying model building.
In the workshop we will approach the AI Topic head on!
What is AI? What can AI do today? What do I need to start my own project?
We do all this using Microsoft's Machine Learning Studio.
Trainer: Philipp von Loringhoven - Chef, Designer, Developer, Markeeter - Data Nerd!
He has acquired a lot of expertise in marketing, business intelligence and product development during his time at the Rocket Internet startups (Wimdu, Lamudi) and Projekt-A (Tirendo).
Today he supports customers of the Austrian digitisation agency TOWA as Director Data Consulting to generate an added value from their data.
Regulating Generative AI - LLMOps pipelines with TransparencyDebmalya Biswas
The growing adoption of Gen AI, esp. LLMs, has re-ignited the discussion around AI Regulations — to ensure that AI/ML systems are responsibly trained and deployed. Unfortunately, this effort is complicated by multiple governmental organizations and regulatory bodies releasing their own guidelines and policies with little to no agreement on the definition of terms.
Rather than trying to understand and regulate all types of AI, we recommend a different (and practical) approach in this talk based on AI Transparency —
to transparently outline the capabilities of the AI system based on its training methodology and set realistic expectations with respect to what it can (and cannot) do.
We outline LLMOps architecture patterns and show how the proposed approach can be integrated at different stages of the LLMOps pipeline capturing the model's capabilities. In addition, the AI system provider also specifies scenarios where (they believe that) the system can make mistakes, and recommends a ‘safe’ approach with guardrails for those scenarios.
leewayhertz.com-Federated learning Unlocking the potential of secure distribu...KristiLBurns
Federated learning is a machine learning technique that enables multiple client devices to collaboratively train a shared model without exchanging individual data with each other or a central server.
byteLAKE and Lenovo presenting Federated Learning at MWC 2019byteLAKE
byteLAKE and Lenovo presenting Federated Learning for IoT live on stage at #MWC19
• real time machine learning
• data stays on edge, only models travel beyond
• leverage on all local AI models across IoT distributed infrastructure
More at: https://www.bytelake.com/en/federated-learning/ and www.byteLAKE.com/en/MWC19
A Privacy Framework for Hierarchical Federated LearningDebmalya Biswas
Federated Learning (FL) enables heterogeneous entities to collaboratively develop an optimized (global) model by sharing data and models in a privacy preserving fashion. We consider a Hierarchical Federated Learning (HFL) environment with data ownership split among the entities representing the edge nodes. Each node can train models on the data they own, as well as request access to data and model(s) owned by their descendant nodes-to optimize their models, perform transfer learning on new data, and develop an ensemble model. Unfortunately, a practical realization of HFL is challenging today due to issues with data/model lineage tracking and providing subsequent privacy guarantees. In this paper, we propose a conceptual framework for HFL by capturing the data/model attributes at each node, including their privacy exposure. The framework enables scenarios where a node output may expose certain attributes of its underlying data, as well as identifying models in the hierarchy that need to be updated once a user whose data was used in their training has opted-out. By designing the computations appropriately and limiting the exposure by the nodes, we show that different levels of privacy can be guaranteed.
Can we use data to train Machine Learning models, perform statistical analysis, yet without putting private data on risk? There are tools and techniques such as Federated Learning, Differential Privacy or Homomorphic Encryption enabling safer work on the data.
Here we describe federated learning based traffic flow prediction system. In federated learning we solve the problem of data security and also provide collaborative learning. model parameter are shared here ,not data
Security and Privacy of Machine LearningPriyanka Aash
Machine learning is a powerful new tool that can be used for security applications (for example, to detect malware) but machine learning itself introduces many new attack surfaces. For example, attackers can control the output of machine learning models by manipulating their inputs or training data. In this session, I give an overview of the emerging field of machine learning security and privacy.
Learning Objectives:
1: Learn about vulnerabilities of machine learning.
2: Explore existing defense techniques (differential privacy).
3: Understand opportunities to join research effort to make new defenses.
(Source: RSA Conference USA 2018)
Explainable AI makes the algorithms to be transparent where they interpret, visualize, explain and integrate for fair, secure and trustworthy AI applications.
Poisoning attacks on Federated Learning based IoT Intrusion Detection SystemSai Kiran Kadam
Attacks on federated learning model are discussed as a part of my research to build a model that overcomes the diverse security issues and vulnerabilities in the cloud in the process of building a unified machine learning model that can benefit multi-user/ multi-companies to work together.
Currently hundreds of tools are promising to make artificial intelligence accessible to the masses. Tools like DataRobot, H20 Driverless AI, Amazon SageMaker or Microsoft Azure Machine Learning Studio.
These tools promise to accelerate the time-to-value of data science projects by simplifying model building.
In the workshop we will approach the AI Topic head on!
What is AI? What can AI do today? What do I need to start my own project?
We do all this using Microsoft's Machine Learning Studio.
Trainer: Philipp von Loringhoven - Chef, Designer, Developer, Markeeter - Data Nerd!
He has acquired a lot of expertise in marketing, business intelligence and product development during his time at the Rocket Internet startups (Wimdu, Lamudi) and Projekt-A (Tirendo).
Today he supports customers of the Austrian digitisation agency TOWA as Director Data Consulting to generate an added value from their data.
Regulating Generative AI - LLMOps pipelines with TransparencyDebmalya Biswas
The growing adoption of Gen AI, esp. LLMs, has re-ignited the discussion around AI Regulations — to ensure that AI/ML systems are responsibly trained and deployed. Unfortunately, this effort is complicated by multiple governmental organizations and regulatory bodies releasing their own guidelines and policies with little to no agreement on the definition of terms.
Rather than trying to understand and regulate all types of AI, we recommend a different (and practical) approach in this talk based on AI Transparency —
to transparently outline the capabilities of the AI system based on its training methodology and set realistic expectations with respect to what it can (and cannot) do.
We outline LLMOps architecture patterns and show how the proposed approach can be integrated at different stages of the LLMOps pipeline capturing the model's capabilities. In addition, the AI system provider also specifies scenarios where (they believe that) the system can make mistakes, and recommends a ‘safe’ approach with guardrails for those scenarios.
leewayhertz.com-Federated learning Unlocking the potential of secure distribu...KristiLBurns
Federated learning is a machine learning technique that enables multiple client devices to collaboratively train a shared model without exchanging individual data with each other or a central server.
Federated learning and its role in the privacy preservation of IoT devicesAlAtfat
Federated learning (FL) is a cutting-edge artificial intelligence approach. It is a decentralized problem-solving technique that allows users to train using massive data. Unprocessed information is stored in advanced technology by a secret confidentiality service, which incorporates machine learning (ML) training while removing data connections. As researchers in the field promote ML configurations containing a large amount of private data, systems and infrastructure must be developed to improve the effectiveness of advanced learning systems. This study examines FL in-depth, focusing on application and system platforms, mechanisms, real-world applications, and process contexts. FL creates robust classifiers without requiring information disclosure, resulting in highly secure privacy policies and access control privileges. The article begins with an overview of FL. Then, we examine technical data in FL, enabling innovation, contracts, and software. Compared with other review articles, our goal is to provide a more comprehensive explanation of the best procedure systems and authentic FL software to enable scientists to create the best privacy preservation solutions for IoT devices. We also provide an overview of similar scientific papers and a detailed analysis of the significant difficulties encountered in recent publications. Furthermore, we investigate the benefits and drawbacks of FL and highlight comprehensive distribution scenarios to demonstrate how specific FL models could be implemented to achieve the desired results.
Data locality and distribution
● massively decentralized, naturally arising
(non-IID) partition
● Data is siloed, held by a small number of
coordinating entities
● system-controlled (e.g. shuffled, balanced)
Data availability
● limited availability, time-of-day variations
● almost all data nodes
FAIR data_ Superior data visibility and reuse without warehousing.pdfAlan Morrison
The advantages of semantic knowledge graphs over data warehousing when it comes to scaling quality, contextualized data for machine learning and advanced analytics purposes.
this ppt about machine learning method federated learning that how helps to train the model without sharing the personal information from local devices
The FAIR data movement and 22 Feb 2023.pdfAlan Morrison
To realize the promise of FAIR data, companies must be data mature. They must adopt data-centric architecture and the #FAIR (findable, accessible, interoperable and reusable) principles. When they do, the data they need will be linked and self-describing. The data when queried will tell you where it is.
A desiloed, #semantic graph data abstraction--the only feasible means behind creating FAIR data at this point--is not only the means to data discovery, but also a path to model-driven development and data sharing at scale, both of which will break an organization's habit of duplicating data and logic.
This webinar highlights fresh enterprise case studies that are starting to realize the dream of #FAIRdata, as well as how these companies are succeeding:
- Zero copy integration: How to think about eliminating #dataduplication and stop the application buying binge that only exacerbates the problem.
- Dynamic, unified data model: Standard graphs provide a means of modeling once, use anywhere, for conceptual, logical and physical purposes all at once.
- Persuasion and teamwork: The #graph approach provides an ideal way to loop business units and domain experts in and empower them to recommend model changes that are easily implemented.
The whole process is bringing #enterprises like Walmart, Uber, Goldman Sachs and Nokia into the age of #contextualcomputing. Learn how to be a fast follower by thinking big, but starting small.
5 Key Data Management Trends of 2022 as observed by a data practitioner. Covers trends on data architecture, data storage, data platforms, and data operations.
ANALYSIS OF SYSTEM ON CHIP DESIGN USING ARTIFICIAL INTELLIGENCEijesajournal
Automation is a powerful word that lies everywhere. It shows that without automation, application will not
get developed. In a semiconductor industry, artificial intelligence played a vital role for implementing the
chip based design through automation .The main advantage of applying the machine learning & deep
learning technique is to improve the implementation rate based upon the capability of the society. The
main objective of the proposed system is to apply the deep learning using data driven approach for
controlling the system. Thus leads to a improvement in design, delay ,speed of operation & costs.
Through this system, huge volume of data’s that are generated by the system will also get control.
ANALYSIS OF SYSTEM ON CHIP DESIGN USING ARTIFICIAL INTELLIGENCEijesajournal
Automation is a powerful word that lies everywhere. It shows that without automation, application will not
get developed. In a semiconductor industry, artificial intelligence played a vital role for implementing the
chip based design through automation .The main advantage of applying the machine learning & deep learning technique is to improve the implementation rate based upon the capability of the society. The main objective of the proposed system is to apply the deep learning using data driven approach for controlling the system. Thus leads to a improvement in design, delay ,speed of operation & costs.Through this system, huge volume of data’s that are generated by the system will also get control.
ANALYSIS OF SYSTEM ON CHIP DESIGN USING ARTIFICIAL INTELLIGENCEijesajournal
Automation is a powerful word that lies everywhere. It shows that without automation, application will not get developed. In a semiconductor industry, artificial intelligence played a vital role for implementing the chip based design through automation .The main advantage of applying the machine learning & deep learning technique is to improve the implementation rate based upon the capability of the society. The main objective of the proposed system is to apply the deep learning using data driven approach for controlling the system. Thus leads to a improvement in design, delay ,speed of operation & costs. Through this system, huge volume of data’s that are generated by the system will also get control.
Cloud Analytics Ability to Design, Build, Secure, and Maintain Analytics Solu...YogeshIJTSRD
Cloud Analytics is another area in the IT field where different services like Software, Infrastructure, storage etc. are offered as services online. Users of cloud services are under constant fear of data loss, security threats, and availability issues. However, the major challenge in these methods is obtaining real time and unbiased datasets. Many datasets are internal and cannot be shared due to privacy issues or may lack certain statistical characteristics. As a result of this, researchers prefer to generate datasets for training and testing purposes in simulated or closed experimental environments which may lack comprehensiveness. Advances in sensor technology, the Internet of things IoT , social networking, wireless communications, and huge collection of data from years have all contributed to a new field of study Big Data is discussed in this paper. Through this analysis and investigation, we provide recommendations for the research public on future directions on providing data based decisions for cloud supported Big Data computing and analytic solutions. This paper concentrates upon the recent trends in Big Data storage and analysing, in the clouds, and also points out the security limitations. Rajan Ramvilas Saroj "Cloud Analytics: Ability to Design, Build, Secure, and Maintain Analytics Solutions on the Cloud" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-5 , August 2021, URL: https://www.ijtsrd.com/papers/ijtsrd43728.pdf Paper URL: https://www.ijtsrd.com/other-scientific-research-area/other/43728/cloud-analytics-ability-to-design-build-secure-and-maintain-analytics-solutions-on-the-cloud/rajan-ramvilas-saroj
FEDERATED LEARNING FOR PRIVACY-PRESERVING: A REVIEW OF PII DATA ANALYSIS IN F...ijseajournal
There has been tremendous growth in the field of AI and machine learning. The developments across these
fields have resulted in a considerable increase in other FinTech fields. Cyber security has been described
as an essential part of the developments associated with technology. Increased cyber security ensures that
people remain protected, and that data remains safe. New methods have been integrated into developing AI
that achieves cyber security. The data analysis capabilities of AI and its cyber security functions have
ensured that privacy has increased significantly. The ethical concept associated with data privacy has also
been advocated across most FinTech regulations. These concepts and considerations have all been
engaged with the need to achieve the required ethical requirements. The concept of federated learning is a
recently developed measure that achieves the abovementioned concept. It ensured the development of AI
and machine learning while keeping privacy in data analysis. The research paper effectively describes the
issue of federated learning for confidentiality. It describes the overall process associated with its
development and some of the contributions it has achieved. The widespread application of federated
learning in FinTech is showcased, and why federated learning is essential for overall growth in FinTech.
Similar to Big Data Stockholm v 7 | "Federated Machine Learning for Collaborative and Secure AI" - Andreas Hellander (20)
Data Natives Frankfurt v 11.0 | "Competitive advantages with knowledge graphs...Dataconomy Media
The challenges of increasing complexity of organizations, companies and projects are obvious and omnipresent. Everywhere there are connections and dependencies that are often not adequately managed or not considered at all because of a lack of technology or expertise to uncover and leverage the relationships in data and information. In his presentation, Axel Morgner talks about graph technology and knowledge graphs as indispensable building blocks for successful companies.
Data Natives Munich v 12.0 | "How to be more productive with Autonomous Data ...Dataconomy Media
Every day we are challenged with more data, more use cases and an ever increasing demand for analytics. In this talk Bjorn will explain how autonomous data management and machine learning help innovators to more productive and give examples how to deliver new data driven projects with less risk at lower costs.
Data Natives meets DataRobot | "Build and deploy an anti-money laundering mo...Dataconomy Media
Compliance departments within banks and other financial institutions are turning to machine learning for improving their Anti Money Laundering compliance activities. Today, the systems that aim to detect potentially suspicious activity are commonly rule-based, and suffer from ultra-high false positive rates. DataRobot will discuss how their Automated Machine Learning platform was successfully used for a real use case to reduce their false positives and to enhance their Anti-Money Laundering activities.
Data Natives Munich v 12.0 | "Political Data Science: A tale of Fake News, So...Dataconomy Media
Trump, Brexit, Cambridge Analytica... In the last few years, we have had to confront the consequences of the use and misuse of data science algorithms in manipulating public opinion through social media. The use of private data to microtarget individuals is a daily practice (and a trillion-dollar industry), which has serious side-effects when the selling product is your political ideology. How can we cope with this new scenario?
Data Natives Vienna v 7.0 | "The Ingredients of Data Innovation" - Robbert de...Dataconomy Media
When taking a deep dive into the world of data, one thing is certain: the ultimate goal is to create something new, something better, something faster. In other words, innovation should always be at the forefront of companies strategic outlook, whether their goal is to pioneer new processes, user experiences, products or services.
Data Natives Cologne v 4.0 | "The Data Lorax: Planting the Seeds of Fairness...Dataconomy Media
What does it take to build a good data product or service? Data practitioners always think about the technology, user experience and commercial viability. But rarely do they think about the implications of the systems they build. This talk will shed light on the impact of AI systems and the unintended consequences of the use of data in different products. It will also discuss our role, as data practitioners, in planting the seeds of fairness in the systems we build.
Data Natives Cologne v 4.0 | "How People Analytics Can Reveal the Hidden Aspe...Dataconomy Media
We all hear about the power of data, big data and data analysis in todays market place. But rarely feel it's touchable effects on our own business decisions and performance.
Let's dive into it and see how can people analytics increase people performance, motivation and business revenue?
Data Natives Amsterdam v 9.0 | "Ten Little Servers: A Story of no Downtime" -...Dataconomy Media
Cloud Infrastructure is a hostile environment: a power supply failure or a network outage leads to downtime and big losses. There is nothing we can trust: a single server, a server rack, even a whole datacenter can fail, and if an application is fragile by design, disruption is inevitable. We must distribute our application and diversify cloud data strategy to survive disturbances of any scale. Apache Cassandra is a cloud-native platform-agnostic database that stores data with a distributed redundancy so it easily survives any issue. What to know how Apple and Netflix handle petabytes of data, keeping it highly available? Join us and listen to a story of 10 little servers and no downtime!
Data Natives Amsterdam v 9.0 | "Point in Time Labeling at Scale" - Timothy Th...Dataconomy Media
In the data industry, having correctly labelled datasets is vital. Timothy Thatcher explains how tagging your data while considering time and location and complex hierarchical rules at scale can be handled.
Data NativesBerlin v 20.0 | "Serving A/B experimentation platform end-to-end"...Dataconomy Media
During the lifetime of an A/B test product managers and analysts in GetYourGuide require various tools and different kinds of data to plan the trial properly, control it during the run and analyze the results at the end. This talk would be about the architecture, tools and data flow for serving their needs.
Data Natives Berlin v 20.0 | "Ten Little Servers: A Story of no Downtime" - A...Dataconomy Media
Cloud Infrastructure is a hostile environment: a power supply failure or a network outage leads to downtime and big losses. There is nothing we can trust: a single server, a server rack, even a whole datacenter can fail, and if an application is fragile by design, disruption is inevitable. We must distribute our application and diversify cloud data strategy to survive disturbances of any scale. Apache Cassandra is a cloud-native platform-agnostic database that stores data with a distributed redundancy so it easily survives any issue. What to know how Apple and Netflix handle petabytes of data, keeping it highly available? Join us and listen to a story of 10 little servers and no downtime!
Big Data Frankfurt meets Thinkport | "The Cloud as a Driver of Innovation" - ...Dataconomy Media
Creativity is the mental ability to create new ideas and designs. Innovation, on the other hand, Means developing useful solutions from new ideas. Creativity can be goal-oriented, Whereas innovation is always goal-oriented. This bedeutet, dass innovation aims to achieve defined goals. The use of cloud services and technologies promises enterprise users many benefits in terms of more flexible use of IT resources and faster access to innovative solutions. That’s why we want to examine the question in this talk, of what role cloud computing plays for innovation in companies.
Thinkport meets Frankfurt | "Financial Time Series Analysis using Wavelets" -...Dataconomy Media
Presentation of Time Series Properties of Financial Instrument and Possibilities in Frequency Decomposition and Information Extraction using FT, STFT and Wavelets with Outlook in Current Research on Wavelet Neural Networks
Big Data Helsinki v 3 | "Distributed Machine and Deep Learning at Scale with ...Dataconomy Media
"With most machine learning (ML) and deep learning (DL) frameworks, it can take hours to move data for ETL, and hours to train models. It's also hard to scale, with data sets increasingly being larger than the capacity of any single server. The amount of the data also makes it hard to incrementally test and retrain models in near real-time.
Learn how Apache Ignite and GridGain help to address limitations like ETL costs, scaling issues and Time-To-Market for the new models and help achieve near-real-time, continuous learning.
Yuriy Babak, the head of ML/DL framework development at GridGain and Apache Ignite committer, will explain how ML/DL work with Apache Ignite, and how to get started.
Topics include:
— Overview of distributed ML/DL including architecture, implementation, usage patterns, pros and cons
— Overview of Apache Ignite ML/DL, including built-in ML/DL algorithms, and how to implement your own
— Model inference with Apache Ignite, including how to train models with other libraries, like Apache Spark, and deploy them in Ignite
— How Apache Ignite and TensorFlow can be used together to build distributed DL model training and inference"
Big Data Helsinki v 3 | "What you should know about PSD2 APIs?" - Joonas TomperiDataconomy Media
PSD2 is coming live in September 2019 and after that, you are able to get account data, make strong authentication and make payments through banks PSD2 APIs.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Key Trends Shaping the Future of Infrastructure.pdf
Big Data Stockholm v 7 | "Federated Machine Learning for Collaborative and Secure AI" - Andreas Hellander
1. Federated Machine Learning
Andreas Hellander
Co-founder and Lead Scientist, Scaleout Systems
Associate Professor in Scientific Computing, Uppsala University
scaleoutsystems.com it.uu.se
2. Main issues with the centralized paradigm
in machine learning:
● Private/Proprietary data — Sharing
valuable business data with someone
else is not an option.
● Regulated data — GDPR, HIPAA, etc.
● Practical blockers — data is too big,
the network connection is expensive,
slow or unreliable.
Also, large datasets relevant to AI
problems are controlled by a small number
of large organizations and there are no
great mechanisms for sharing that data
with the data science community.
scaleoutsystems.com
The data centralization problem
1. Collect and centralize data from
different sources (data lake, cloud).
2. Create ML model using centralised
data (cluster computing)
3. How can parties come together to create joint
ML models without sharing/pooling data?
4. Federated Machine Learning
Federated Machine Learning (FedML) is a
distributed machine learning approach
which enables training on decentralised
data.
● Train local machine learning model on
local/private data.
● Combine local model updates into a
global, federated model.
Federated learning addresses the
fundamental problems of centralized AI
such as privacy, ownership, and locality of
data.
scaleoutsystems.com/federated-machine-learning
5. The key benefit of FedML
Lets parties form alliances/networks to
build stronger models than what could be
attained by the parties in isolation.
● Data security and privacy where data
never moves.
● Powerful data network effects in
industries where data cannot be
transferred.
● Reduced data transfer costs when
data is very large or networks
unreliable.
scaleoutsystems.com/federated-machine-learning
N. Gauraha, O. Spjuth, A. Hellander (2019), manuscript in preparation
6. Early example
FedML on Gboard:
● Local model for search suggestion,
with context and whether suggestion
was clicked
● On device the history is processed,
and then only a model update is
suggested to Google
● Based on Federated Averaging, a
scheme to aggregate weights from
locally trained neural nets:
https://arxiv.org/pdf/1602.05629.pdf
scaleoutsystems.com/federated-machine-learning
https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
7. Smart software on top of decentralized
infrastructure/instruments ● Let’s an instrument/software vendor
build smarter software.
● Digital pathology, medical dosimetry,
predictive maintenance etc.
● Sensitive data does not need to be
shared.
● Powerful network effects possible.
scaleoutsystems.com/federated-machine-learning
Federated
Model
Software services
Federated learning system
Infrastructure vendor
8. Integrity-preserving E-health
● Digital tools/video surveillance in
home care.
● Train and deploy models based on
homeowners’ private interactions
without collecting central data.
scaleoutsystems.com/federated-machine-learning
9. Privacy-preservation features of FedML
● Input privacy simplified since data do
not move (handled according to local
policies)
● Output privacy - depends on the
algorithm, how easy it is to invert the
model etc.
● What can be learned from the
coordination of computation?
○ Different for federated averaging
and ensemble methods
(algorithm dependent)
scaleoutsystems.com/federated-machine-learning
UN Handbook for Privacy-Preserving Techniques
10. Differential privacy & homomorphic encryption in FedML
Differential privacy: Add
carefully calibrated noise
(protects against inference
attacks)
Homomorphic encryption: Methods
work on encrypted data
Secure multiparty computation:
Aggregate/compute without a
third party trust
provider/server.
scaleoutsystems.com/federated-machine-learning
11. R&D challenges
Scalability and ML
performance
How do we (re)design
algorithms and frameworks
to scale out to the fog and
edge?
Decentralized computation
How can we do FedML
without a third-party trust
provider?
Adversarial ML
How can we make the
system robust to dishonest
members and external
threats?
FedML is a research area that integrates many differents areas of
computer science and mathematics.
scaleoutsystems.com/federated-machine-learning
12. Backdooring federated learning
● Big threat to a FedML comes from
within the alliance / from
compromised members.
● Large alliances can be expected to be
relatively robust to data poisoning
attacks.
● Bagdasaryan et al. shows how their
proposed approach of model
replacement can efficiently introduce
backdoors in a global model.
● Secure aggregation/MPC makes it
impossible to detect a malicious
model update, and who submitted it!
scaleoutsystems.com/federated-machine-learning
Bagdasaryan et al. How to backdoor federated learning (2019) https://arxiv.org/pdf/1807.00459.pdf
13. Federated learning in production
Secure model
communication,
anomaly detection,
etc.
API Federated components
Global model
serving
ML pipeline
APIML pipeline
APIML pipeline
14. A problem that spans many complex areas
● Decentralized computing / fog computing
● Information and security/systems security expertise
● Trust-mechanisms (third-party or decentralized protocol)
● Machine learning algorithms designed for/adapted to a decentralized setting
● Adversarial ML
○ Data poisoning
○ Inference attacks
○ …
A considerable increase in system and developer complexity
compared to the standard paradigm!
scaleoutsystems.com/federated-machine-learning
15. Scaleout Federated Platform
Scaleout Studio | Developing Scaleout Store | Package & Deploying Scaleout Serve | Serving
Scaleout Federated Platform
ML studio
- Ingestion
- Prepare & Analyse Data
- Modeling & Testing
- Training
ML workflow automation
- Automated ML Studio
Pipelines
API
API
Model management
- Versioning
- Annotation
- Storage
- Distribution
API
Model
serving
- Traffic
management
- Authentication
/Authorization
- Policies
- Monitoring
Monitoring &
Visualizations
API
API
Endpoint registry
Graphical User Interface
Incl Pipeline Visualization
AuthenticationandAuthorization
Model Sharing
Joint Training
Federation
Orchestration
Federation Identity &
Security
Federation Cross Validation
& Holdout Set
scaleoutsystems.com/federated-machine-learning
16. scaleoutsystems.com
Thank you!
SCALEOUT
Bridging the gap between research and
production grade systems in machine
learning. Learn more about our Lean AI
framework, and our Federated Machine
Learning platform.
ANDREAS HELLANDER
andreas.hellander@it.uu.se
SALMAN TOOR
salman.toor@it.uu.se
Scaleout FedML platform demo at
Testa Center, GE Healthcare
https://www.youtube.com/watch?v=K-JUNkAYs-4