This document discusses probabilistic modeling in Python using the pomegranate library. It begins with an overview of probabilistic modeling tasks and when to use pomegranate. It then discusses the major models supported by pomegranate including general mixture models, hidden Markov models, Bayesian networks, and Bayes classifiers. It provides examples of using these models and shows how pomegranate allows for complex probabilistic models to be built from simpler component distributions and submodels. The document concludes by demonstrating how mixtures of hidden Markov models can be trained in parallel using pomegranate.
Lecture 01: Machine Learning for Language Technology - IntroductionMarina Santini
What Is Machine Learning? Machine learning is programming computers to optimize a performance criterion using example data or past experience. We have a model defined up to some parameters, and learning is the execution of a computer program to optimize the parameters of the model using the training data or past experience. The model may be predictive to make predictions in the future, or descriptive to gain knowledge from data, or both. Machine learning uses the theory of statistics in building mathematical models, because the core task is making inference from a sample. (Alpaydin, 2010)
In this lecture, we discuss supervised learning starting from the simplest case. We introduce the concepts of: Margin, Noise, and Bias.
Automated machine learning lectures given at the Advanced Course on Data Science & Machine Learning. AutoML, hyperparameter optimization, Bayesian optimization, Neural Architecture Search, Meta-learning, MAML
As the complexity of choosing optimised and task specific steps and ML models is often beyond non-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML.
Although it focuses on end users without expert knowledge, AutoML also offers new tools to machine learning experts, for example to:
1. Perform architecture search over deep representations
2. Analyse the importance of hyperparameters.
« Le « Machine Learning » – « Apprentissage statistique » ou « Analyse prédictive » - sort des labos de recherche et des cercles de spécialistes pour être de plus en plus être utilisé au sein des entreprises, et pas seulement les startups. En témoigne l’essor de la toolkit OpenSource Scikit-learn très vite répandue internationalement comme l’un des nouveaux standards de cette nouvelle façon de faire du logiciel, mais aussi la disponibilité depuis juillet 2014 d’Azure ML, le service de Machine Learning de Microsoft Azure. Dans cette session nous vous proposons un aperçu du développement de logiciel d’apprentissage statistique en Python avec SciKit-Learn. Nous invitons l'un des principaux contributeurs de cette toolkit, Olivier Grisel , ingénieur de recherche dans l’équipe équipe Inria PARIETAL à Saclay, à venir nous en présenter un aperçu dans une session interactive et basée sur de nombreux exemples et démos. Pour en savoir plus: http://scikit-learn.org https://team.inria.fr/parietal/ https://twitter.com/ogrisel
Lecture 01: Machine Learning for Language Technology - IntroductionMarina Santini
What Is Machine Learning? Machine learning is programming computers to optimize a performance criterion using example data or past experience. We have a model defined up to some parameters, and learning is the execution of a computer program to optimize the parameters of the model using the training data or past experience. The model may be predictive to make predictions in the future, or descriptive to gain knowledge from data, or both. Machine learning uses the theory of statistics in building mathematical models, because the core task is making inference from a sample. (Alpaydin, 2010)
In this lecture, we discuss supervised learning starting from the simplest case. We introduce the concepts of: Margin, Noise, and Bias.
Automated machine learning lectures given at the Advanced Course on Data Science & Machine Learning. AutoML, hyperparameter optimization, Bayesian optimization, Neural Architecture Search, Meta-learning, MAML
As the complexity of choosing optimised and task specific steps and ML models is often beyond non-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML.
Although it focuses on end users without expert knowledge, AutoML also offers new tools to machine learning experts, for example to:
1. Perform architecture search over deep representations
2. Analyse the importance of hyperparameters.
« Le « Machine Learning » – « Apprentissage statistique » ou « Analyse prédictive » - sort des labos de recherche et des cercles de spécialistes pour être de plus en plus être utilisé au sein des entreprises, et pas seulement les startups. En témoigne l’essor de la toolkit OpenSource Scikit-learn très vite répandue internationalement comme l’un des nouveaux standards de cette nouvelle façon de faire du logiciel, mais aussi la disponibilité depuis juillet 2014 d’Azure ML, le service de Machine Learning de Microsoft Azure. Dans cette session nous vous proposons un aperçu du développement de logiciel d’apprentissage statistique en Python avec SciKit-Learn. Nous invitons l'un des principaux contributeurs de cette toolkit, Olivier Grisel , ingénieur de recherche dans l’équipe équipe Inria PARIETAL à Saclay, à venir nous en présenter un aperçu dans une session interactive et basée sur de nombreux exemples et démos. Pour en savoir plus: http://scikit-learn.org https://team.inria.fr/parietal/ https://twitter.com/ogrisel
As proposed by the Paper, High-Resolution Image Synthesis with Latent Diffusion Models, latent diffusion models are a simple and efficient way that improve both the training and sampling efficiency of denoising diffusion models while retaining their quality
Poster presented at the Thirty-Second International Joint Conference on Artificial Intelligence, 2023, Macao, SAR. https://doi.org/10.24963/ijcai.2023/554
Data Scientists and Machine Learning practitioners, nowadays, seem to be churning out models by the dozen and they continuously experiment to find ways to improve their accuracies. They also use a variety of ML and DL frameworks & languages , and a typical organization may find that this results in a heterogenous, complicated bunch of assets that require different types of runtimes, resources and sometimes even specialized compute to operate efficiently.
But what does it mean for an enterprise to actually take these models to "production" ? How does an organization scale inference engines out & make them available for real-time applications without significant latencies ? There needs to be different techniques for batch (offline) inferences and instant, online scoring. Data needs to be accessed from various sources and cleansing, transformations of data needs to be enabled prior to any predictions. In many cases, there maybe no substitute for customized data handling with scripting either.
Enterprises also require additional auditing and authorizations built in, approval processes and still support a "continuous delivery" paradigm whereby a data scientist can enable insights faster. Not all models are created equal, nor are consumers of a model - so enterprises require both metering and allocation of compute resources for SLAs.
In this session, we will take a look at how machine learning is operationalized in IBM Data Science Experience (DSX), a Kubernetes based offering for the Private Cloud and optimized for the HortonWorks Hadoop Data Platform. DSX essentially brings in typical software engineering development practices to Data Science, organizing the dev->test->production for machine learning assets in much the same way as typical software deployments. We will also see what it means to deploy, monitor accuracies and even rollback models & custom scorers as well as how API based techniques enable consuming business processes and applications to remain relatively stable amidst all the chaos.
Speaker
Piotr Mierzejewski, Program Director Development IBM DSX Local, IBM
This is the slide from my talk at FULokoja Ingressive meetup.
XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. In prediction problems involving unstructured and structured data (images, text, etc.) artificial neural networks tend to outperform all other algorithms or frameworks. However, when it comes to small-to-medium structured/tabular data, decision tree-based algorithms are considered best-in-class right now. XGBoost model has the best combination of prediction performance and processing time compared to other algorithms.
Knowledge distillation aims at transferring “knowledge” acquired in one model (teacher) to another model (student) that is typically smaller.
Previous approaches can be expressed as a form of training the student with output activations of data examples represented by the teacher.
We introduce a novel approach, dubbed relational knowledge distillation (Relational KD), that transfers relations among data examples represented by the teacher.
As concrete realizations of Relational KD, we propose distance-wise and angle-wise distillation losses that penalize structural differences in relations.
Experiments conducted on different benchmark tasks show that the Relational KD improves the performance of the educated student networks with a significant margin, and even outperforms the teacher’s performance.
An Introduction to TensorFlow architectureMani Goswami
Introduces you to the internals of TensorFlow and deep dives into distributed version of TensorFlow. Refer to https://github.com/manigoswami/tensorflow-examples for examples.
Part 2 of the Deep Learning Fundamentals Series, this session discusses Tuning Training (including hyperparameters, overfitting/underfitting), Training Algorithms (including different learning rates, backpropagation), Optimization (including stochastic gradient descent, momentum, Nesterov Accelerated Gradient, RMSprop, Adaptive algorithms - Adam, Adadelta, etc.), and a primer on Convolutional Neural Networks. The demos included in these slides are running on Keras with TensorFlow backend on Databricks.
A presentation about the development of the ideas from the autoencoder to the Stable Diffusion text-to-image model.
Models covered: autoencoder, VAE, VQ-VAE, VQ-GAN, latent diffusion, and stable diffusion.
Distance-based bias in model-directed optimization of additively decomposable...Martin Pelikan
For many optimization problems it is possible to define a distance metric between problem variables that correlates with the likelihood and strength of interactions between the variables. For example, one may define a metric so that the dependencies between variables that are closer to each other with respect to the metric are expected to be stronger than the dependencies between variables that are further apart. The purpose of this paper is to describe a method that combines such a problem-specific distance metric with information mined from probabilistic models obtained in previous runs of estimation of distribution algorithms with the goal of solving future problem instances of similar type with increased speed, accuracy and reliability. While the focus of the paper is on additively decomposable problems and the hierarchical Bayesian optimization algorithm, it should be straightforward to generalize the approach to other model-directed optimization techniques and other problem classes. Compared to other techniques for learning from experience put forward in the past, the proposed technique is both more practical and more broadly applicable.
The causal discovery of Bayesian networks with the presence of latent variables is a popular topic in artificial intelligence, as sources and volumes of data continue to grow with the popularity of Bayesian modelling methods. Causal discovery is based upon searching the space of causal models for those which can best explain a pattern of probabilistic dependencies shown in the data. Frequently, however, some of those dependencies are generated by causal structures involving variables which have not been measured, i.e., latent variables. Some such patterns of dependency “reveal" themselves, in that no model based solely upon the observed variables can explain them as well as a model using a latent variable. Here we present an algorithm for finding such patterns systematically, so that they may be applied in latent variable discovery in a more rigorous fashion.
Xuhui Zhang, Kevin Korb, Ann Nicholson and Steven Mascaro
Seventh Annual Conference of the Australasian Bayesian Network Modelling Society conference (ABNMS2015) - Monash Caulfield, Melbourne, Australia 2015: http://abnms.org/conferences/abnms2015/
As proposed by the Paper, High-Resolution Image Synthesis with Latent Diffusion Models, latent diffusion models are a simple and efficient way that improve both the training and sampling efficiency of denoising diffusion models while retaining their quality
Poster presented at the Thirty-Second International Joint Conference on Artificial Intelligence, 2023, Macao, SAR. https://doi.org/10.24963/ijcai.2023/554
Data Scientists and Machine Learning practitioners, nowadays, seem to be churning out models by the dozen and they continuously experiment to find ways to improve their accuracies. They also use a variety of ML and DL frameworks & languages , and a typical organization may find that this results in a heterogenous, complicated bunch of assets that require different types of runtimes, resources and sometimes even specialized compute to operate efficiently.
But what does it mean for an enterprise to actually take these models to "production" ? How does an organization scale inference engines out & make them available for real-time applications without significant latencies ? There needs to be different techniques for batch (offline) inferences and instant, online scoring. Data needs to be accessed from various sources and cleansing, transformations of data needs to be enabled prior to any predictions. In many cases, there maybe no substitute for customized data handling with scripting either.
Enterprises also require additional auditing and authorizations built in, approval processes and still support a "continuous delivery" paradigm whereby a data scientist can enable insights faster. Not all models are created equal, nor are consumers of a model - so enterprises require both metering and allocation of compute resources for SLAs.
In this session, we will take a look at how machine learning is operationalized in IBM Data Science Experience (DSX), a Kubernetes based offering for the Private Cloud and optimized for the HortonWorks Hadoop Data Platform. DSX essentially brings in typical software engineering development practices to Data Science, organizing the dev->test->production for machine learning assets in much the same way as typical software deployments. We will also see what it means to deploy, monitor accuracies and even rollback models & custom scorers as well as how API based techniques enable consuming business processes and applications to remain relatively stable amidst all the chaos.
Speaker
Piotr Mierzejewski, Program Director Development IBM DSX Local, IBM
This is the slide from my talk at FULokoja Ingressive meetup.
XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. In prediction problems involving unstructured and structured data (images, text, etc.) artificial neural networks tend to outperform all other algorithms or frameworks. However, when it comes to small-to-medium structured/tabular data, decision tree-based algorithms are considered best-in-class right now. XGBoost model has the best combination of prediction performance and processing time compared to other algorithms.
Knowledge distillation aims at transferring “knowledge” acquired in one model (teacher) to another model (student) that is typically smaller.
Previous approaches can be expressed as a form of training the student with output activations of data examples represented by the teacher.
We introduce a novel approach, dubbed relational knowledge distillation (Relational KD), that transfers relations among data examples represented by the teacher.
As concrete realizations of Relational KD, we propose distance-wise and angle-wise distillation losses that penalize structural differences in relations.
Experiments conducted on different benchmark tasks show that the Relational KD improves the performance of the educated student networks with a significant margin, and even outperforms the teacher’s performance.
An Introduction to TensorFlow architectureMani Goswami
Introduces you to the internals of TensorFlow and deep dives into distributed version of TensorFlow. Refer to https://github.com/manigoswami/tensorflow-examples for examples.
Part 2 of the Deep Learning Fundamentals Series, this session discusses Tuning Training (including hyperparameters, overfitting/underfitting), Training Algorithms (including different learning rates, backpropagation), Optimization (including stochastic gradient descent, momentum, Nesterov Accelerated Gradient, RMSprop, Adaptive algorithms - Adam, Adadelta, etc.), and a primer on Convolutional Neural Networks. The demos included in these slides are running on Keras with TensorFlow backend on Databricks.
A presentation about the development of the ideas from the autoencoder to the Stable Diffusion text-to-image model.
Models covered: autoencoder, VAE, VQ-VAE, VQ-GAN, latent diffusion, and stable diffusion.
Distance-based bias in model-directed optimization of additively decomposable...Martin Pelikan
For many optimization problems it is possible to define a distance metric between problem variables that correlates with the likelihood and strength of interactions between the variables. For example, one may define a metric so that the dependencies between variables that are closer to each other with respect to the metric are expected to be stronger than the dependencies between variables that are further apart. The purpose of this paper is to describe a method that combines such a problem-specific distance metric with information mined from probabilistic models obtained in previous runs of estimation of distribution algorithms with the goal of solving future problem instances of similar type with increased speed, accuracy and reliability. While the focus of the paper is on additively decomposable problems and the hierarchical Bayesian optimization algorithm, it should be straightforward to generalize the approach to other model-directed optimization techniques and other problem classes. Compared to other techniques for learning from experience put forward in the past, the proposed technique is both more practical and more broadly applicable.
The causal discovery of Bayesian networks with the presence of latent variables is a popular topic in artificial intelligence, as sources and volumes of data continue to grow with the popularity of Bayesian modelling methods. Causal discovery is based upon searching the space of causal models for those which can best explain a pattern of probabilistic dependencies shown in the data. Frequently, however, some of those dependencies are generated by causal structures involving variables which have not been measured, i.e., latent variables. Some such patterns of dependency “reveal" themselves, in that no model based solely upon the observed variables can explain them as well as a model using a latent variable. Here we present an algorithm for finding such patterns systematically, so that they may be applied in latent variable discovery in a more rigorous fashion.
Xuhui Zhang, Kevin Korb, Ann Nicholson and Steven Mascaro
Seventh Annual Conference of the Australasian Bayesian Network Modelling Society conference (ABNMS2015) - Monash Caulfield, Melbourne, Australia 2015: http://abnms.org/conferences/abnms2015/
Deep dive into the mathematics and algorithms of neural nets. Covers the sigmoid activation function, cross-entropy loss function, gradient descent and the derivatives used in back propagation.
Understanding Protein Function on a Genome-scale through the Analysis of Molecular Networks
Cornell Medical School, Physiology, Biophysics and Systems Biology (PBSB) graduate program, 2009.01.26, 16:00-17:00; [I:CORNELL-PBSB] (Long networks talk, incl. the following topics: why networks w. amsci*, funnygene*, net. prediction intro, memint*, tse*, essen*, sandy*, metagenomics*, netpossel*, tyna*+ topnet*, & pubnet* . Fits easily into 60’ w. 10’ questions. PPT works on mac & PC and has many photos w. EXIF tag kwcornellpbsb .)
Date Given: 01/26/2009
Are you sure about that?! Uncertainty Quantification in AIinovex GmbH
With the advent of Deep Learning (DL), the field of AI made a giant leap forward and it is nowadays applied in many industrial use-cases. Especially critical systems like autonomous driving, require that DL methods not only produce a prediction but also state the certainty about the prediction in order to assess risks and failure.
In my talk, I will give an introduction to different kinds of uncertainty, i.e. epistemic and aleatoric. To have a baseline for comparison, the classical method of Gaussian Processes for regression problems is presented. I then elaborate on different DL methods for uncertainty quantification like Quantile Regression, Monte-Carlo Dropout, and Deep Ensembles. The talk is concluded with a comparison of these techniques to Gaussian Processes and the current state of the art.
Speaker: Dr. Florian Wilhelm, Simon Bachstein, inovex
Event: PyCon/PyData Berlin 2019
Datum: 10.10.2019
Mehr Tech-Vorträge: inovex.de/vortraege
Mehr Tech-Artikel: inovex.de/blog
With the advent of Deep Learning (DL), the field of AI made a giant leap forward and it is nowadays applied in many industrial use-cases. Especially critical systems like autonomous driving, require that DL methods not only produce a prediction but also state the certainty about the prediction in order to assess risks and failure.
In my talk, I will give an introduction to different kinds of uncertainty, i.e. epistemic and aleatoric. To have a baseline for comparison, the classical method of Gaussian Processes for regression problems is presented. I then elaborate on different DL methods for uncertainty quantification like Quantile Regression, Monte-Carlo Dropout, and Deep Ensembles. The talk is concluded with a comparison of these techniques to Gaussian Processes and the current state of the art.
Michal Mucha: Build and Deploy an End-to-end Streaming NLP Insight System | P...PyData
At this workshop, you will build your own messaging insights system - data ingestion from a live data source (Reddit), queueing, deploying a machine learning model, and serving messages with insights to your mobile phone!
Unit testing data with marbles - Jane Stewart Adams, Leif WalshPyData
In the same way that we need to make assertions about how code functions, we need to make assertions about data, and unit testing is a promising framework. In this talk, we'll explore what is unique about unit testing data, and see how Two Sigma's open source library Marbles addresses these unique challenges in several real-world scenarios.
The TileDB Array Data Storage Manager - Stavros Papadopoulos, Jake BolewskiPyData
TileDB is an open-source storage manager for multi-dimensional sparse and dense array data. It has a novel architecture that addresses some of the pain points in storing array data on “big-data” and “cloud” storage architectures. This talk will highlight TileDB’s design and its ability to integrate with analysis environments relevant to the PyData community such as Python, R, Julia, etc.
Using Embeddings to Understand the Variance and Evolution of Data Science... ...PyData
In this talk I will discuss exponential family embeddings, which are methods that extend the idea behind word embeddings to other data types. I will describe how we used dynamic embeddings to understand how data science skill-sets have transformed over the last 3 years using our large corpus of jobs. The key takeaway is that these models can enrich analysis of specialized datasets.
Deploying Data Science for Distribution of The New York Times - Anne BauerPyData
How many newspapers should be distributed to each store for sale every day? The data science group at The New York Times addresses this optimization problem using custom time series modeling and analytical solutions, while also incorporating qualitative business concerns. I'll describe our modeling and data engineering approaches, written in Python and hosted on Google Cloud Platform.
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaPyData
However, the the graph theory jargon can make graph analytics seem more intimidating for self-study than is necessary. In this talk, the audience will be exposed to some of the basic concepts of graph theory (no prerequisite math knowledge needed!) and a few of the Python tools available for graph analysis.
Do Your Homework! Writing tests for Data Science and Stochastic Code - David ...PyData
To productionize data science work (and have it taken seriously by software engineers, CTOs, clients, or the open source community), you need to write tests! Except… how can you test code that performs nondeterministic tasks like natural language parsing and modeling? This talk presents an approach to testing probabilistic functions in code, illustrated with concrete examples written for Pytest.
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData
Those of us who use TensorFlow often focus on building the model that's most predictive, not the one that's most deployable. So how to put that hard work to work? In this talk, we'll walk through a strategy for taking your machine learning models from Jupyter Notebook into production and beyond.
Mining dockless bikeshare and dockless scootershare trip data - Stefanie Brod...PyData
In September 2017, dockless bikeshare joined the transportation options in the District of Columbia. In March 2018, scooter share followed. During the pilot of these technologies, Python has helped District Department of Transportation answer some critical questions. This talk will discuss how Python was used to answer research questions and how it supported the evaluation of this demonstration.
Avoiding Bad Database Surprises: Simulation and Scalability - Steven LottPyData
There are many stories of developers creating databases that don't operate at scale. The application is good, but the database won't work the realistic volumes of data. It's like a horror movie where they never looked behind the door, ran into the dark forest and night, and discovered the database was the monster killing their application. How can we leverage Python to avoid scaling problems?
Machine learning often requires us to think spatially and make choices about what it means for two instances to be close or far apart. So which is best - Euclidean? Manhattan? Cosine? It all depends! In this talk, we'll explore open source tools and visual diagnostic strategies for picking good distance metrics when doing machine learning on text.
End-to-End Machine learning pipelines for Python driven organizations - Nick ...PyData
The recent advances in machine learning and artificial intelligence are amazing! Yet, in order to have real value within a company, data scientists must be able to get their models off of their laptops and deployed within a company’s data pipelines and infrastructure. In this session, I'll demonstrate how one-off experiments can be transformed into scalable ML pipelines with minimal effort.
We will be using Beautiful Soup to Webscrape the IMDB website and create a function that will allow you to create a dictionary object on specific metadata of the IMDB profile for any IMDB ID you pass through as an argument.
1D Convolutional Neural Networks for Time Series Modeling - Nathan Janos, Jef...PyData
This talk describes an experimental approach to time series modeling using 1D convolution filter layers in a neural network architecture. This approach was developed at System1 for forecasting marketplace value of online advertising categories.
Extending Pandas with Custom Types - Will AydPyData
Pandas v.0.23 brought to life a new extension interface through which you can extend NumPy's type system. This talk will explain what that means in more detail and provide practical examples of how the new interface can be leveraged to drastically improve your reporting.
Machine learning models are increasingly used to make decisions that affect people’s lives. With this power comes a responsibility to ensure that model predictions are fair. In this talk I’ll introduce several common model fairness metrics, discuss their tradeoffs, and finally demonstrate their use with a case study analyzing anonymized data from one of Civis Analytics’s client engagements.
What's the Science in Data Science? - Skipper SeaboldPyData
The gold standard for validating any scientific assumption is to run an experiment. Data science isn’t any different. Unfortunately, it’s not always possible to design the perfect experiment. In this talk, we’ll take a realistic look at measurement using tools from the social sciences to conduct quasi-experiments with observational data.
Applying Statistical Modeling and Machine Learning to Perform Time-Series For...PyData
Forecasting time-series data has applications in many fields, including finance, health, etc. There are potential pitfalls when applying classic statistical and machine learning methods to time-series problems. This talk will give folks the basic toolbox to analyze time-series data and perform forecasting using statistical and machine learning models, as well as interpret and convey the outputs.
Solving very simple substitution ciphers algorithmically - Stephen Enright-WardPyData
A historical text may now be unreadable, because its language is unknown, or its script forgotten (or both), or because it was deliberately enciphered. Deciphering needs two steps: Identify the language, then map the unknown script to a familiar one. I’ll present an algorithm to solve a cartoon version of this problem, where the language is known, and the cipher is alphabet rearrangement.
The Face of Nanomaterials: Insightful Classification Using Deep Learning - An...PyData
Artificial intelligence is emerging as a new paradigm in materials science. This talk describes how physical intuition and (insightful) machine learning can solve the complicated task of structure recognition in materials at the nanoscale.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Maxwell W Libbrecht - pomegranate: fast and flexible probabilistic modeling in python
1. fast and flexible probabilistic modeling in python
jmschreiber91
@jmschrei
@jmschreiber91
Jacob Schreiber
PhD student, Paul G. Allen School of
Computer Science, University of
Washington, Seattle WA
Maxwell Libbrecht
Postdoc, Department of Genome Sciences,
University of Washington, Seattle WA
Assistant Professor, Simon Fraser University,
Vancouver BC
noble.gs.washington.edu/~maxwl/
2. fast and flexible probabilistic modeling in python
jmschreiber91
@jmschrei
@jmschreiber91
Jacob Schreiber
PhD student, Paul G. Allen School of
Computer Science, University of
Washington, Seattle WA
Maxwell Libbrecht
Postdoc, Department of Genome Sciences,
University of Washington, Seattle WA
Assistant Professor, Simon Fraser University,
Vancouver BC
noble.gs.washington.edu/~maxwl/
4. What is probabilistic modeling?
4goo.gl/cbE7rs
Tasks:
• Sample from a distribution.
• Learn parameters of a
distribution from data.
• Predict which distribution
generated a data example.
5. When should you use pomegranate?
5
complexity of probabilistic modeling task
simple complexz }| {
• logistic regression
• sample from a Gaussian
distribution
• speed is not important
Use scipy or
scikit-learn
z }| {
• distributions not implemented in
pomegranate
• problem-specific inference
algorithms
Use custom
packages (in R or
stand-alone)
z }| {
Use
pomegranate
goo.gl/cbE7rs
6. When should you use pomegranate?
6
z }| {
Use pomegranate
probabilistic modeling tasks
orthogonaltasks
• Scientific computing: Use numpy, scipy
• Non-probabilistic supervised learning (SVMs,
neural networks): Use scikit-learn
• Visualizing probabilistic models: Use matplotlib
• Bayesian statistics: Check out PyMC3
goo.gl/cbE7rs
7. Overview: this talk
7
Overview
Major Models/Model Stacks
1. General Mixture Models
2. Hidden Markov Models
3. Bayesian Networks
4. Bayes Classifiers
Finale: Train a mixture of HMMs in parallel
8. The API is common to all models
8
All models have these methods!
All models composed of
distributions (like GMM, HMM...)
have these methods too!
model.log_probability(X) / model.probability(X)
model.sample()
model.fit(X, weights, inertia)
model.summarize(X, weights)
model.from_summaries(inertia)
model.predict(X)
model.predict_proba(X)
model.predict_log_proba(X)
Model.from_samples(X, weights)
goo.gl/cbE7rs
9. Overview: supported models
Six Main Models:
1. Probability Distributions
2. General Mixture Models
3. Markov Chains
4. Hidden Markov Models
5. Bayes Classifiers / Naive Bayes
6. Bayesian Networks
9
Two Helper Models:
1. k-means++/kmeans||
2. Factor Graphs
12. 12
Models can be learned from data
X = numpy.random.normal(0, 1, 100)
a = NormalDistribution.from_samples(X)
13. Overview: model stacking in pomegranate
13
Distributions
Bayes Classifiers
Markov Chains
General Mixture Models
Hidden Markov Models
Bayesian Networks
D BC MC GMM HMM BN
Sub-model
Wrapper model
14. Overview: model stacking in pomegranate
14
Distributions
Bayes Classifiers
Markov Chains
General Mixture Models
Hidden Markov Models
Bayesian Networks
D BC MC GMM HMM BN
Sub-model
Wrapper model
15. Overview: model stacking in pomegranate
15
Distributions
Bayes Classifiers
Markov Chains
General Mixture Models
Hidden Markov Models
Bayesian Networks
D BC MC GMM HMM BN
Sub-model
Wrapper model
16. Overview: model stacking in pomegranate
16
Distributions
Bayes Classifiers
Markov Chains
General Mixture Models
Hidden Markov Models
Bayesian Networks
D BC MC GMM HMM BN
17. 17
pomegranate can be faster than numpy
Fitting a Normal Distribution to 1,000 samples
18. 18
pomegranate can be faster than numpy
Fitting Multivariate Gaussian to 10,000,000 samples of 10
dimensions
21. 21
pomegranate uses additive summarization
pomegranate reduces data to sufficient statistics for updates
and so only has to go datasets once (for all models).
Here is an example of the Normal Distribution sufficient
statistics
22. 22
pomegranate supports out-of-core learning
Batches from a dataset can be reduced to additive summary
statistics, enabling exact updates from data that can’t fit in memory.
24. 24
pomegranate supports semisupervised learning
Summary statistics from supervised models can be added to
summary statistics from unsupervised models to train a single model
on a mixture of labeled and unlabeled data.
30. 30
Example ‘blast’ from Gossip Girl
Spotted: Lonely Boy. Can't believe the love of his life has
returned. If only she knew who he was. But everyone knows
Serena. And everyone is talking. Wonder what Blair Waldorf
thinks. Sure, they're BFF's, but we always thought Blair's
boyfriend Nate had a thing for Serena.
31. 31
How do we encode these ‘blasts’?
Better lock it down with Nate, B. Clock's ticking.
+1 Nate
-1 Blair
32. 32
How do we encode these ‘blasts’?
This just in: S and B committing a crime of fashion. Who
doesn't love a five-finger discount. Especially if it's the middle
one.
-1 Blair
-1 Serena
54. Bayesian network inference
54
Inference is done using Belief Bropagation on a factor graph.
Messages are sent from one side to the other until convergence.
Rain
Sprinkler
Grass
Wet
R/S/GW
Factor
Grass
Wet
Rain
Sprinkler
Marginals Factors
55. Inference in a medical diagnostic network
55
BRCA 2 BRCA 1 LCT
BLOATLE LOA VOM AC
PREGLIOC
56. Inference in a medical diagnostic network
56
BRCA 2 BRCA 1 LCT
BLOATLE LOA VOM AC
PREGLIOC
57. Bayesian network structure learning
57
???
Three primary ways:
● “Search and score” / Exact
● “Constraint Learning” / PC
● Heuristics
82. Overview: this talk
82
Overview
Major Models/Model Stacks
1. General Mixture Models
2. Hidden Markov Models
3. Bayesian Networks
4. Bayes Classifiers
Finale: Train a mixture of HMMs in parallel
83. Training a mixture of HMMs in parallel
83
Creating a mixture of HMMs is just as simple as passing the
HMMs into a GMM as if it were any other distribution
88. fast and flexible probabilistic modeling in python
jmschreiber91
@jmschrei
@jmschreiber91
Jacob Schreiber
PhD student, Paul G. Allen School of
Computer Science, University of
Washington, Seattle WA
Maxwell Libbrecht
Postdoc, Department of Genome Sciences,
University of Washington, Seattle WA
Assistant Professor, Simon Fraser University,
Vancouver BC
noble.gs.washington.edu/~maxwl/