This document discusses Bayesian optimization for hyperparameter tuning. It begins by introducing the modelling workflow of a data scientist and focusing on the model selection step. It then explains that most models have hyperparameters that need to be chosen and the goal is to find hyperparameters that optimize model performance on holdout data. Bayesian optimization is introduced as a method for hyperparameter tuning that builds a probabilistic model of performance using Gaussian processes. It iteratively evaluates hyperparameters chosen by an acquisition function that balances exploitation and exploration. An example of using Bayesian optimization to tune an SVM's hyperparameters is provided, along with practical considerations and open-source tools.
Achieving Algorithmic Transparency with Shapley Additive Explanations (H2O Lo...Sri Ambati
Abstract:
Explainability in the age of the EU GDPR is becoming an increasingly pertinent consideration for Machine Learning. At QuantumBlack, we address the traditional Accuracy vs. Interpretability trade-off, by leveraging modern XAI techniques such as LIME and SHAP, to enable individualised explanations without necessary limiting the utility and performance of the otherwise ‘black-box’ models. The talk focuses on Shapley additive explanations (Lundberg et al. 2017) that integrate Shapley values from the Game Theory for consistent and locally accurate explanations; provides illustrative examples and touches upon the wider XAI theory.
Bio:
Dr Torgyn Shaikhina is a Data Scientist at QuantumBlack, STEM Ambassador, and the founder of the Next Generation Programmers outreach initiative. Her background is in decision support systems for Healthcare and Biomedical Engineering with a focus on Machine Learning with limited information.
In this presentation I review various data science techniques and discuss their usefulness to pricing actuaries working in general insurance.
This presentation was originally given at the TIGI webinar in 2020.
https://www.actuaries.org.uk/learn-develop/attend-event/tigi-2020-technical-issues-general-insurance
This deck quickly walks through fundamentals of Deep Learning and describes how symbolic engine of MXNet implements such networks. It then introduces gluon and provides code examples. The last section of the presentation introduces latest developments in gluon family of tools to include GluonNLP, an NLP toolkit with SOTA implementation of NLP algorithms, GluonCV, a Computer Vision toolkit with SOTA implementation of Vision algorithms, and MXNet backend for Keras.
This presentation describes two major papers in multi-variate time-series using deep neural networks. The first paper, DeepAR was developed at Amazon to deal with forecasting of millions of items where the same model can be applied to millions of products. DeepAR is implemented as a built-in algorithm of Amazon SageMaker. Code example is provided.
The second paper, Long- and Short-Term Temporal Patterns with Deep Neural Networks is developed at CMU and introduces a novel way to detect both short term and long term seasonality in data through introduction of skip-rnn.
A Gluon implementation of the paper is provided in the presentation.
A HYBRID K-HARMONIC MEANS WITH ABCCLUSTERING ALGORITHM USING AN OPTIMAL K VAL...IJCI JOURNAL
Large quantities of data are emerging every year and an accurate clustering algorithm is needed to derive
information from these data. K-means clustering algorithm is popular and simple, but has many limitations
like its sensitivity to initialization, provides local optimum solutions. K-harmonic means clustering is an
improved variant of K-means which is insensitive to the initialization of centroids, but still in some cases it
ends up with local optimum solutions. Clustering using Artificial Bee Colony (ABC) algorithm always gives
global optimum solutions. In this paper a new hybrid clustering algorithm (KHM-ABC) is presented by
combining both K-harmonic means and ABC algorithm to perform accurate clustering. Experimental
results indicate that the performance of the proposed algorithm is superior to the available algorithms in
terms of the quality of clusters.
In this deck I’m going to show you how SigOpt can help you amplify your trading models by optimally tuning them using our black-box optimization platform.
Achieving Algorithmic Transparency with Shapley Additive Explanations (H2O Lo...Sri Ambati
Abstract:
Explainability in the age of the EU GDPR is becoming an increasingly pertinent consideration for Machine Learning. At QuantumBlack, we address the traditional Accuracy vs. Interpretability trade-off, by leveraging modern XAI techniques such as LIME and SHAP, to enable individualised explanations without necessary limiting the utility and performance of the otherwise ‘black-box’ models. The talk focuses on Shapley additive explanations (Lundberg et al. 2017) that integrate Shapley values from the Game Theory for consistent and locally accurate explanations; provides illustrative examples and touches upon the wider XAI theory.
Bio:
Dr Torgyn Shaikhina is a Data Scientist at QuantumBlack, STEM Ambassador, and the founder of the Next Generation Programmers outreach initiative. Her background is in decision support systems for Healthcare and Biomedical Engineering with a focus on Machine Learning with limited information.
In this presentation I review various data science techniques and discuss their usefulness to pricing actuaries working in general insurance.
This presentation was originally given at the TIGI webinar in 2020.
https://www.actuaries.org.uk/learn-develop/attend-event/tigi-2020-technical-issues-general-insurance
This deck quickly walks through fundamentals of Deep Learning and describes how symbolic engine of MXNet implements such networks. It then introduces gluon and provides code examples. The last section of the presentation introduces latest developments in gluon family of tools to include GluonNLP, an NLP toolkit with SOTA implementation of NLP algorithms, GluonCV, a Computer Vision toolkit with SOTA implementation of Vision algorithms, and MXNet backend for Keras.
This presentation describes two major papers in multi-variate time-series using deep neural networks. The first paper, DeepAR was developed at Amazon to deal with forecasting of millions of items where the same model can be applied to millions of products. DeepAR is implemented as a built-in algorithm of Amazon SageMaker. Code example is provided.
The second paper, Long- and Short-Term Temporal Patterns with Deep Neural Networks is developed at CMU and introduces a novel way to detect both short term and long term seasonality in data through introduction of skip-rnn.
A Gluon implementation of the paper is provided in the presentation.
A HYBRID K-HARMONIC MEANS WITH ABCCLUSTERING ALGORITHM USING AN OPTIMAL K VAL...IJCI JOURNAL
Large quantities of data are emerging every year and an accurate clustering algorithm is needed to derive
information from these data. K-means clustering algorithm is popular and simple, but has many limitations
like its sensitivity to initialization, provides local optimum solutions. K-harmonic means clustering is an
improved variant of K-means which is insensitive to the initialization of centroids, but still in some cases it
ends up with local optimum solutions. Clustering using Artificial Bee Colony (ABC) algorithm always gives
global optimum solutions. In this paper a new hybrid clustering algorithm (KHM-ABC) is presented by
combining both K-harmonic means and ABC algorithm to perform accurate clustering. Experimental
results indicate that the performance of the proposed algorithm is superior to the available algorithms in
terms of the quality of clusters.
In this deck I’m going to show you how SigOpt can help you amplify your trading models by optimally tuning them using our black-box optimization platform.
Common Problems in Hyperparameter OptimizationSigOpt
Originally given at MLConf NYC 2017.
All large machine learning pipelines have tunable parameters, commonly referred to as hyperparameters. Hyperparameter optimization is the process by which we find the values for these parameters that cause our system to perform the best. SigOpt provides a Bayesian optimization platform that is commonly used for hyperparameter optimization, and I’m going to share some of the common problems we’ve seen when integrating into machine learning pipelines.
AWS makes it easy to build, train, tune, and deploy Machine Learning (ML) models. If you're excited to get started with ML on AWS but want a refresher on the ML concepts behind build, train, tune, and deploy, this Dev Chat is for you.
Originally delivered as a Dev Chat at AWS Summit SF by Software Engineer Alexandra Johnson
Advanced Optimization for the Enterprise WebinarSigOpt
Building on the TWIML eBook, TWIMLcon event and TWIML podcast series that explore Machine Learning Platforms in great detail, this webinar examines the machine learning platforms that power enterprise leaders in AI. SigOpt CEO Scott Clark will provide an overview of critical technical capabilities that our customers have prioritized in their ML platforms.
Review these slides to learn about:
- Critical capabilities for data, experiment and model management
- Tradeoffs between building and buying these capabilities
- Lessons from the implementation of these platforms by AI leaders
Why focus on these platforms and the capabilities that power them? Nearly every company is investing in machine learning that differentiates products or generates revenue. These so-called "differentiated models" represent the biggest opportunity for AI to transform the business. Most of these teams find success hiring expert data scientists and machine learning engineers who can build these models. But most of these teams also struggle to create a more sustainable, scalable and reproducible process for model development, and have begun building ML platforms to tackle this challenge.
A review of the paper “Ad Click Prediction: a View from the Trenches”
The paper discusses predicting ad click--through rates (CTR) which is a massive-scale learning problem central to the multi-billion dollar online advertising industry.
Presented by Mazen & Arzam in the Data Intensive Computing class at KTH, Stockholm, Sweden.
Link of the paper: http://research.google.com/pubs/pub41159.html
Plotcon 2016 Visualization Talk by Alexandra JohnsonSigOpt
Machine learning is full of ideas that are far abstracted away from the underlying data and difficult to understand. Luckily, this represents an amazing opportunity for visualization! These slides dive into the machine learning meta-problem of hyperparameter optimization. We'll show 4 opportunities for visualization in helping people understand, implement, and evaluate hyperparameter optimization strategies.
SigOpt CEO Scott Clark provides insights for modeling at scale in systematic trading. SigOpt works with algorithmic trading firms that collectively represent $300 billion in assets under management (AUM). In this presentation, Scott draws on this experience to provide a few critical insights to how these companies effectively model at scale. Alongside these insights, Scott shares a more specific case study from working with Two Sigma, a leading systematic investment manager.
In this presentation at R user group, I share about the various advance techniques I used for Kaggle competitions. Includes: Interactive visualization via leaflet, geospatial clustering via local Moran's I, feature creation, text categorization via splitTag techniques and ensemble modeling.
Full code can be downloaded here: https://github.com/thiakx/RUGS-Meetup
Train / test data from Kaggle: http://www.kaggle.com/c/see-click-predict-fix/data
Interactive map demo: http://www.thiakx.com/misc/playground/scfMap/scfMap.html
SigOpt at O'Reilly - Best Practices for Scaling Modeling PlatformsSigOpt
Companies are increasingly building modeling platforms to empower their researchers to efficiently scale the development and productionalization of their models. Scott Clark and Matt Greenwood share a case study from a leading algorithmic trading firm to illustrate best practices for building these types of platforms in any industry. Join in to learn how Two Sigma, a leading quantitative investment and technology firm, solved its model optimization problem.
Using SigOpt to Tune Deep Learning Models with Nervana CloudSigOpt
In this talk I'll show how the Bayesian Optimization methods used by SigOpt, coupled with the incredibly scalable deep learning architecture provided with ncloud and neon, allow anyone it easily tune their models to quickly achieve higher accuracy. I'll walk through the techniques and show an explicit example with results.
Used a 40GB dataset made available by Avito via Kaggle to demonstrate how to handle big data for machine learning using limited memory. Instead of taking the incremental learning route to train a classifier, we used an intelligent technique to create a representative sample of the dataset.
Since ad clicks are very rare events, naively sampling the data would have lead to significantly biased predictions. This sampling bias was addressed by assigning an importance weight to each data example selected.
The resulting dataset could easily fit into memory and so was then trained using logistic regression.
Data Trend Analysis by Assigning Polynomial Function For Given Data SetIJCERT
This paper aims at explaining the method of creating a polynomial equation out of the given data set which can be used as a representation of the data itself and can be used to run aggregation against itself to find the results. This approach uses least-squares technique to construct a model of data and fit to a polynomial. Differential calculus technique is used on this equation to generate the aggregated results that represents the original data set.
ESTIMATING PROJECT DEVELOPMENT EFFORT USING CLUSTERED REGRESSION APPROACHcscpconf
Due to the intangible nature of “software”, accurate and reliable software effort estimation is a challenge in the software Industry. It is unlikely to expect very accurate estimates of software
development effort because of the inherent uncertainty in software development projects and the complex and dynamic interaction of factors that impact software development. Heterogeneity exists in the software engineering datasets because data is made available from diverse sources.
This can be reduced by defining certain relationship between the data values by classifying them into different clusters. This study focuses on how the combination of clustering and
regression techniques can reduce the potential problems in effectiveness of predictive efficiency due to heterogeneity of the data. Using a clustered approach creates the subsets of data having a degree of homogeneity that enhances prediction accuracy. It was also observed in this study that ridge regression performs better than other regression techniques used in the analysis.
Common Problems in Hyperparameter OptimizationSigOpt
Originally given at MLConf NYC 2017.
All large machine learning pipelines have tunable parameters, commonly referred to as hyperparameters. Hyperparameter optimization is the process by which we find the values for these parameters that cause our system to perform the best. SigOpt provides a Bayesian optimization platform that is commonly used for hyperparameter optimization, and I’m going to share some of the common problems we’ve seen when integrating into machine learning pipelines.
AWS makes it easy to build, train, tune, and deploy Machine Learning (ML) models. If you're excited to get started with ML on AWS but want a refresher on the ML concepts behind build, train, tune, and deploy, this Dev Chat is for you.
Originally delivered as a Dev Chat at AWS Summit SF by Software Engineer Alexandra Johnson
Advanced Optimization for the Enterprise WebinarSigOpt
Building on the TWIML eBook, TWIMLcon event and TWIML podcast series that explore Machine Learning Platforms in great detail, this webinar examines the machine learning platforms that power enterprise leaders in AI. SigOpt CEO Scott Clark will provide an overview of critical technical capabilities that our customers have prioritized in their ML platforms.
Review these slides to learn about:
- Critical capabilities for data, experiment and model management
- Tradeoffs between building and buying these capabilities
- Lessons from the implementation of these platforms by AI leaders
Why focus on these platforms and the capabilities that power them? Nearly every company is investing in machine learning that differentiates products or generates revenue. These so-called "differentiated models" represent the biggest opportunity for AI to transform the business. Most of these teams find success hiring expert data scientists and machine learning engineers who can build these models. But most of these teams also struggle to create a more sustainable, scalable and reproducible process for model development, and have begun building ML platforms to tackle this challenge.
A review of the paper “Ad Click Prediction: a View from the Trenches”
The paper discusses predicting ad click--through rates (CTR) which is a massive-scale learning problem central to the multi-billion dollar online advertising industry.
Presented by Mazen & Arzam in the Data Intensive Computing class at KTH, Stockholm, Sweden.
Link of the paper: http://research.google.com/pubs/pub41159.html
Plotcon 2016 Visualization Talk by Alexandra JohnsonSigOpt
Machine learning is full of ideas that are far abstracted away from the underlying data and difficult to understand. Luckily, this represents an amazing opportunity for visualization! These slides dive into the machine learning meta-problem of hyperparameter optimization. We'll show 4 opportunities for visualization in helping people understand, implement, and evaluate hyperparameter optimization strategies.
SigOpt CEO Scott Clark provides insights for modeling at scale in systematic trading. SigOpt works with algorithmic trading firms that collectively represent $300 billion in assets under management (AUM). In this presentation, Scott draws on this experience to provide a few critical insights to how these companies effectively model at scale. Alongside these insights, Scott shares a more specific case study from working with Two Sigma, a leading systematic investment manager.
In this presentation at R user group, I share about the various advance techniques I used for Kaggle competitions. Includes: Interactive visualization via leaflet, geospatial clustering via local Moran's I, feature creation, text categorization via splitTag techniques and ensemble modeling.
Full code can be downloaded here: https://github.com/thiakx/RUGS-Meetup
Train / test data from Kaggle: http://www.kaggle.com/c/see-click-predict-fix/data
Interactive map demo: http://www.thiakx.com/misc/playground/scfMap/scfMap.html
SigOpt at O'Reilly - Best Practices for Scaling Modeling PlatformsSigOpt
Companies are increasingly building modeling platforms to empower their researchers to efficiently scale the development and productionalization of their models. Scott Clark and Matt Greenwood share a case study from a leading algorithmic trading firm to illustrate best practices for building these types of platforms in any industry. Join in to learn how Two Sigma, a leading quantitative investment and technology firm, solved its model optimization problem.
Using SigOpt to Tune Deep Learning Models with Nervana CloudSigOpt
In this talk I'll show how the Bayesian Optimization methods used by SigOpt, coupled with the incredibly scalable deep learning architecture provided with ncloud and neon, allow anyone it easily tune their models to quickly achieve higher accuracy. I'll walk through the techniques and show an explicit example with results.
Used a 40GB dataset made available by Avito via Kaggle to demonstrate how to handle big data for machine learning using limited memory. Instead of taking the incremental learning route to train a classifier, we used an intelligent technique to create a representative sample of the dataset.
Since ad clicks are very rare events, naively sampling the data would have lead to significantly biased predictions. This sampling bias was addressed by assigning an importance weight to each data example selected.
The resulting dataset could easily fit into memory and so was then trained using logistic regression.
Data Trend Analysis by Assigning Polynomial Function For Given Data SetIJCERT
This paper aims at explaining the method of creating a polynomial equation out of the given data set which can be used as a representation of the data itself and can be used to run aggregation against itself to find the results. This approach uses least-squares technique to construct a model of data and fit to a polynomial. Differential calculus technique is used on this equation to generate the aggregated results that represents the original data set.
ESTIMATING PROJECT DEVELOPMENT EFFORT USING CLUSTERED REGRESSION APPROACHcscpconf
Due to the intangible nature of “software”, accurate and reliable software effort estimation is a challenge in the software Industry. It is unlikely to expect very accurate estimates of software
development effort because of the inherent uncertainty in software development projects and the complex and dynamic interaction of factors that impact software development. Heterogeneity exists in the software engineering datasets because data is made available from diverse sources.
This can be reduced by defining certain relationship between the data values by classifying them into different clusters. This study focuses on how the combination of clustering and
regression techniques can reduce the potential problems in effectiveness of predictive efficiency due to heterogeneity of the data. Using a clustered approach creates the subsets of data having a degree of homogeneity that enhances prediction accuracy. It was also observed in this study that ridge regression performs better than other regression techniques used in the analysis.
Automatic Model Tuning Using Amazon SageMaker (AIM412) - AWS re:Invent 2018Amazon Web Services
In many cases, what separates good models from great ones is the choice of hyperparameters. For example, what is the number of layers you should use; what should be the learning rate; what should be regularization parameters, and so on. In this session, learn how Amazon SageMaker makes discovering the best set of hyperparameters an informed process during training.
MCL310_Building Deep Learning Applications with Apache MXNet and GluonAmazon Web Services
In this workshop, learn how to set up a deep learning environment and explore neural network architectures, namely multilayer perceptrons, convolution neural networks (CNNs), and LSTMs. You'll learn to model problems and use cases that many customers tackle by using Gluon, the new new intuitive, dynamic programming interface for Apache MXNet. You'll also learn how to model common use cases with deep learning on AWS.
Bayesian Optimization for Balancing Metrics in Recommender SystemsViral Gupta
Most large-scale online recommender systems like newsfeed ranking, people recommendations, job recommendations, etc. often have multiple utilities or metrics that need to be simultaneously optimized. The machine learning models that are trained to optimize a single utility are combined together through parameters to generate the final ranking function. These combination parameters drive business metrics. Finding the right choice of the parameters is often done through online A/B experimentation, which can be incredibly complex and time-consuming, especially considering the non-linear effects of these parameters on the metrics of interest.
In this tutorial, we will talk about how we can apply Bayesian Optimization techniques to obtain the parameters for such complex online systems in order to balance the competing metrics. First, we will provide an in-depth introduction to Bayesian Optimization, covering some of the basics as well as the recent advances in the field. Second, we will talk about how to formulate a real-world recommender system problem as a black-box optimization problem that can be solved via Bayesian Optimization. We will focus on a few key problems such as newsfeed ranking, people recommendations, job recommendations, etc. Third, we will talk about the architecture of the solution and how we are able to deploy it for large-scale systems. Finally, we will discuss the extensions and some of the future directions in this domain.
AWS reInvent 2017 recap - Optimizing Costs as You Scale on AWSAmazon Web Services
The cloud offers a first-in-a-career-opportunity to constantly optimize your costs as you grow and stay on the bleeding edge of innovation. By developing a cost-conscious culture and assigning the responsibility for efficiency to the appropriate business owners, you can deliver innovation efficiently and cost effectively. This session will review a wide range of cost planning, monitoring, and optimization strategies featuring real-world experience from AWS customers.
Spark ML with High Dimensional Labels Michael Zargham and Stefan PanayotovDatabricks
This talk is detailed extension of our Spark Summit East talk on the same topic. We will review the hurdles we faced and the work arounds we developed with the help from Databricks support in our attempt to build a custom machine learning model and use it to predict the TV ratings for different networks and demographics. Attendees should leave this session with enough knowledge to recognize situations where our method would be applicable to implement it.
Specifically, we dig into the details of the data characteristics that make our problem inherently challenging and how we compose existing tools in the ML and Dataframes APIs to create a machine learning pipeline capable of learning real valued vector labels despite relatively low dimensional feature spaces. Our deep dive will include feature engineering techniques employed, the reference architecture for our n-dimensional regression technique and the extra data formatting steps required for applying the built in evaluator tools to n-dimensional models.
For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-sidhu
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Sammy Sidhu, Senior Engineer at DeepScale, presents the "A Shallow Dive into Training Deep Neural Networks" tutorial at the May 2017 Embedded Vision Summit.
In this talk, Sidhu introduces the basics of training deep neural network models for vision tasks. He begins by explaining fundamental training concepts and terms, including loss functions and gradients. He then provides an accessible explanation of how the training process works. Next, he highlights common challenges in training deep neural networks, such as overfitting, and explores proven techniques for addressing these challenges, including regularization and data augmentation. Throughout, he illustrates training techniques and challenges using examples taken from real-world applications.
In this video I’m going to show you how SigOpt can help you amplify your machine learning and AI models by optimally tuning them using our black-box optimization platform.
Video: https://youtu.be/EjGrRxXWg8o
The SigOpt platform provides an ensemble of state-of-the-art Bayesian and Global optimization algorithms via a simple Software-as-a-Service API.
AWS Metering provides customers with detailed usage information (down to a specific Amazon EC2 instance or Amazon S3 bucket used in a single hour), enabling them to gain deep insights into their utilization of cloud resources. However, this level of transparency is not available across most customers’ traditional IT infrastructure, making it difficult to understand what resources are being used, when, and by whom. Join us in this session to learn how to meter, measure, and understand your usage from AWS, on-premises data centers, containers, serverless compute, even other clouds across your IT infrastructure. We show you how to meter your non-AWS resources to make smarter decisions about your business and investment in the cloud.
Production model lifecycle management 2016 09Greg Makowski
This talk covers going over the various stages of building data mining models, putting them into production and eventually replacing them. A common theme throughout are three attributes of predictive models: accuracy, generalization and description. I assert you can have it all, and having all three is important for managing the lifecycle. A subtle point is that this is a step to developing embedded, automated data mining systems which can figure out themselves when they need to be updated.
LinkedIn talk at Netflix ML Platform meetup Sep 2019Faisal Siddiqi
In this talk at the Netflix Machine Learning Platform Meetup on 12 Sep 2019, Kinjal Basu from LinkedIn discussed Online Parameter Selection for web-based Ranking vis Bayesian Optimization
Commercial Management and Cost Optimization on AWS - AWS Online Tech TalksAmazon Web Services
Learning Objectives:
- Understand primary levers available to optimize your AWS environment
- Aware of tools that will help you to optimize your AWS environment
- Understand organizational mechanisms used by customers to promote optimisation
Tuning for Systematic Trading: Talk 2: Deep LearningSigOpt
This talk explains how to train deep learning and other expensive models with parallelism and multitask optimization to reduce wall clock time. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.
Auto Scaling Prime Time: Target Tracking Hits the Bullseye at Netflix - CMP31...Amazon Web Services
Auto Scaling allows cloud resources to scale automatically in reaction to the dynamic needs of customers, which helps to improve application availability and reduce costs. New target tracking scaling policies for Auto Scaling make it easy to set up dynamic scaling for your application in just a few steps. With target tracking, you simply select a load metric for your application, set the target value, and Auto Scaling adjusts resources as needed to maintain that target. In this session, you will learn how you can use target tracking to setup sound scaling policies “without the fuss”, and improve availability under fluctuating demand. Netflix is spending $6 billion on original content this year, with shows like The Crown, Narcos, and Stranger Things, and plenty more in the future. Hear how they’re using target tracking scaling policies to improve performance, reliability and availability around the world at prime times, without over-provisioning - and without guesswork. They will share some best practices examples of how target tracking allows their infrastructure to automatically adapt to changing traffic patterns in order to keep their audience entertained and their costs on target.
Parametric estimation is one of the four primary methods that project companies use to produce estimates for the cost, duration and effort of a project.
For parametric estimation, the person in charge of the estimates will model (or describe) the project using a set of algorithms. For instance: let’s say that your project includes carrying out a survey of 300 people. Each interview contains 20 multiple-choice questions, and past experience has shown that they take 10 minutes to administer. According to parametric estimation, the total effort for this task will be:
E = nb of interviews × 10 minutes = 3000 minutes = 5 hours
The Well-Architected workshop is a free, advanced-level workshop that describes the benefits of the AWS Well-Architected Framework. This enables customers to review and improve their cloud architectures and better understand the business impact of their design decisions. It addresses general design principles, best practices, and guidance in five pillars of the Well-Architected Framework. We recommend that attendees of this course have the following pre-requisites: Strong working knowledge of AWS core services and features, as well as previous architectural experience.
Most large-scale online recommender systems like newsfeed ranking, people recommendations, job recommendations, etc. often have multiple utilities or metrics that need to be simultaneously optimized. The machine learning models that are trained to optimize a single utility are combined together through parameters to generate the final ranking function. These combination parameters drive business metrics. Finding the right choice of the parameters is often done through online A/B experimentation, which can be incredibly complex and time-consuming, especially considering the non-linear effects of these parameters on the metrics of interest.
In this tutorial, we will talk about how we can apply Bayesian Optimization techniques to obtain the parameters for such complex online systems in order to balance the competing metrics. First, we will provide an in-depth introduction to Bayesian Optimization, covering some of the basics as well as the recent advances in the field. Second, we will talk about how to formulate a real-world recommender system problem as a black-box optimization problem that can be solved via Bayesian Optimization. We will focus on a few key problems such as newsfeed ranking, people recommendations, job recommendations, etc. Third, we will talk about the architecture of the solution and how we are able to deploy it for large-scale systems. Finally, we will discuss the extensions and some of the future directions in this domain.
Learn the best practices and considerations for cost optimising your AWS environment. We will cover best practices for right sizing, scheduling instances to reduce costs, and finally, how you can save up to 75% on OnDemand costs using reserved instances.
Save up to 90% and Run Production Workloads on Spot - CMP307 - re:Invent 2017Amazon Web Services
In this session, you learn how to effectively harness Spot Instances for production workloads. Amazon EC2 Spot Instances enable you to use spare EC2 computing capacity, capacity that is often 90% less than on-demand prices. We explore application requirements to use Spot Instances, best practices learned from thousands of customers, and the services that make it easy. Finally, we run through practical examples of how to use Spot for the most common production workloads, the common pitfalls customers run into, and how to avoid them.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
