The document discusses how to accelerate and amplify the impact of modelers. It describes SigOpt's platform which allows for automated hyperparameter optimization, tracking of experiments, and reuse of insights. This helps make modeling faster, cheaper, and better. The document advocates balancing flexibility and standardization, maximizing resource utilization through techniques like parallelization, and unlocking new capabilities such as optimizing expensive models or exploring architectures.
This session is a continuation of “Apply MLOps at Scale” at Data+AI Summit Europe 2020 and “Automated Production Ready ML at Scale” at Spark AI Summit at Europe 2019. In this session you will learn how H&M is continuing to evolve and develop their AI platform in order to democratize and accelerate AI usage across the full H&M group, including speed to production, data abstraction, feature store, pipeline orchestration, etc.
Our existing reference architecture has been adapted by multiple product teams managing 100’s of models across the entire H&M value chain and enables data scientists to develop model in a highly interactive environment, enabling engineers to manage large scale model training and model serving pipeline with full traceability. The current evolution aims to both reduce the time to introduce new features to the market as well as the learning feedback loop by democratizing AI in the organisation and persistent focus on sound MLOps principles.
A Reference Architecture for Digitalization in the Pharmaceutical IndustryCapgemini
A Reference Architecture for Digitalization in the Pharmaceutical Industry - Alina Chircu, Bentley University; Levent Sözer, Capgemini Germany; Eldar Sultanow, Capgemini Germany
INFORMATIK 2017
47. Jahrestagung der Gesellschaft für Informatik e.V. (GI) | 25.-29.9.2017 | Chemnitz
Workshop Enterprise Architecture Management in Forschung und Praxis
IT Operation Management Automation Roadmap post PandemicManasKumarLenka1
Captures a roadmap as to how Post Pandemic, Organizations Like CGI can in grow in ITOM automation space considering they have existing IP and experience IN RPA , BPM space
Deep reinforcement learning for industrial process automation nicolas deruytterBluecrux
Are you familiar with the fascinating world of: Artificial Intelligence? Autonomous systems? Advanced Process Control? Machine Learning? Predictive Maintenance? This presentation features them all!
Presented by Nicolas Deruytter, Founder & CEO ML6 on Supply Chain 4.0 : ready to operate in the digital era? (29 Nov, 2018)
This session was recorded in NYC on October 22nd, 2019.
Video recording of the session can be viewed here: https://youtu.be/Z0quYTZr6C0
Description: How businesses can recession proof themselves by using the power of the Ascend Analytical Sandbox; and how Experian is leveraging its vast data to make sure every borrower is presented in the best light in front of the lenders.
Bio: Ankit is the Product & Innovation Expert at Experian leading the overall roadmap for the Ascend Analytical Sandbox; a one-stop shop for insights, model development, and results measurement.
This session is a continuation of “Apply MLOps at Scale” at Data+AI Summit Europe 2020 and “Automated Production Ready ML at Scale” at Spark AI Summit at Europe 2019. In this session you will learn how H&M is continuing to evolve and develop their AI platform in order to democratize and accelerate AI usage across the full H&M group, including speed to production, data abstraction, feature store, pipeline orchestration, etc.
Our existing reference architecture has been adapted by multiple product teams managing 100’s of models across the entire H&M value chain and enables data scientists to develop model in a highly interactive environment, enabling engineers to manage large scale model training and model serving pipeline with full traceability. The current evolution aims to both reduce the time to introduce new features to the market as well as the learning feedback loop by democratizing AI in the organisation and persistent focus on sound MLOps principles.
A Reference Architecture for Digitalization in the Pharmaceutical IndustryCapgemini
A Reference Architecture for Digitalization in the Pharmaceutical Industry - Alina Chircu, Bentley University; Levent Sözer, Capgemini Germany; Eldar Sultanow, Capgemini Germany
INFORMATIK 2017
47. Jahrestagung der Gesellschaft für Informatik e.V. (GI) | 25.-29.9.2017 | Chemnitz
Workshop Enterprise Architecture Management in Forschung und Praxis
IT Operation Management Automation Roadmap post PandemicManasKumarLenka1
Captures a roadmap as to how Post Pandemic, Organizations Like CGI can in grow in ITOM automation space considering they have existing IP and experience IN RPA , BPM space
Deep reinforcement learning for industrial process automation nicolas deruytterBluecrux
Are you familiar with the fascinating world of: Artificial Intelligence? Autonomous systems? Advanced Process Control? Machine Learning? Predictive Maintenance? This presentation features them all!
Presented by Nicolas Deruytter, Founder & CEO ML6 on Supply Chain 4.0 : ready to operate in the digital era? (29 Nov, 2018)
This session was recorded in NYC on October 22nd, 2019.
Video recording of the session can be viewed here: https://youtu.be/Z0quYTZr6C0
Description: How businesses can recession proof themselves by using the power of the Ascend Analytical Sandbox; and how Experian is leveraging its vast data to make sure every borrower is presented in the best light in front of the lenders.
Bio: Ankit is the Product & Innovation Expert at Experian leading the overall roadmap for the Ascend Analytical Sandbox; a one-stop shop for insights, model development, and results measurement.
PTC Joins New Salesforce Analytics Cloud Ecosystem to Extend Internet of Thin...PTC
Continuing with its Internet of Things strategy to enable customers to bring smart, connected products to market faster, PTC (Nasdaq: PTC) today announced it has joined the Salesforce Analytics Cloud ecosystem.
Build an AI Roadmap and Win the Consumer Goods Intelligence RaceGib Bassett
My presentation from Salesforce Connections 2018. In it, I describe why it's important to think about a use case driven strategy for advanced analytics and AI.
No fewer than 80% have digital transformation at the centre of their corporate strategy with the aim of improving efficiency, driving innovation and becoming more agile. Though it's clear that insight into the data they hold is going to help them get there, many organisations find themselves at a crossroads. Big data, machine learning, data science: these are all initiatives every company knows they should take on in order to evolve their business, yet few know how to tackle the projects for successful outcomes.
PTC helps companies around the world reinvent the way they design, manufacture, operate, and service things in and for a smart, connected world. In 1986 we revolutionized digital 3D design, and in 1998 were first to market with Internet-based product lifecycle management. Today, our leading industrial innovation platform and field-proven solutions enable you to unlock value at the convergence of the physical and digital worlds. With PTC, manufacturers and an ecosystem of partners and developers can capitalize on the promise of the Internet of Things and augmented reality technology today and drive the future of innovation.
[AI Webinar Series P1] - How Advanced Text Analytics Can Increase the Operati...JK Tech
Digitization is considered as the next step-change that will have a bigger impact on businesses than even the internet. To win in the digital journey, companies must act now, or be left behind wondering what happened!
In this webinar series, JKT Smart Analytics demonstrates how they empower their customers to create maximum business value out of this eminent Digital data explosion through digital business empowerment by leveraging the digitization to increase their top-line revenue – customer experience, optimize the bottom-line costs – operational efficiency, enhancing the safety factor and reinventing the business process in line with the changing world.
This webinar is focused on how our AI-based text analytics solutions – First, JKT Social Media Radar; a SaaS-based AI NLP Platform, helping organizations to gain insights on market and customer perceptions on their brands, products & services. Secondly, Sales Promotion Recommendation Engine helps customers to enhance their top-line growth and streamline the bottom-line costs.
KEY TAKEAWAYS:
1) How should a business plan their journey through the Digital data revolution?
2) How can a company make use of digital data to create effective data strategies for the increased outcome(s)?
3) How IT practitioners can catalyst the digital data mining journey and attract business adoption?
4) JKT Social Media Radar solution – What, Why, Supporting Business applications, and more.
5) How can companies reduce operational costs by automating human effort-intensive tasks using cognitive Analytics?
Data Science Innovation Summit Philadelphia 2019 - parivedaRyan Gross
Pariveda's Ryan Gross presented on the ways that companies are transforming themselves using data and data science. Many of the challenges that organizations run into are cultural and/or process related. The presentation goes through a framework for getting your organization started successfully with Data Science.
With the growing demands of digitally empowered customers, companies are looking to deliver differentiated CX as a source of long-term competitive advantage. Learn how to deliver best-in-class customer experience solutions that add value to your business.
Sap Leonardo - what is it, and why would I want one?Tom Raftery
A quick run through of the technologies running through SAP's Innovation portfolio of products, called SAP Leonardo, and use cases where it has been deployed successfully with customers
This talk discusses the intuition behind Bayesian optimization with and without multiple metrics. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.
SigOpt at O'Reilly - Best Practices for Scaling Modeling PlatformsSigOpt
Companies are increasingly building modeling platforms to empower their researchers to efficiently scale the development and productionalization of their models. Scott Clark and Matt Greenwood share a case study from a leading algorithmic trading firm to illustrate best practices for building these types of platforms in any industry. Join in to learn how Two Sigma, a leading quantitative investment and technology firm, solved its model optimization problem.
Metric Management: a SigOpt Applied Use CaseSigOpt
These slides correspond to a recording of a live webcast of a demo of Metric Management functionality in SigOpt, keeping model size down while increasing validation accuracy for a road sign image classification problem.
Tuning for Systematic Trading: Talk 3: Training, Tuning, and Metric StrategySigOpt
This talk explains how you can train and tune efficiently using metric strategy to assign, store, and optimize a variety of metrics, even changing them over time. Tobias Andreassen, who supports a number of our systematic trading customers, explained how he helps customers tune more efficiently with these SigOpt features in real-world scenarios.
PTC Joins New Salesforce Analytics Cloud Ecosystem to Extend Internet of Thin...PTC
Continuing with its Internet of Things strategy to enable customers to bring smart, connected products to market faster, PTC (Nasdaq: PTC) today announced it has joined the Salesforce Analytics Cloud ecosystem.
Build an AI Roadmap and Win the Consumer Goods Intelligence RaceGib Bassett
My presentation from Salesforce Connections 2018. In it, I describe why it's important to think about a use case driven strategy for advanced analytics and AI.
No fewer than 80% have digital transformation at the centre of their corporate strategy with the aim of improving efficiency, driving innovation and becoming more agile. Though it's clear that insight into the data they hold is going to help them get there, many organisations find themselves at a crossroads. Big data, machine learning, data science: these are all initiatives every company knows they should take on in order to evolve their business, yet few know how to tackle the projects for successful outcomes.
PTC helps companies around the world reinvent the way they design, manufacture, operate, and service things in and for a smart, connected world. In 1986 we revolutionized digital 3D design, and in 1998 were first to market with Internet-based product lifecycle management. Today, our leading industrial innovation platform and field-proven solutions enable you to unlock value at the convergence of the physical and digital worlds. With PTC, manufacturers and an ecosystem of partners and developers can capitalize on the promise of the Internet of Things and augmented reality technology today and drive the future of innovation.
[AI Webinar Series P1] - How Advanced Text Analytics Can Increase the Operati...JK Tech
Digitization is considered as the next step-change that will have a bigger impact on businesses than even the internet. To win in the digital journey, companies must act now, or be left behind wondering what happened!
In this webinar series, JKT Smart Analytics demonstrates how they empower their customers to create maximum business value out of this eminent Digital data explosion through digital business empowerment by leveraging the digitization to increase their top-line revenue – customer experience, optimize the bottom-line costs – operational efficiency, enhancing the safety factor and reinventing the business process in line with the changing world.
This webinar is focused on how our AI-based text analytics solutions – First, JKT Social Media Radar; a SaaS-based AI NLP Platform, helping organizations to gain insights on market and customer perceptions on their brands, products & services. Secondly, Sales Promotion Recommendation Engine helps customers to enhance their top-line growth and streamline the bottom-line costs.
KEY TAKEAWAYS:
1) How should a business plan their journey through the Digital data revolution?
2) How can a company make use of digital data to create effective data strategies for the increased outcome(s)?
3) How IT practitioners can catalyst the digital data mining journey and attract business adoption?
4) JKT Social Media Radar solution – What, Why, Supporting Business applications, and more.
5) How can companies reduce operational costs by automating human effort-intensive tasks using cognitive Analytics?
Data Science Innovation Summit Philadelphia 2019 - parivedaRyan Gross
Pariveda's Ryan Gross presented on the ways that companies are transforming themselves using data and data science. Many of the challenges that organizations run into are cultural and/or process related. The presentation goes through a framework for getting your organization started successfully with Data Science.
With the growing demands of digitally empowered customers, companies are looking to deliver differentiated CX as a source of long-term competitive advantage. Learn how to deliver best-in-class customer experience solutions that add value to your business.
Sap Leonardo - what is it, and why would I want one?Tom Raftery
A quick run through of the technologies running through SAP's Innovation portfolio of products, called SAP Leonardo, and use cases where it has been deployed successfully with customers
This talk discusses the intuition behind Bayesian optimization with and without multiple metrics. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.
SigOpt at O'Reilly - Best Practices for Scaling Modeling PlatformsSigOpt
Companies are increasingly building modeling platforms to empower their researchers to efficiently scale the development and productionalization of their models. Scott Clark and Matt Greenwood share a case study from a leading algorithmic trading firm to illustrate best practices for building these types of platforms in any industry. Join in to learn how Two Sigma, a leading quantitative investment and technology firm, solved its model optimization problem.
Metric Management: a SigOpt Applied Use CaseSigOpt
These slides correspond to a recording of a live webcast of a demo of Metric Management functionality in SigOpt, keeping model size down while increasing validation accuracy for a road sign image classification problem.
Tuning for Systematic Trading: Talk 3: Training, Tuning, and Metric StrategySigOpt
This talk explains how you can train and tune efficiently using metric strategy to assign, store, and optimize a variety of metrics, even changing them over time. Tobias Andreassen, who supports a number of our systematic trading customers, explained how he helps customers tune more efficiently with these SigOpt features in real-world scenarios.
This webinar, hosted by SigOpt co-founder and CEO Scott Clark, explains how advanced features can help you achieve your modeling goals. These features include metric definition and multimetric optimization, conditional parameters, and multitask optimization for long training cycles.
Tuning for Systematic Trading: Talk 2: Deep LearningSigOpt
This talk explains how to train deep learning and other expensive models with parallelism and multitask optimization to reduce wall clock time. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.
SigOpt CEO Scott Clark provides insights for modeling at scale in systematic trading. SigOpt works with algorithmic trading firms that collectively represent $300 billion in assets under management (AUM). In this presentation, Scott draws on this experience to provide a few critical insights to how these companies effectively model at scale. Alongside these insights, Scott shares a more specific case study from working with Two Sigma, a leading systematic investment manager.
Experimentation to Industrialization: Implementing MLOpsDatabricks
In this presentation, drawing upon Thorogood’s experience with a customer’s global Data & Analytics division as their MLOps delivery partner, we share important learnings and takeaways from delivering productionized ML solutions and shaping MLOps best practices and organizational standards needed to be successful.
We open by providing high-level context & answering key questions such as “What is MLOps exactly?” & “What are the benefits of establishing MLOps Standards?”
The subsequent presentation focuses on our learnings & best practices. We start by discussing common challenges when refactoring experimentation use-cases & how to best get ahead of these issues in a global organization. We then outline an Engagement Model for MLOps addressing: People, Processes, and Tools. ‘Processes’ highlights how to manage the often siloed data science use case demand pipeline for MLOps & documentation to facilitate seamless integration with an MLOps framework. ‘People’ provides context around the appropriate team structures & roles to be involved in an MLOps initiative. ‘Tools’ addresses key requirements of tools used for MLOps, considering the match of services to use-cases.
Tuning the Untunable - Insights on Deep Learning OptimizationSigOpt
Patrick Hayes originally gave this talk at ODSC West in 2018. During this talk, Patrick discusses a couple key barriers to deep learning optimization and how SigOpt solves them. First, Patrick discusses the problem of lengthy training cycles and how novel techniques like multitask optimization are designed to use partial information to solve this challenge. Second, Patrick discusses automated cluster management and how solving this problem makes it much easier to manage training cycles for these models.
Advanced Optimization for the Enterprise WebinarSigOpt
Building on the TWIML eBook, TWIMLcon event and TWIML podcast series that explore Machine Learning Platforms in great detail, this webinar examines the machine learning platforms that power enterprise leaders in AI. SigOpt CEO Scott Clark will provide an overview of critical technical capabilities that our customers have prioritized in their ML platforms.
Review these slides to learn about:
- Critical capabilities for data, experiment and model management
- Tradeoffs between building and buying these capabilities
- Lessons from the implementation of these platforms by AI leaders
Why focus on these platforms and the capabilities that power them? Nearly every company is investing in machine learning that differentiates products or generates revenue. These so-called "differentiated models" represent the biggest opportunity for AI to transform the business. Most of these teams find success hiring expert data scientists and machine learning engineers who can build these models. But most of these teams also struggle to create a more sustainable, scalable and reproducible process for model development, and have begun building ML platforms to tackle this challenge.
In this video I’m going to show you how SigOpt can help you amplify your machine learning and AI models by optimally tuning them using our black-box optimization platform.
Video: https://youtu.be/EjGrRxXWg8o
The SigOpt platform provides an ensemble of state-of-the-art Bayesian and Global optimization algorithms via a simple Software-as-a-Service API.
A presentation covers how data science is connected to build effective machine learning solutions. How to build end to end solutions in Azure ML. How to build, model, and evaluate algorithms in Azure ML.
Driving Digital Transformation with Machine Learning in Oracle AnalyticsPerficient, Inc.
The adoption of machine learning (ML) is increasing at near-breakneck speed. As organizations seek innovative ideas on how to improve the business, Oracle Analytics Cloud with ML capabilities is leading the charge. With built-in drag-and-drop functions into visualizations and autonomous prediction execution, Oracle Analytics puts the power of machine learning in your hands.
We covered how Oracle Analytics can connect various data sources, allow you to apply ML without being statistically savvy, and easily build your story in presentation format.
Discussion included:
-In-depth look at Oracle Analytics Cloud
-How to connect different data sources like SaaS applications, data lakes, external data sources and more
-Custom-trained ML models demonstration
-Real-world business use case from end to end
In this deck I’m going to show you how SigOpt can help you amplify your trading models by optimally tuning them using our black-box optimization platform.
MLOps and Data Quality: Deploying Reliable ML Models in ProductionProvectus
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.
Managing the Machine Learning Lifecycle with MLflowDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. MLflow is an open-source project from Databricks aiming to solve some of these challenges such as experiment tracking, reproducibility, model packaging, deployment, and governance, in order to manage and accelerate the lifecycle of your ML projects.
Using Data Science to Build an End-to-End Recommendation SystemVMware Tanzu
We get recommendations everyday: Facebook recommends people we should connect with; Amazon recommends products we should buy; and Google Maps recommends routes to take. What all these recommendation systems have in common are data science and modern software development.
Recommendation systems are also valuable for companies in industries as diverse as retail, telecommunications, and energy. In a recent engagement, for example, Pivotal data scientists and developers worked with a large energy company to build a machine learning-based product recommendation system to deliver intelligent and targeted product recommendations to customers to increase revenue.
In this webinar, Pivotal data scientist Ambarish Joshi will take you step-by-step through the engagement, explaining how he and his Pivotal colleagues worked with the customer to collect and analyze data, develop predictive models, and operationalize the resulting insights and surface them via APIs to customer-facing applications. In addition, you will learn how to:
- Apply agile practices to data science and analytics.
- Use test-driven development for feature engineering, model scoring, and validating scripts.
- Automate data science pipelines using pyspark scripts to generate recommendations.
- Apply a microservices-based architecture to integrate product recommendations into mobile applications and call center systems.
Presenters: Ambarish Joshi and Jeff Kelly, Pivotal
Data Scientists and Machine Learning practitioners, nowadays, seem to be churning out models by the dozen and they continuously experiment to find ways to improve their accuracies. They also use a variety of ML and DL frameworks & languages , and a typical organization may find that this results in a heterogenous, complicated bunch of assets that require different types of runtimes, resources and sometimes even specialized compute to operate efficiently.
But what does it mean for an enterprise to actually take these models to "production" ? How does an organization scale inference engines out & make them available for real-time applications without significant latencies ? There needs to be different techniques for batch (offline) inferences and instant, online scoring. Data needs to be accessed from various sources and cleansing, transformations of data needs to be enabled prior to any predictions. In many cases, there maybe no substitute for customized data handling with scripting either.
Enterprises also require additional auditing and authorizations built in, approval processes and still support a "continuous delivery" paradigm whereby a data scientist can enable insights faster. Not all models are created equal, nor are consumers of a model - so enterprises require both metering and allocation of compute resources for SLAs.
In this session, we will take a look at how machine learning is operationalized in IBM Data Science Experience (DSX), a Kubernetes based offering for the Private Cloud and optimized for the HortonWorks Hadoop Data Platform. DSX essentially brings in typical software engineering development practices to Data Science, organizing the dev->test->production for machine learning assets in much the same way as typical software deployments. We will also see what it means to deploy, monitor accuracies and even rollback models & custom scorers as well as how API based techniques enable consuming business processes and applications to remain relatively stable amidst all the chaos.
Speaker
Piotr Mierzejewski, Program Director Development IBM DSX Local, IBM
Similar to Modeling at Scale: SigOpt at TWIMLcon 2019 (20)
Optimizing BERT and Natural Language Models with SigOpt Experiment ManagementSigOpt
SigOpt Machine Learning Engineer Meghana Ravikumar explains how she reduced the size of a BERT natural language model trained on the SQUAD 2.0 question-answer database, to reduce its size while maintaining performance using a "distillation" process optimized with SigOpt's Experiment Management functionality.
SigOpt's Fay Kallel, Head of Product, and Jim Blomo, Head of Engineering, describe the latest updates to SigOpt, a suite of features that help you manage your modeling process.
Efficient NLP by Distilling BERT and Multimetric OptimizationSigOpt
SigOpt ML Engineer Meghana Ravikumar explains how to use multimetric optimization to achieve a more efficient, compact BERT model to perform on a question-answering task.
SigOpt Research Engineer Michael McCourt and DarwinAI CTO Alexander Wong explain how they used SigOpt and hyperparameter optimization to successfully improve accuracy of detecting COVID-19 cases from chest X-Rays, using the COVID-Net model and the COVIDx open dataset.
Tuning Data Augmentation to Boost Model PerformanceSigOpt
In this webinar, SigOpt ML Engineer Meghana Ravikumar presents on and builds an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning—fine tuning and feature extraction—and the impact of Multitask optimization, a more efficient form of Bayesian optimization, on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will use image augmentation to double the size of the dataset to boost the classifier’s performance. Instead of manually tuning the hyperparameters associated with image augmentation, we will use Multitask Optimization to learn these hyperparameters using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also explore the features of these augmented images and the downstream implications for our image classifier.
SigOpt helps your algorithmic traders and data scientists build better models faster. Learn how to integrate SigOpt into your modeling platform for quick ROI for your data science team.
Interactive Tradeoffs Between Competing Offline Metrics with Bayesian Optimiz...SigOpt
Many real world applications - machine learning models, simulators, etc. - have multiple competing metrics that define performance; these require practitioners to carefully consider potential tradeoffs. However, assessing and ranking this tradeoff is nontrivial, especially when the number of metrics is more than two. Often times, practitioners scalarize the metrics into a single objective, e.g., using a weighted sum.
In this talk, we pose this problem as a constrained multi-objective optimization problem. By setting and updating the constraints, we can efficiently explore only the region of the Pareto efficient frontier of the model/system of most interest. We motivate this problem with the application of an experimental design setting, where we are trying to fabricate high performance glass substrate for solar cell panels.
As data science workloads grow, so does their need for infrastructure. But, is it fair to ask data scientists to also become infrastructure experts? If not the data scientists, then, who is responsible for spinning up and managing data science infrastructure? This talk will address the context in which ML infrastructure is emerging, walk through two examples of ML infrastructure tools for launching hyperparameter optimization jobs, and end with some thoughts for building better tools in the future.
Originally given as a talk at the PyData Ann Arbor meetup (https://www.meetup.com/PyData-Ann-Arbor/events/260380989/)
SigOpt at Uber Science Symposium - Exploring the spectrum of black-box optimi...SigOpt
At the inaugural Uber science symposium, SigOpt research engineer Bolong (Harvey) Cheng shares insights on black-box optimization from his experience working with both leading academics and innovative enterprises.
Training and tuning models with lengthy training cycles like those in deep learning can be extremely expensive and may sometimes involve techniques that degrade performance. We'll explore recent research on optimization strategies to efficiently tune these types of deep learning models. We will provide benchmarks and comparisons to other popular methods for optimizing the models, and we'll recommend valuable areas for further applied research.
SigOpt at GTC - Reducing operational barriers to optimizationSigOpt
Advanced hardware like NVIDIA technology lowers technical barriers to model size and scope, but issues remain in areas like model performance and training infrastructure management. We'll discuss operational challenges to training models at scale with a particular focus on how training management and hyperparameter tuning can inform each other to accomplish specific goals. We'll also explore techniques like parallelism and scheduling, discuss their impact on model optimization, and compare various techniques. We'll also evaluate results of this approach. In particular, we'll focus on how new tools that automate training orchestration accelerate model development and increase the volume and quality of models in production.
SigOpt at MLconf - Reducing Operational Barriers to Model TrainingSigOpt
In this talk at MLconf NYC, Alexandra Johnson, platform engineering lead at SigOpt, discusses common operational challenges with scaling model training and how solutions are designed to
Machine learning infrastructure solve data scientists' problems using infrastructure tools. This talk shows the case study of building SigOpt Orchestrate, an ML infrastructure tool. The talk highlights how data scientists' concerns as user mapped to solutions with some of today's most popular infrastructure tools.
To learn more about SigOpt Orchestrate: https://sigopt.com/orchestrate
Originally given as a talk for UC Berkeley's Women in Electrical Engineering and Computer Science group on January 24, 2019.
AWS makes it easy to build, train, tune, and deploy Machine Learning (ML) models. If you're excited to get started with ML on AWS but want a refresher on the ML concepts behind build, train, tune, and deploy, this Dev Chat is for you.
Originally delivered as a Dev Chat at AWS Summit SF by Software Engineer Alexandra Johnson
Tips and techniques for hyperparameter optimizationSigOpt
All machine learning and artificial intelligence pipelines - from reinforcement agents to deep neural nets - have tunable hyperparameters. Optimizing these hyperparameters can take a model from scrappy prototype to production-ready system. This presentation shows techniques for performing hyperparameter optimization from an engineer who builds advanced and widely used optimization tools.
MLconf 2017 Seattle Lunch Talk - Using Optimal Learning to tune Deep Learning...SigOpt
In this talk we introduce Bayesian Optimization as an efficient way to optimize machine learning model parameters, especially when evaluating different parameters is time-consuming or expensive. Deep learning pipelines are notoriously expensive to train and often have many tunable parameters including hyperparameters, the architecture, feature transformations that can have a large impact on the efficacy of the model.
We will motivate the problem by giving several example applications using multiple open source deep learning frameworks and open datasets. We’ll compare the results of Bayesian Optimization to standard techniques like grid search, random search, and expert tuning.
Using Optimal Learning to Tune Deep Learning PipelinesSigOpt
SigOpt talk from NVIDIA GTC 2017 and AWS SF AI Day
We'll introduce Bayesian optimization as an efficient way to optimize machine learning model parameters, especially when evaluating different parameters is time consuming or expensive. Deep learning pipelines are notoriously expensive to train and often have many tunable parameters, including hyperparameters, the architecture, and feature transformations, that can have a large impact on the efficacy of the model. We'll provide several example applications using multiple open source deep learning frameworks and open datasets. We'll compare the results of Bayesian optimization to standard techniques like grid search, random search, and expert tuning. Additionally, we'll present a robust benchmark suite for comparing these methods in general.
Common Problems in Hyperparameter OptimizationSigOpt
Originally given at MLConf NYC 2017.
All large machine learning pipelines have tunable parameters, commonly referred to as hyperparameters. Hyperparameter optimization is the process by which we find the values for these parameters that cause our system to perform the best. SigOpt provides a Bayesian optimization platform that is commonly used for hyperparameter optimization, and I’m going to share some of the common problems we’ve seen when integrating into machine learning pipelines.
Opendatabay - Open Data Marketplace.pptxOpendatabay
Opendatabay.com unlocks the power of data for everyone. Open Data Marketplace fosters a collaborative hub for data enthusiasts to explore, share, and contribute to a vast collection of datasets.
First ever open hub for data enthusiasts to collaborate and innovate. A platform to explore, share, and contribute to a vast collection of datasets. Through robust quality control and innovative technologies like blockchain verification, opendatabay ensures the authenticity and reliability of datasets, empowering users to make data-driven decisions with confidence. Leverage cutting-edge AI technologies to enhance the data exploration, analysis, and discovery experience.
From intelligent search and recommendations to automated data productisation and quotation, Opendatabay AI-driven features streamline the data workflow. Finding the data you need shouldn't be a complex. Opendatabay simplifies the data acquisition process with an intuitive interface and robust search tools. Effortlessly explore, discover, and access the data you need, allowing you to focus on extracting valuable insights. Opendatabay breaks new ground with a dedicated, AI-generated, synthetic datasets.
Leverage these privacy-preserving datasets for training and testing AI models without compromising sensitive information. Opendatabay prioritizes transparency by providing detailed metadata, provenance information, and usage guidelines for each dataset, ensuring users have a comprehensive understanding of the data they're working with. By leveraging a powerful combination of distributed ledger technology and rigorous third-party audits Opendatabay ensures the authenticity and reliability of every dataset. Security is at the core of Opendatabay. Marketplace implements stringent security measures, including encryption, access controls, and regular vulnerability assessments, to safeguard your data and protect your privacy.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
StarCompliance is a leading firm specializing in the recovery of stolen cryptocurrency. Our comprehensive services are designed to assist individuals and organizations in navigating the complex process of fraud reporting, investigation, and fund recovery. We combine cutting-edge technology with expert legal support to provide a robust solution for victims of crypto theft.
Our Services Include:
Reporting to Tracking Authorities:
We immediately notify all relevant centralized exchanges (CEX), decentralized exchanges (DEX), and wallet providers about the stolen cryptocurrency. This ensures that the stolen assets are flagged as scam transactions, making it impossible for the thief to use them.
Assistance with Filing Police Reports:
We guide you through the process of filing a valid police report. Our support team provides detailed instructions on which police department to contact and helps you complete the necessary paperwork within the critical 72-hour window.
Launching the Refund Process:
Our team of experienced lawyers can initiate lawsuits on your behalf and represent you in various jurisdictions around the world. They work diligently to recover your stolen funds and ensure that justice is served.
At StarCompliance, we understand the urgency and stress involved in dealing with cryptocurrency theft. Our dedicated team works quickly and efficiently to provide you with the support and expertise needed to recover your assets. Trust us to be your partner in navigating the complexities of the crypto world and safeguarding your investments.
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
5. SigOpt. Confidential.
Your firewall
Training
Data
AI, ML, DL,
Simulation Model
Model Evaluation
or Backtest
Testing
Data
New
Configurations
Objective
Metric
Better
Results
EXPERIMENT INSIGHTS
Track, organize, analyze and
reproduce any model
ENTERPRISE PLATFORM
Built to fit any stack and scale
with your needs
OPTIMIZATION ENGINE
Explore and exploit with a
variety of techniques
RESTAPI
Configuration
Parameters or
Hyperparameters
Your data
and models
stay private
Iterative, automated optimization
Integrates
with any
modeling
stack
6. $300B+
in assets under management
Current SigOpt algorithmic
trading customers represent
$500B+
in market capitalization
Current SigOpt enterprise customers
across six industries represent
7. SigOpt. Confidential.
“We’ve integrated SigOpt’s optimization service and
are now able to get better results faster and cheaper
than any solution we’ve seen before.”
Matt Adereth
Managing Director
8. Lessons
1. Balance flexibility with standardization
2. Maximize resource utilization
3. Unlock new modeling capabilities
10. Simulations & Evaluation
Modeling FrameworkData
Management
Data access
Data pipelines
Data labeling
Feature repo.
Feature prov.
Backfills,
versioning
Compute Environment
Model Execution
Model Serving
Instrumentation
Data validation
Phased deploys
Online A/B tests
Batch scoring
On-Premise
Optimization & Experimentation
Hybrid Multi-Cloud
Tracking, Visualization,
Version Control
Automated Hyperparameter
Optimization
Distributed Tuning & Job
Scheduling
Workflows
Auto Feature &
AutoML
Coding
Environment
Framework
Support
Library
Backtests Metric Iteration
Model
Evaluation
Portfolio
Optimization
11. Simulations & Evaluation
Strategy: Customize and differentiate
Modeling Framework
Strategy: Modeler choice and flexibility
Data
Management
Strategy:
Customize and
differentiate
Compute Environment
Strategy: Modeler choice and flexibility
Model
Execution
Strategy:
Customize and
differentiate
Optimization & Experimentation
Strategy: Standardize and scale
12. Simulations & Evaluation
Strategy: Customize and differentiate
Modeling Framework
Strategy: Modeler choice and flexibility
Data
Management
Strategy:
Customize and
differentiate
Compute Environment
Strategy: Modeler choice and flexibility
Model
Execution
Strategy:
Customize and
differentiateOptimization & Experimentation
Strategy: Standardize and scale
13. Simulations & Evaluation
Strategy: Customize and differentiate
Modeling Framework
Strategy: Modeler choice and flexibility
Data
Management
Strategy:
Customize and
differentiate
Compute Environment
Strategy: Modeler choice and flexibility
Model
Execution
Strategy:
Customize and
differentiate
Optimization & Experimentation
Strategy: Standardize and scale
14. Simulations & Evaluation
Strategy: Customize and differentiate
Modeling Framework
Strategy: Modeler choice and flexibility
Data
Management
Strategy:
Customize and
differentiate
Compute Environment
Strategy: Modeler choice and flexibility
Model
Execution
Strategy:
Customize and
differentiate
Optimization & Experimentation
Strategy: Standardize and scale
15. Data
Management
Data access
Data pipelines
Data labeling
Feature repo.
Feature prov.
Backfills,
versioning
Simulation & Evaluation
Modeling Framework
Hardware Environment
On-Premise Hybrid Multi-Cloud
Optimization & Experimentation
Insights, Tracking,
Collaboration
Model Search,
Hyperparameter Tuning
Resource Scheduler,
Management
Backtests Metric Iteration
Model
Evaluation
Portfolio
Optimization
Model
Execution
Model Serving
Instrumentation
Data validation
Phased deploys
Online A/B tests
Batch scoring
Flexibility + Standardization
16. SigOpt. Confidential.
Benefits of Robust Experimentation & Optimization
Learn fast, fail fast
Give yourself the best chance at finding good use
cases while avoiding false negatives
Connect outputs to outcomes
Define, select and iterate on your metrics
with end-to-end evaluation
Find the global maximum
Early non-optimized decisions in the process limit
your ability to maximize performance
Boost productivity
Automate modeling tasks so modelers spend
more time applying their expertise
21. SigOpt. Confidential.
Pro Con
Manual Search Leverages expertise Not scalable, inconsistent
Grid Search Simple to implement Not scalable, often infeasible
Random Search Scalable Inefficient
Evolutionary Algorithms Effective at architecture search Very resource intensive
Bayesian Optimization Efficient, effective Can be tough to parallelize
22. SigOpt. Confidential.
Pro Con
Manual Search Leverages expertise Not scalable, inconsistent
Grid Search Simple to implement Not scalable, often infeasible
Random Search Scalable Inefficient
Evolutionary Algorithms Effective at architecture search Very resource intensive
Bayesian Optimization Efficient, effective Can be tough to parallelize
23. SigOpt. Confidential.
Pro Con
Manual Search Leverages expertise Not scalable, inconsistent
Grid Search Simple to implement Not scalable, often infeasible
Random Search Scalable Inefficient
Evolutionary Algorithms Effective at architecture search Very resource intensive
Bayesian Optimization Efficient, effective Can be tough to parallelize
24. SigOpt. Confidential.
Pro Con
Manual Search Leverages expertise Not scalable, inconsistent
Grid Search Simple to implement Not scalable, often infeasible
Random Search Scalable Inefficient
Evolutionary Algorithms Effective at architecture search Very resource intensive
Bayesian Optimization Efficient, effective Can be tough to parallelize
25. SigOpt. Confidential.
Pro Con
Manual Search Leverages expertise Not scalable, inconsistent
Grid Search Simple to implement Not scalable, often infeasible
Random Search Scalable Inefficient
Evolutionary Algorithms Effective at architecture search Very resource intensive
Bayesian Optimization Efficient, effective Can be tough to parallelize
26. Your firewall
Training
Data
AI, ML, DL,
Simulation Model
Model Evaluation
or Backtest
Testing
Data
New
Configurations
Objective
Metric
Better
Results
EXPERIMENT INSIGHTS
Track, organize, analyze and
reproduce any model
ENTERPRISE PLATFORM
Built to fit any stack and scale
with your needs
OPTIMIZATION ENGINE
Explore and exploit with a
variety of techniques
RESTAPI
Configuration
Parameters or
Hyperparameters
100x
Asynchronous
Parallelization
27. SigOpt. Confidential.
24 Days
to optimize a model without
SigOpt
3 Days
to optimize a model with
SigOpt
Learn more: https://twosigma.com/news/article/why-two-sigma-is-using-sigopt-for-automated-parameter-tuning/
30. SigOpt. Confidential.
Unlock new modeling capabilities
30
Accelerate Model Optimization
● Multitask optimization efficiently
optimizes “expensive” models
● Relevant docs
● Image classification use case
Solve for Competing Business Objectives
● Multimetric optimization empowers
you to evaluate trade-offs
● Relevant docs
● Sequence classification use case
Explore Model Architectures
● Conditional parameters result in more
efficient architecture search
● Relevant docs
● NLP use case
31. SigOpt. Confidential.
Train and tune expensive models
31
Accelerate Model Optimization
● Multitask optimization efficiently
optimizes “expensive” models
● Relevant docs
● Image classification use case
Solve for Competing Business Objectives
● Multimetric optimization empowers
you to evaluate trade-offs
● Relevant docs
● Sequence classification use case
Explore Model Architectures
● Conditional parameters result in more
efficient architecture search
● Relevant docs
● NLP use case
Combine the intelligence of Bayesian
optimization with the efficiency of early
termination techniques
32. SigOpt. Confidential.
Compare and select the right metric
32
Accelerate Model Optimization
● Multitask optimization efficiently
optimizes “expensive” models
● Relevant docs
● Image classification use case
Solve for Competing Business Objectives
● Multimetric optimization empowers
you to evaluate trade-offs
● Relevant docs
● Sequence classification use case
Explore Model Architectures
● Conditional parameters result in more
efficient architecture search
● Relevant docs
● NLP use case
Optimize multiple metrics at the same
time to inform your metric definition and
selection process
33. SigOpt. Confidential.
Identify the right tool for the job
33
Accelerate Model Optimization
● Multitask optimization efficiently
optimizes “expensive” models
● Relevant docs
● Image classification use case
Solve for Competing Business Objectives
● Multimetric optimization empowers
you to evaluate trade-offs
● Relevant docs
● Sequence classification use case
Explore Model Architectures
● Conditional parameters result in more
efficient architecture search
● Relevant docs
● NLP use case
Take into account the conditionality of
certain parameter types in the
optimization process
34. Lessons
1. Balance flexibility with standardization
2. Maximize resource utilization
3. Unlock new modeling capabilities
35. Try our solution
Our team is here to help.
Find our table at Booth 02,
and sign up to demo SigOpt today.
Join our webinar on
modeling platforms
https://sigopt.com/company
/events/webinar-capabilities
-ml-platforms/
Listen to our recent podcast with Two Sigma
https://twimlai.com/twiml-talk-273-supporting
-rapid-model-development-at-two-sigma-
with-matt-adereth-scott-clark/
Download TWIML’s Modeling Platforms ebook
https://sigopt.com/guide-for-
machine-learning-platforms/