Optimization is the most widely adopted technology of Prescriptive Analytics, but also the most challenging to implement. In this two-part workshop, we show how AMPL gets you going without elaborate training, extra programmers, or premature commitments. We start by introducing model-based optimization, the key approach to streamlining the optimization modeling cycle and building successful applications today. Then we demonstrate how AMPL’s design of a language and system for model-based optimization is able to offer exceptional power of expression while maintaining ease of use. The remainder of the presentation takes a single example through successive stages of the optimization modeling life cycle: prototyping, integration, and deployment.
Model-Based Optimization for Effective and Reliable Decision-MakingBob Fourer
Optimization originated as an advanced mathematical technique, but it has become an accessible and widely used decision-making tool. A key factor in the spread of successful optimization applications has been the adoption of a model-based approach: A domain expert or operations analyst focuses on modeling the problem of interest, while the computation of a solution is left to general-purpose, off-the-shelf solvers; powerful yet intuitive modeling software manages the difficulties of translating between the human modeler’s formulation and the solver software’s needs. This talk introduces model-based optimization by contrasting it to a method-based approach that relies on customized implementation of rules and algorithms. Model-based implementations are illustrated using the AMPL modeling language and popular solvers. The presentation concludes by surveying the variety of modeling languages and solvers available for model-based optimization today.
AMPL Workshop, part 2: From Formulation to DeploymentBob Fourer
Optimization is the most widely adopted technology of Prescriptive Analytics, but also the most challenging to implement. In this two-part workshop, we show how AMPL gets you going without elaborate training, extra programmers, or premature commitments. We start by introducing model-based optimization, the key approach to streamlining the optimization
modeling cycle and building successful applications today. Then we demonstrate how AMPL’s design of a language and system for model-based optimization is able to offer exceptional power of expression while maintaining ease of use. The remainder of the presentation takes a single example through successive stages of the optimization modeling life cycle: prototyping, integration, and deployment.
As optimization methods have been applied more broadly and effectively, a key factor in their success has been the adoption of a model-based approach. A researcher or analyst focuses on modeling the problem of interest, while the computation of a solution is left to general-purpose, off-the-shelf solvers; independent modeling languages and systems manage the difficulties of translating between the human modeler’s ideas and the computer software’s needs. This tutorial introduces model-based optimization with examples from the AMPL modeling language and various popular solvers; the presentation concludes by surveying current software, with special attention to the role of constraint programming.
Developing Optimization Applications Quickly and Effectively with Algebraic M...Bob Fourer
Can you negotiate the complexities of the optimization modeling lifecycle, to deliver a working application before the funding runs out or the problem owner loses interest? Algebraic languages streamline the key steps of model formulation, testing, and revision, while offering powerful facilities for incorporating models into larger systems and deploying them to users. This presentation introduces algebraic modeling for optimization by carrying a single illustrative example through multiple contexts, from interactively evolved formula- tions to scripted iterative schemes to embedded applica- tions. We feature the new Python API for AMPL, and the QuanDec system for creating web-based collaborative applications from AMPL models.
Recent advances on large scheduling problems in CP OptimizerPhilippe Laborie
1) CP Optimizer is a modeling language that extends ILP and CP with interval variables and functions, allowing compact formulations for scheduling problems. It uses an automatic search algorithm that is complete, anytime, efficient, and scalable.
2) Performance improvements in the upcoming CP Optimizer 12.9 release allow it to solve classical RCPSP benchmarks quickly and address much larger instances with up to 500,000 tasks.
3) The automatic search employs techniques like failure-directed search, large neighborhood search, and iterative diving to cooperate and solve problems faster.
IRJET- Solving Supply Chain Network Design Problem using Center of Gravit...IRJET Journal
This document discusses using a gravity location model to solve a supply chain network design problem for pump manufacturing companies. Specifically, it summarizes using a gravity model to identify optimal geographic locations for new warehouses. The model considers transportation costs from suppliers and to markets to minimize total costs. The authors apply the model to collect geographic data from 4 pump companies and determine new warehouse locations for each that reduce transportation costs and times.
This document discusses segmenting stores into groups using multivariate analysis on store demographic and market share data. It covers:
1) The objective is to segment 2000 stores and interpret the segments, compute price elasticity for each, and discuss pricing strategies to maximize profits.
2) The approach involves understanding variable relationships, using these for segmentation, and starting with data summaries.
3) Dimension reduction techniques like principal component analysis and factor analysis are used to reduce highly correlated demographic variables into fewer underlying "factors" without much information loss.
4) Cluster analysis is then used to group similar stores based on factor scores and market shares, aiming to maximize between-cluster variance and minimize within-cluster variance.
As optimization (or prescriptive analytics) has grown as a tool for business decision-making, a key factor in its success has been the adoption of model-based optimization. Using this approach, an analyst’s major work is to describe a problem of interest by means of an algebraic model, while the computation of a solution is left to general-purpose, off-the-shelf software. Powerful modeling systems manage the difficulties of translating between the human modeler’s ideas and the computer software’s needs. This tutorial introduces model-based optimization and offers a guide to its effective use.
Model-Based Optimization for Effective and Reliable Decision-MakingBob Fourer
Optimization originated as an advanced mathematical technique, but it has become an accessible and widely used decision-making tool. A key factor in the spread of successful optimization applications has been the adoption of a model-based approach: A domain expert or operations analyst focuses on modeling the problem of interest, while the computation of a solution is left to general-purpose, off-the-shelf solvers; powerful yet intuitive modeling software manages the difficulties of translating between the human modeler’s formulation and the solver software’s needs. This talk introduces model-based optimization by contrasting it to a method-based approach that relies on customized implementation of rules and algorithms. Model-based implementations are illustrated using the AMPL modeling language and popular solvers. The presentation concludes by surveying the variety of modeling languages and solvers available for model-based optimization today.
AMPL Workshop, part 2: From Formulation to DeploymentBob Fourer
Optimization is the most widely adopted technology of Prescriptive Analytics, but also the most challenging to implement. In this two-part workshop, we show how AMPL gets you going without elaborate training, extra programmers, or premature commitments. We start by introducing model-based optimization, the key approach to streamlining the optimization
modeling cycle and building successful applications today. Then we demonstrate how AMPL’s design of a language and system for model-based optimization is able to offer exceptional power of expression while maintaining ease of use. The remainder of the presentation takes a single example through successive stages of the optimization modeling life cycle: prototyping, integration, and deployment.
As optimization methods have been applied more broadly and effectively, a key factor in their success has been the adoption of a model-based approach. A researcher or analyst focuses on modeling the problem of interest, while the computation of a solution is left to general-purpose, off-the-shelf solvers; independent modeling languages and systems manage the difficulties of translating between the human modeler’s ideas and the computer software’s needs. This tutorial introduces model-based optimization with examples from the AMPL modeling language and various popular solvers; the presentation concludes by surveying current software, with special attention to the role of constraint programming.
Developing Optimization Applications Quickly and Effectively with Algebraic M...Bob Fourer
Can you negotiate the complexities of the optimization modeling lifecycle, to deliver a working application before the funding runs out or the problem owner loses interest? Algebraic languages streamline the key steps of model formulation, testing, and revision, while offering powerful facilities for incorporating models into larger systems and deploying them to users. This presentation introduces algebraic modeling for optimization by carrying a single illustrative example through multiple contexts, from interactively evolved formula- tions to scripted iterative schemes to embedded applica- tions. We feature the new Python API for AMPL, and the QuanDec system for creating web-based collaborative applications from AMPL models.
Recent advances on large scheduling problems in CP OptimizerPhilippe Laborie
1) CP Optimizer is a modeling language that extends ILP and CP with interval variables and functions, allowing compact formulations for scheduling problems. It uses an automatic search algorithm that is complete, anytime, efficient, and scalable.
2) Performance improvements in the upcoming CP Optimizer 12.9 release allow it to solve classical RCPSP benchmarks quickly and address much larger instances with up to 500,000 tasks.
3) The automatic search employs techniques like failure-directed search, large neighborhood search, and iterative diving to cooperate and solve problems faster.
IRJET- Solving Supply Chain Network Design Problem using Center of Gravit...IRJET Journal
This document discusses using a gravity location model to solve a supply chain network design problem for pump manufacturing companies. Specifically, it summarizes using a gravity model to identify optimal geographic locations for new warehouses. The model considers transportation costs from suppliers and to markets to minimize total costs. The authors apply the model to collect geographic data from 4 pump companies and determine new warehouse locations for each that reduce transportation costs and times.
This document discusses segmenting stores into groups using multivariate analysis on store demographic and market share data. It covers:
1) The objective is to segment 2000 stores and interpret the segments, compute price elasticity for each, and discuss pricing strategies to maximize profits.
2) The approach involves understanding variable relationships, using these for segmentation, and starting with data summaries.
3) Dimension reduction techniques like principal component analysis and factor analysis are used to reduce highly correlated demographic variables into fewer underlying "factors" without much information loss.
4) Cluster analysis is then used to group similar stores based on factor scores and market shares, aiming to maximize between-cluster variance and minimize within-cluster variance.
As optimization (or prescriptive analytics) has grown as a tool for business decision-making, a key factor in its success has been the adoption of model-based optimization. Using this approach, an analyst’s major work is to describe a problem of interest by means of an algebraic model, while the computation of a solution is left to general-purpose, off-the-shelf software. Powerful modeling systems manage the difficulties of translating between the human modeler’s ideas and the computer software’s needs. This tutorial introduces model-based optimization and offers a guide to its effective use.
Machinelearning: The next step in manufacturing performance Blackberry&Cross
Minitab, nuestro aliado desde 2004, ha dado esta presentación sobre Aprendizaje Automático y aqui les compartimos el futuro.
Haga click, no tanto "code".
The winning team was Karan Sarao from India. They used a two-stage modeling process with five initial models that were then scored on a validation set and those scores incorporated into a final blended model. Key aspects of their solution included feature engineering by creating new derived variables and extensive model tuning using cross-validation. Their top-performing model was an extreme gradient boosted model.
This document provides an overview of a business analytics course from EduPristine. It defines business analytics as the application of computer technology, statistics, and domain knowledge to solve business problems. It discusses the different types of analytics including descriptive, inquisitive, predictive, and prescriptive. The course aims to equip professionals with tools and techniques to answer important business questions by exploring data patterns. Topics covered include linear regression, logistic regression, decision trees, clustering, and time series modeling. Case studies are used to apply analytic techniques to domains like insurance, banking, retail, and automotive.
Analytics is the application of computer technology ,statistics and domain knowledge to solve problems in business and industry ,to aid efficient and effective design making.
This document provides an overview of a business analytics course from EduPristine. It defines business analytics as the application of computer technology, statistics, and domain knowledge to solve business problems and make more informed decisions. The document outlines topics that will be covered in the course, including descriptive, predictive, and prescriptive analytics. It also lists common business domains and tools that analytics can be applied to, such as marketing, finance, and retail. The goal of the training is to equip professionals with the skills to explore data, identify patterns, predict relationships, and solve real-world business problems.
StratPro offers optimization solutions that evaluate all possible changes using a custom model to meet management needs for optimization. It works across industries with flexible solutions. StratPro uses the CPLEX library in Delphi on Windows. It has five elements to handle model creation and analysis: structure, viewing the model, data sorting, and consistency analysis. StratPro provides optimized solutions that maximize benefits within restrictions. It offers information on optimal quantities, resources, profits, goals, and contributions to help management make informed decisions.
The document discusses machine learning concepts and approaches for practical implementation in enterprises. It defines key terms like business analytics, predictive analytics, and machine learning. Business analytics answer questions about past data through queries, while predictive analytics uses algorithms to predict future probabilities and outcomes. The document also outlines challenges to enterprise adoption of machine learning and how vendors are helping to address skills gaps through cloud-based tools and services.
Better Living Through Analytics - Strategies for Data DecisionsProduct School
Data is king! Get ready to understand how a successful analytics team can empower managers from product, marketing, and other areas to make effective, data-driven decisions.
Louis Cialdella, a data scientist at ZipRecruiter, shared some case studies and successful strategies that he has used at ZipRecruiter as well as previous experiences. The purpose of this data talk was to enlighten people on how to make sure that analysts can successfully partner with other departments and get them the information they need to do great things.
AdaptiveGRC provides a unique GRC platform that supports multiple audit, compliance and risk processes at a competitive price. It was evaluated against competitors based on 8 criteria including ease of use, price, functionality, configurability, industry focus, customer service, integration capabilities, and company stability. Based on the evaluation, AdaptiveGRC generally scored well across most criteria, particularly for ease-of-use, price, functionality integration, and customer service. The document recommends AdaptiveGRC if these factors are important, and provides recommendations for other solutions based on specific criteria priorities.
The team proposed a solution to optimize salesperson activities for an online bank company using analytics. They recommended using clustering algorithms to identify high-value merchant leads and prioritize them. A route optimization algorithm would then generate optimized routes for salespeople to maximize coverage of priority leads. This could potentially increase profits up to 25% and reduce operating costs up to 17%. Visualization tools would provide an optimized call plan to deploy for a pilot test targeting selected merchant clusters.
Pricing Maturity Assessment, the results and finding per industry: Chemicals, Postal and Logistics, Telecommunications, HighTech, Automotive, LifeSciences, Machinery and Equipment, FMCG and Retail.
More on https://highlyscalable.wordpress.com/
Data Mining Problems in Retail is an analytical report that studies how retailers can make sense of their
data by adopting advanced data analysis and optimization techniques that enable automated decision
making in the area of marketing and pricing. The report analyzes dozens of practical case studies and
research reports and presents a systematic view on the problem.
We hope that this article will be useful for data scientists, marketing specialists, and business analysts
who are looking beyond the basic statistical and data mining techniques to build comprehensive
data-driven business optimization processes and solutions.
This document contains Sharvesh Gopal's resume. It summarizes his educational background with a Bachelor's degree in Mechanical Engineering from SSN College of Engineering from 2010 to 2014. It also lists his skills such as statistics, customer service, and programming languages. His experience includes strategic analysis projects in consumer electronics and global market sizing. Additional projects involve data analysis using Python and R as well as statistical analysis of FIFA player data and air pollution monitoring data.
This document provides an overview of a faculty development program on operations management focusing on supply chain analytics. It discusses the origin and evolution of supply chain management. It also defines different types of analytics including data analytics and big data analytics. The document outlines various decision science models such as quantitative, qualitative, simulation-based, and advanced models. It provides examples of applications of decision science in various organizations. Finally, it introduces linear programming techniques for prescriptive analytics and provides an example linear programming model to maximize profit from production.
This presentation discusses how prescriptive analytics can help energy traders, originators, portfolio managers and risk managers make faster, accurate decisions in the most complex scenarios
This document summarizes new features in IBM's predictive analytics software products SPSS and Decision Optimization. It discusses how these products empower all users to access more data sources and deploy analytics at scale both on-premises and in the cloud. New features like expanded open source capabilities and Python integration allow for more flexible and powerful predictive modeling. Case studies demonstrate how these advanced analytics solutions have provided significant value and returns for customers across industries.
This document provides an overview and summary of new features in IBM SPSS Predictive Analytics and IBM Decision Optimization. It discusses how predictive analytics can help organizations in various industries and functional areas. Key new features highlighted include empowering every user, unlocking more data faster, ground to cloud deployment options, optional coding and open source integration, and making predictive analytics accessible everywhere. The document demonstrates how these solutions have provided quantified benefits to customers.
This brief work is aimed in the direction of basics of data sciences and model building with focus on implementation on fairly sizable dataset. It focuses on cleaning the data, visualization, EDA, feature scaling, feature normalization, k-nearest neighbor, logistic regression, random forests, cross validation without delving too deep into any of them but giving a start to a new learner.
Recommender Systems from A to Z – Model TrainingCrossing Minds
This second meetup will be about training different models for our recommender system. We will review the simple models we can build as a baseline. After that, we will present the recommender system as an optimization problem and discuss different training losses. We will mention linear models and matrix factorization techniques. We will end the presentation with a simple introduction to non-linear models and deep learning.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
More Related Content
Similar to AMPL Workshop, part 1: Model-Based Optimization, Plain and Simple
Machinelearning: The next step in manufacturing performance Blackberry&Cross
Minitab, nuestro aliado desde 2004, ha dado esta presentación sobre Aprendizaje Automático y aqui les compartimos el futuro.
Haga click, no tanto "code".
The winning team was Karan Sarao from India. They used a two-stage modeling process with five initial models that were then scored on a validation set and those scores incorporated into a final blended model. Key aspects of their solution included feature engineering by creating new derived variables and extensive model tuning using cross-validation. Their top-performing model was an extreme gradient boosted model.
This document provides an overview of a business analytics course from EduPristine. It defines business analytics as the application of computer technology, statistics, and domain knowledge to solve business problems. It discusses the different types of analytics including descriptive, inquisitive, predictive, and prescriptive. The course aims to equip professionals with tools and techniques to answer important business questions by exploring data patterns. Topics covered include linear regression, logistic regression, decision trees, clustering, and time series modeling. Case studies are used to apply analytic techniques to domains like insurance, banking, retail, and automotive.
Analytics is the application of computer technology ,statistics and domain knowledge to solve problems in business and industry ,to aid efficient and effective design making.
This document provides an overview of a business analytics course from EduPristine. It defines business analytics as the application of computer technology, statistics, and domain knowledge to solve business problems and make more informed decisions. The document outlines topics that will be covered in the course, including descriptive, predictive, and prescriptive analytics. It also lists common business domains and tools that analytics can be applied to, such as marketing, finance, and retail. The goal of the training is to equip professionals with the skills to explore data, identify patterns, predict relationships, and solve real-world business problems.
StratPro offers optimization solutions that evaluate all possible changes using a custom model to meet management needs for optimization. It works across industries with flexible solutions. StratPro uses the CPLEX library in Delphi on Windows. It has five elements to handle model creation and analysis: structure, viewing the model, data sorting, and consistency analysis. StratPro provides optimized solutions that maximize benefits within restrictions. It offers information on optimal quantities, resources, profits, goals, and contributions to help management make informed decisions.
The document discusses machine learning concepts and approaches for practical implementation in enterprises. It defines key terms like business analytics, predictive analytics, and machine learning. Business analytics answer questions about past data through queries, while predictive analytics uses algorithms to predict future probabilities and outcomes. The document also outlines challenges to enterprise adoption of machine learning and how vendors are helping to address skills gaps through cloud-based tools and services.
Better Living Through Analytics - Strategies for Data DecisionsProduct School
Data is king! Get ready to understand how a successful analytics team can empower managers from product, marketing, and other areas to make effective, data-driven decisions.
Louis Cialdella, a data scientist at ZipRecruiter, shared some case studies and successful strategies that he has used at ZipRecruiter as well as previous experiences. The purpose of this data talk was to enlighten people on how to make sure that analysts can successfully partner with other departments and get them the information they need to do great things.
AdaptiveGRC provides a unique GRC platform that supports multiple audit, compliance and risk processes at a competitive price. It was evaluated against competitors based on 8 criteria including ease of use, price, functionality, configurability, industry focus, customer service, integration capabilities, and company stability. Based on the evaluation, AdaptiveGRC generally scored well across most criteria, particularly for ease-of-use, price, functionality integration, and customer service. The document recommends AdaptiveGRC if these factors are important, and provides recommendations for other solutions based on specific criteria priorities.
The team proposed a solution to optimize salesperson activities for an online bank company using analytics. They recommended using clustering algorithms to identify high-value merchant leads and prioritize them. A route optimization algorithm would then generate optimized routes for salespeople to maximize coverage of priority leads. This could potentially increase profits up to 25% and reduce operating costs up to 17%. Visualization tools would provide an optimized call plan to deploy for a pilot test targeting selected merchant clusters.
Pricing Maturity Assessment, the results and finding per industry: Chemicals, Postal and Logistics, Telecommunications, HighTech, Automotive, LifeSciences, Machinery and Equipment, FMCG and Retail.
More on https://highlyscalable.wordpress.com/
Data Mining Problems in Retail is an analytical report that studies how retailers can make sense of their
data by adopting advanced data analysis and optimization techniques that enable automated decision
making in the area of marketing and pricing. The report analyzes dozens of practical case studies and
research reports and presents a systematic view on the problem.
We hope that this article will be useful for data scientists, marketing specialists, and business analysts
who are looking beyond the basic statistical and data mining techniques to build comprehensive
data-driven business optimization processes and solutions.
This document contains Sharvesh Gopal's resume. It summarizes his educational background with a Bachelor's degree in Mechanical Engineering from SSN College of Engineering from 2010 to 2014. It also lists his skills such as statistics, customer service, and programming languages. His experience includes strategic analysis projects in consumer electronics and global market sizing. Additional projects involve data analysis using Python and R as well as statistical analysis of FIFA player data and air pollution monitoring data.
This document provides an overview of a faculty development program on operations management focusing on supply chain analytics. It discusses the origin and evolution of supply chain management. It also defines different types of analytics including data analytics and big data analytics. The document outlines various decision science models such as quantitative, qualitative, simulation-based, and advanced models. It provides examples of applications of decision science in various organizations. Finally, it introduces linear programming techniques for prescriptive analytics and provides an example linear programming model to maximize profit from production.
This presentation discusses how prescriptive analytics can help energy traders, originators, portfolio managers and risk managers make faster, accurate decisions in the most complex scenarios
This document summarizes new features in IBM's predictive analytics software products SPSS and Decision Optimization. It discusses how these products empower all users to access more data sources and deploy analytics at scale both on-premises and in the cloud. New features like expanded open source capabilities and Python integration allow for more flexible and powerful predictive modeling. Case studies demonstrate how these advanced analytics solutions have provided significant value and returns for customers across industries.
This document provides an overview and summary of new features in IBM SPSS Predictive Analytics and IBM Decision Optimization. It discusses how predictive analytics can help organizations in various industries and functional areas. Key new features highlighted include empowering every user, unlocking more data faster, ground to cloud deployment options, optional coding and open source integration, and making predictive analytics accessible everywhere. The document demonstrates how these solutions have provided quantified benefits to customers.
This brief work is aimed in the direction of basics of data sciences and model building with focus on implementation on fairly sizable dataset. It focuses on cleaning the data, visualization, EDA, feature scaling, feature normalization, k-nearest neighbor, logistic regression, random forests, cross validation without delving too deep into any of them but giving a start to a new learner.
Recommender Systems from A to Z – Model TrainingCrossing Minds
This second meetup will be about training different models for our recommender system. We will review the simple models we can build as a baseline. After that, we will present the recommender system as an optimization problem and discuss different training losses. We will mention linear models and matrix factorization techniques. We will end the presentation with a simple introduction to non-linear models and deep learning.
Similar to AMPL Workshop, part 1: Model-Based Optimization, Plain and Simple (20)
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
AMPL Workshop, part 1: Model-Based Optimization, Plain and Simple
1. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
1
Model-Based Optimization
AMPL Optimization Inc.
www.ampl.com — +1 773-336-AMPL
INFORMS Business Analytics Conference
Technology Workshop, 15 April 2018
From Formulation to Deployment with AMPL
2. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based Optimization, Plain and Simple:
From Formulation to Deployment with AMPL
Optimization is the most widely adopted
technology of Prescriptive Analytics, but
also the most challenging to implement:
• How can you prototype an optimization
application fast enough to get results
before the problem owner loses interest?
• How can you integrate optimization into
your enterprise’s decision-making
systems?
• How can you deploy optimization models
to support analysis and action throughout
your organization?
In this presentation, we show how AMPL
gets you going without elaborate training,
extra programmers, or premature
commitments. We start by introducing
model-based optimization, the key
approach to streamlining the optimization
modeling cycle and building successful
applications today. Then we demonstrate
how AMPL’s design of a language and
system for model-based optimization is able
to offer exceptional power of expression
while maintaining ease of use.
The remainder of the presentation takes a
single example through successive stages of
the optimization modeling lifecycle:
• Prototyping in an interactive command
environment.
• Integration via AMPL scripts and
through APIs to all popular programming
languages.
• Deployment with QuanDec, which turns
an AMPL model into an interactive,
collaborative decision-making tool.
Our example is simple enough for
participants to follow its development
through the course of this short workshop,
yet rich enough to serve as a foundation for
appreciating model-based optimization in
practice.
3. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Part 1. Model-based optimization, plain and simple
https://ampl.com/MEETINGS/TALKS/2018_04_Baltimore_Workshop1.pdf
Comparison of method-based and model-based approaches
Modeling languages for optimization
Algebraic modeling languages: AMPL
Solvers for broad model classes
Part 2. From formulation to deployment with AMPL
https://ampl.com/MEETINGS/TALKS/2018_04_Baltimore_Workshop2.pdf
Building models: AMPL’s interactive environment
Developing applications: AMPL scripts
Extending script applications with Python: pyMPL
Embedding into applications: AMPL APIs
Creating an interactive decision-making tool: QuanDec
3
Outline
4. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
4
Part 1
Model-Based Optimization,
Plain and Simple
7. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
In general terms,
Given a function of some decision variables
Choose values of the variables to
make the function as large or as small as possible
Subject to restrictions on the values of the variables
In practice,
A paradigm for a very broad variety of problems
A successful approach for finding solutions
7
Mathematical Optimization
8. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Example: Balanced Assignment
Motivation
meeting of employees from around the world
Given
several employee categories
(title, location, department, male/female)
a specified number of project groups
Assign
each employee to a project group
So that
the groups have about the same size
the groups are as “diverse” as possible
9. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Method-Based Approach
Define an algorithm to build a balanced assignment
Start with all groups empty
Make a list of people (employees)
For each person in the list:
Add to the group whose resulting “sameness” will be least
Balanced Assignment
Initialize all groups G = { }
Repeat for each person p
sMin = Infinity
Repeat for each group G
s = total "sameness" in G ∪ {p}
if s < sMin then
sMin = s
GMin = G
Assign person p to group GMin
10. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Method-Based Approach (cont’d)
Define a computable concept of “sameness”
Sameness of a pair of people:
Number of categories in which they are the same
Sameness in a group:
Sum of the sameness of all pairs of people in the group
Refine the algorithm to get better results
Order the list of people
Locally improve the initial “greedy” solution
by swapping group members
Seek further improvement through
local search metaheuristics
Balanced Assignment
11. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based Approach
Formulate a “minimal sameness” model
Define decision variables for assignment of people to groups
Specify valid assignments through constraints on the variables
Formulate sameness as an objective to be minimized
Total sameness = sum of the sameness of all groups
Send to an off-the-shelf solver
Choice of many for common problem types
Balanced Assignment
12. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based Formulation
Given
set of people
set of categories of people
type of person within category , for all ∈ , ∈
and
number of groups
lower limit on people in a group
upper limit on people in a group
Define
| ∈ : |, for all ∈ , ∈
sameness of persons and
Balanced Assignment
13. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Determine
∈ 0,1 1 if person is assigned to group
0 otherwise, for all ∈ , 1, . . . ,
To minimize
∑ ∑ ∑∈∈
total sameness of all pairs of people in all groups
Subject to
∑ 1, for each ∈
each person must be assigned to one group
∑ ∈ , for each 1, . . . ,
each group must be assigned an acceptable number of people
13
Model-Based Formulation (cont’d)
14. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based Solution
Optimize with an off-the-shelf solver
Choose among many alternatives
Linearize and send to a mixed-integer linear solver
CPLEX, Gurobi, Xpress; CBC
Send quadratic formulation to a mixed-integer solver
that automatically linearizes products involving binary variables
CPLEX, Gurobi, Xpress
Send quadratic formulation to a nonlinear solver
Mixed-integer nonlinear: Knitro, BARON
Continuous nonlinear (might come out integer): MINOS, Ipopt, . . .
Balanced Assignment
15. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Where Is the Work?
Method-based
Programming an implementation of the method
Model-based
Constructing a formulation of the model
16. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Complications in Balanced Assignment
“Total Sameness” is problematical
Hard for client to relate to goal of diversity
Minimize “total variation” instead
Sum over all types: most minus least assigned to any group
Client has special requirements
No employee should be “isolated” within their group
No group can have exactly one woman
Every person must have a group-mate
from the same location and of equal or adjacent rank
Room capacities are variable
Different groups have different size limits
Minimize “total deviation”
Sum over all types: greatest violation of target range for any group
17. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Method-Based (cont’d)
Revise or replace the initial solution method
Total variation is less suitable to a greedy algorithm
Re-think improvement procedures
Total variation is harder to locally improve
Client constraints are challenging to enforce
Revise or re-implement the method
Even small changes to the problem
can necessitate major changes to the method
Balanced Assignment
18. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Add variables
fewest people of category , type in any group,
most people of category , type in any group,
for each ∈ , ∈ ⋃ ∈
Add defining constraints
∑ ∈ : , for each 1, . . . , ; ∈ , ∈
∑ ∈ : , for each 1, . . . , ; ∈ , ∈
Minimize total variation
∑ ∑ ∈∈ )
. . . generalizes to handle varying group sizes
18
Model-Based (cont’d)
Balanced Assignment
19. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based (cont’d)
To express client requirement for women in a group, let
∈ : , ⁄ female
Add constraints
∑ 0∈ or ∑ 2∈ , for each 1, . . . ,
Balanced Assignment
20. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based (cont’d)
To express client requirement for women in a group, let
∈ : , ⁄ female
Define logic variables
∈ 0,1 1 if any women assigned to group
0 otherwise, for all 1, . . . ,
Add constraints relating logic to assignment variables
2 ∑ ∈ , for each 1, . . . ,
Balanced Assignment
21. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based (cont’d)
To express client requirements for group-mates, let
∈ : , , , , for all ∈ , ∈
⊆ ranks adjacent to rank , for all ∈
Add constraints
∑ 0∈ or ∑ ∑ ∑ 2∈∈∈ ,
for each ∈ , ∈ , 1, . . . ,
Balanced Assignment
22. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Model-Based (cont’d)
To express client requirements for group-mates, let
∈ : , , , , for all ∈ , ∈
⊆ ranks adjacent to rank , for all ∈
Define logic variables
∈ 0,1 1 if group has anyone from location of rank
0 otherwise, for all ∈ , ∈ , 1, . . . ,
Add constraints relating logic to assignment variables
∑ ∈ ,
∑ ∑ ∑ 2∈∈∈ ,
for each ∈ , ∈ , 1, . . . ,
Balanced Assignment
23. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Problems that are challenging to formulate
Certain types of complex logic
Sequencing, scheduling
Black-box functions
Very large, specialized problems embedded in apps
Routing delivery trucks nationwide
Finding shortest routes in mapping apps
Metaheuristic frameworks
Evolutionary methods, simulated annealing, . . .
Applications associated with computer science
Constraint programming
Training deep neural networks
23
Method-Based Remains Attractive for . . .
24. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Diverse application areas
Operations research & management science
Business analytics
Engineering & science
Economics & finance
Diverse kinds of users
Anyone who took an “optimization” class
Anyone else with a technical background
Newcomers to optimization
. . . and trends favor this direction
Steadily faster and more powerful off-the-shelf solvers
Expanding options to incorporate models within hybrid schemes
24
Model-Based Has Become Standard for . . .
25. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Background
The modeling lifecycle
Matrix generators
Modeling languages
Algebraic modeling languages
Design & marketing approaches
AMPL: General-purpose, solver-independent
Balanced assignment model in AMPL
Formulation
Solution
Solvers
25
Software for Model-Based Optimization
26. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
26
The Optimization Modeling Lifecycle
Communicate with Client
Build Model
Generate Optimization Problem
Submit Problem to Solver
Report & Analyze Results
Prepare Data
27. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Goals for optimization software
Repeat the cycle quickly and reliably
Get results before client loses interest
Deploy for application
Complication: two forms of an optimization problem
Modeler’s form
Mathematical description, easy for people to work with
Solver’s form
Explicit data structure, easy for solvers to compute with
Challenge: translate between these two forms
27
Managing the Modeling Lifecycle
28. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Write a program
Read data and compute objective & constraint coefficients
Communicate with the solver via its API
Convert the solver’s solution for viewing or processing
Some attractions
Ease of embedding into larger systems
Access to advanced solver features
Serious disadvantages
Difficult environment for modeling
program does not resemble the modeler’s form
model is not separate from data
Very slow modeling cycle
hard to check the program for correctness
hard to distinguish modeling from programming errors
28
Matrix Generators
29. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Over the past seven years we have perceived that the size distribution of
general structure LP problems being run on commercial LP codes has
remained about stable. . . . A 3000 constraint LP model is still considered
large and very few LP problems larger than 6000 rows are being solved on a
production basis. . . . That this distribution has not noticeably changed
despite a massive change in solution economics is unexpected.
We do not feel that the linear programming user’s most pressing need over the
next few years is for a new optimizer that runs twice as fast on a machine that
costs half as much (although this will probably happen). Cost of optimization
is just not the dominant barrier to LP model implementation. The process
required to manage the data, formulate and build the model, report on and
analyze the results costs far more, and is much more of a barrier to effective
use of LP, than the cost/performance of the optimizer.
Why aren’t more larger models being run? It is not because they could not be
useful; it is because we are not successful in using them. . . . They become
unmanageable. LP technology has reached the point where anything that can
be formulated and understood can be optimized at a relatively modest cost.
C.B. Krabek, R.J. Sjoquist and D.C. Sommer, The APEX Systems: Past and Future.
SIGMAP Bulletin 29 (April 1980) 3–23.
30. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Describe your model
Write your symbolic model in a
computer-readable modeler’s form
Prepare data for the model
Let computer translate to & from the solver’s form
Limited drawbacks
Need to learn a new language
Incur overhead in translation
Make formulations clearer and hence easier to steal?
Great advantages
Faster modeling cycles
More reliable modeling
More maintainable applications
30
Modeling Languages
31. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
The aim of this system is to provide one representation of a model which is
easily understood by both humans and machines. . . . With such a notation,
the information content of the model representation is such that a machine
can not only check for algebraic correctness and completeness, but also
interface automatically with solution algorithms and report writers.
. . . a significant portion of total resources in a modeling exercise . . . is spent
on the generation, manipulation and reporting of models. It is evident that
this must be reduced greatly if models are to become effective tools in
planning and decision making.
The heart of it all is the fact that solution algorithms need a data structure
which, for all practical purposes, is impossible to comprehend by humans,
while, at the same time, meaningful problem representations for humans are
not acceptable to machines. We feel that the two translation processes
required (to and from the machine) can be identified as the main source of
difficulties and errors. GAMS is a system that is designed to eliminate these
two translation processes, thereby lifting a technical barrier to effective
modeling . . .
J. Bisschop and A. Meeraus, On the Development of a General Algebraic Modeling System
in a Strategic Planning Environment. Mathematical Programming Study 20 (1982) 1–29.
32. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
These two forms of a linear program — the modeler’s form and the
algorithm’s form — are not much alike, and yet neither can be done
without. Thus any application of linear optimization involves translating
the one form to the other. This process of translation has long been
recognized as a difficult and expensive task of practical linear
programming.
In the traditional approach to translation, the work is divided between
modeler and machine. . . .
There is also a quite different approach to translation, in which as much
work as possible is left to the machine. The central feature of this
alternative approach is a modeling language that is written by the modeler
and translated by the computer. A modeling language is not a
programming language; rather, it is a declarative language that expresses
the modeler’s form of a linear program in a notation that a computer
system can interpret.
R. Fourer, Modeling Languages Versus Matrix Generators for Linear Programming.
ACM Transactions on Mathematical Software 9 (1983) 143–183.
33. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Algebraic formulation
Define data in terms of sets & parameters
Analogous to database keys & records
Define decision variables
Minimize or maximize a function of decision variables
Subject to equations or inequalities
that constrain the values of the variables
Advantages
Familiar
Powerful
Proven
33
Algebraic Modeling Languages
34. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Design approaches
Executable: object libraries for programming languages
Declarative: specialized optimization languages
Marketing approaches
Solver-independent vs. solver-specific
Licensed vs. open-source
34
Algebraic Modeling Languages
35. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Concept
Create an algebraic modeling language
inside a general-purpose programming language
Redefine operators like + and <=
to return constraint objects rather than simple values
Advantages
Ready integration with applications
Good access to advanced solver features
Disadvantages
Programming issues complicate description of the model
Modeling and programming bugs are hard to separate
Efficiency issues are more of a concern
35
Executable
Algebraic Modeling Languages
36. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
36
Executable
Examples (Gurobi/Python)
model.addConstrs(x[i] + x[j] <= 1
for i in range(5) for j in range(5))
for i,j in arcs:
m.addConstr(gurobipy.quicksum(flow[h,i,j] for h in commodities)
<= capacity[i,j], 'cap_%s_%s' % (i, j))
Algebraic Modeling Languages
38. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Concept
Design a language specifically for optimization modeling
Resembles mathematical notation as much as possible
Extend to command scripts and database links
Connect to external applications via APIs
Disadvantages
Adds a system between application and solver
Does not have a full object-oriented programming framework
Advantages
Streamlines model development
Promotes validation and maintenance of models
Works with many popular programming languages
38
Declarative
Algebraic Modeling Languages
39. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Solver-specific
OPL for CPLEX (IBM)
MOSEL* for Xpress (FICO)
OPTMODEL for SAS/OR (SAS)
Solver-independent
Open-source: CMPL, Gnu MathProg
Licensed: AIMMS, AMPL, GAMS, MPL
39
Declarative
Algebraic Modeling Languages
40. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Many enhancements and extensions
Interactive development environments
Generalized constraint forms
Variety of data sources
spreadsheets, relational databases
Programming features
loops, tests, assignments
Extensions for deployment
APIs for embedding models in applications
Tools for building applications around models
40
Declarative
Algebraic Modeling Languages
41. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
Features
Algebraic modeling language
Built specially for optimization
Designed to support many solvers
Design goals
Powerful, general expressions
Natural, easy-to-learn modeling principles
Efficient processing that scales well with problem size
41
42. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
42
Modeling Language Formulation
Sets, parameters, variables (for people)
set PEOPLE; # individuals to be assigned
set CATEG;
param type {PEOPLE,CATEG} symbolic;
# categories by which people are classified;
# type of each person in each category
param numberGrps integer > 0;
param minInGrp integer > 0;
param maxInGrp integer >= minInGrp;
# number of groups; bounds on size of groups
var Assign {i in PEOPLE, j in 1..numberGrps} binary;
# Assign[i,j] is 1 if and only if
# person i is assigned to group j
43. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
43
Modeling Language Formulation
Variables, constraints (for variation)
set TYPES {k in CATEG} := setof {i in PEOPLE} type[i,k];
# all types found in each category
var MinType {k in CATEG, TYPES[k]};
var MaxType {k in CATEG, TYPES[k]};
# fewest and most people of each type, over all groups
subj to MinTypeDefn {j in 1..numberGrps, k in CATEG, l in TYPES[k]}:
MinType[k,l] <= sum {i in PEOPLE: type[i,k] = l} Assign[i,j];
subj to MaxTypeDefn {j in 1..numberGrps, k in CATEG, l in TYPES[k]}:
MaxType[k,l] >= sum {i in PEOPLE: type[i,k] = l} Assign[i,j];
# values of MinTypeDefn and MaxTypeDefn variables
# must be consistent with values of Assign variables
∑ ∈ : , for each 1, . . . , ; ∈ , ∈
44. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
44
Modeling Language Formulation
Objective, constraints (for assignment)
minimize TotalVariation:
sum {k in CATEG, l in TYPES[k]} (MaxType[k,l] - MinType[k,l]);
# Total variation over all types
subj to AssignAll {i in PEOPLE}:
sum {j in 1..numberGrps} Assign[i,j] = 1;
# Each person must be assigned to one group
subj to GroupSize {j in 1..numberGrps}:
minInGrp <= sum {i in PEOPLE} Assign[i,j] <= maxInGrp;
# Each group must have an acceptable size
∑ ∈ , for each 1, . . . ,
45. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
45
Modeling Language Data
210 people
set PEOPLE :=
BIW AJH FWI IGN KWR KKI HMN SML RSR TBR
KRS CAE MPO CAR PSL BCG DJA AJT JPY HWG
TLR MRL JDS JAE TEN MKA NMA PAS DLD SCG
VAA FTR GCY OGZ SME KKA MMY API ASA JLN
JRT SJO WMS RLN WLB SGA MRE SDN HAN JSG
AMR DHY JMS AGI RHE BLE SMA BAN JAP HER
MES DHE SWS ACI RJY TWD MMA JJR MFR LHS
JAD CWU PMY CAH SJH EGR JMQ GGH MMH JWR
MJR EAZ WAD LVN DHR ABE LSR MBT AJU SAS
JRS RFS TAR DLT HJO SCR CMY GDE MSL CGS
HCN JWS RPR RCR RLS DSF MNA MSR PSY MET
DAN RVY PWS CTS KLN RDN ANV LMN FSM KWN
CWT PMO EJD AJS SBK JWB SNN PST PSZ AWN
DCN RGR CPR NHI HKA VMA DMN KRA CSN HRR
SWR LLR AVI RHA KWY MLE FJL ESO TJY WHF
TBG FEE MTH RMN WFS CEH SOL ASO MDI RGE
LVO ADS CGH RHD MBM MRH RGF PSA TTI HMG
ECA CFS MKN SBM RCG JMA EGL UJT ETN GWZ
MAI DBN HFE PSO APT JMT RJE MRZ MRK XYF
JCO PSN SCS RDL TMN CGY GMR SER RMS JEN
DWO REN DGR DET FJT RJZ MBY RSN REZ BLW ;
46. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
46
Modeling Language Data
4 categories, 18 types
set CATEG := dept loc rate title ;
param type:
dept loc rate title :=
BIW NNE Peoria A Assistant
KRS WSW Springfield B Assistant
TLR NNW Peoria B Adjunct
VAA NNW Peoria A Deputy
JRT NNE Springfield A Deputy
AMR SSE Peoria A Deputy
MES NNE Peoria A Consultant
JAD NNE Peoria A Adjunct
MJR NNE Springfield A Assistant
JRS NNE Springfield A Assistant
HCN SSE Peoria A Deputy
DAN NNE Springfield A Adjunct
.......
param numberGrps := 12 ;
param minInGrp := 16 ;
param maxInGrp := 19 ;
47. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
47
Modeling Language Solution
Model + data = problem instance to be solved (CPLEX)
ampl: model BalAssign.mod;
ampl: data BalAssign.dat;
ampl: option solver cplex;
ampl: option show_stats 1;
ampl: solve;
2556 variables:
2520 binary variables
36 linear variables
654 constraints, all linear; 25632 nonzeros
210 equality constraints
432 inequality constraints
12 range constraints
1 linear objective; 36 nonzeros.
CPLEX 12.8.0.0: optimal integer solution; objective 16
59597 MIP simplex iterations
387 branch-and-bound nodes 8.063 sec
48. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
48
Modeling Language Solution
Model + data = problem instance to be solved (Gurobi)
ampl: model BalAssign.mod;
ampl: data BalAssign.dat;
ampl: option solver gurobi;
ampl: option show_stats 1;
ampl: solve;
2556 variables:
2520 binary variables
36 linear variables
654 constraints, all linear; 25632 nonzeros
210 equality constraints
432 inequality constraints
12 range constraints
1 linear objective; 36 nonzeros.
Gurobi 7.5.0: optimal solution; objective 16
338028 simplex iterations
1751 branch-and-cut nodes 66.344 sec
49. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
49
Modeling Language Solution
Model + data = problem instance to be solved (Xpress)
ampl: model BalAssign.mod;
ampl: data BalAssign.dat;
ampl: option solver xpress;
ampl: option show_stats 1;
ampl: solve;
2556 variables:
2520 binary variables
36 linear variables
654 constraints, all linear; 25632 nonzeros
210 equality constraints
432 inequality constraints
12 range constraints
1 linear objective; 36 nonzeros.
XPRESS 8.4(32.01.08): Global search complete
Best integer solution found 16
6447 branch and bound nodes 61.125 sec
50. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
50
Modeling Language Formulation (revised)
Add bounds on variables
var MinType {k in CATEG, t in TYPES[k]}
<= floor (card {i in PEOPLE: type[i,k] = t} / numberGrps);
var MaxType {k in CATEG, t in TYPES[k]
>= ceil (card {i in PEOPLE: type[i,k] = t} / numberGrps);
ampl: include BalAssign+.run
Presolve eliminates 72 constraints.
...
Gurobi 7.5.0: optimal solution; objective 16
2203 simplex iterations 0.203 sec
51. Model-Based Optimization, Plain and Simple
INFORMS Analytics 2018 — 15 April 2018
CPLEX, Gurobi, Xpress; CBC, MOSEK
Linear
Continuous variables
Primal simplex, dual simplex, interior-point
Integer (including zero-one) variables
Branch-and-bound + feasibility heuristics + cut generation
Automatic transformations to integer:
piecewise-linear, discrete variable domains, indicator constraints
Quadratic extensions
Convex elliptic objectives and constraints
Convex conic constraints
Variable binary in objective
Transformed to linear (or to convex if binary binary)
Nonconvex (CPLEX)
52
Solvers: “Linear”