The document discusses research on learning to improve the efficiency of machine learning algorithms through speedup learning. It provides three key points:
1) Early work on explanation-based learning for speedup had limited success, but techniques like memoization and clause learning led to major improvements in SAT solvers.
2) More recent approaches use predictive models trained on dynamic features to learn optimal policies for controlling search algorithms, like setting noise levels or restart policies.
3) Open problems remain in developing optimal predictive policies with partial information and approximations, to continue improving search and reasoning performance.
Bridging the Gap: Machine Learning for Ubiquitous Computing -- EvaluationThomas Ploetz
Tutorial @Ubicomp 2015: Bridging the Gap -- Machine Learning for Ubiquitous Computing (evaluation session).
A tutorial on promises and pitfalls of Machine Learning for Ubicomp (and Human Computer Interaction). From Practitioners for Practitioners.
Presenter: Nils Hammerla <n.hammerla@gmail.com>
video recording of talks as they wer held at Ubicomp:
https://youtu.be/LgnnlqOIXJc?list=PLh96aGaacSgXw0MyktFqmgijLHN-aQvdq
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Machine Learning and Data Mining: 14 Evaluation and CredibilityPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. In this lecture we discuss the evaluation of classification algorithms.
Bridging the Gap: Machine Learning for Ubiquitous Computing -- EvaluationThomas Ploetz
Tutorial @Ubicomp 2015: Bridging the Gap -- Machine Learning for Ubiquitous Computing (evaluation session).
A tutorial on promises and pitfalls of Machine Learning for Ubicomp (and Human Computer Interaction). From Practitioners for Practitioners.
Presenter: Nils Hammerla <n.hammerla@gmail.com>
video recording of talks as they wer held at Ubicomp:
https://youtu.be/LgnnlqOIXJc?list=PLh96aGaacSgXw0MyktFqmgijLHN-aQvdq
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Machine Learning and Data Mining: 14 Evaluation and CredibilityPier Luca Lanzi
Course "Machine Learning and Data Mining" for the degree of Computer Engineering at the Politecnico di Milano. In this lecture we discuss the evaluation of classification algorithms.
Online Coreset Selection for Rehearsal-based Continual LearningMLAI2
A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a target dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.
Heuristic design of experiments w meta gradient searchGreg Makowski
Once you have started learning about predictive algorithms, and the basic knowledge discovery in databases process, what is the next level of detail to learn for a consulting project?
* Give examples of the many model training parameters
* Track results in a "model notebook"
* Use a model metric that combines both accuracy and generalization to rank models
* How to strategically search over the model training parameters - use a gradient descent approach
* One way to describe an arbitrarily complex predictive system is by using sensitivity analysis
K-Nearest neighbor is one of the most commonly used classifier based in lazy learning. It is one of the most commonly used methods in recommendation systems and document similarity measures. It mainly uses Euclidean distance to find the similarity measures between two data points.
Talk of Ali Mousavi "Event-Modelling An Engineering Solution for Control and Analysis of Complex Systems" at 116th regular meeting of INCOSE Russian chapter, 14-Sep-2016
AI Data Summit 2019 Thompson Sampling - Thompson Sampling Tutorial “The redis...Hanan Shteingart
AI Data Summit 2019 Thompson Sampling - Thompson Sampling Tutorial “The rediscovery of a swiss army knife”
aka “Squeezing a 96-page review into a 25 min talk”
Based on Russo, D. J., Van Roy, B., Kazerouni, A., Osband, I., & Wen, Z. (2018). A tutorial on Thompson sampling. Foundations and Trends® inMachine Learning, 11(1), 1-96.
Online Coreset Selection for Rehearsal-based Continual LearningMLAI2
A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a target dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.
Heuristic design of experiments w meta gradient searchGreg Makowski
Once you have started learning about predictive algorithms, and the basic knowledge discovery in databases process, what is the next level of detail to learn for a consulting project?
* Give examples of the many model training parameters
* Track results in a "model notebook"
* Use a model metric that combines both accuracy and generalization to rank models
* How to strategically search over the model training parameters - use a gradient descent approach
* One way to describe an arbitrarily complex predictive system is by using sensitivity analysis
K-Nearest neighbor is one of the most commonly used classifier based in lazy learning. It is one of the most commonly used methods in recommendation systems and document similarity measures. It mainly uses Euclidean distance to find the similarity measures between two data points.
Talk of Ali Mousavi "Event-Modelling An Engineering Solution for Control and Analysis of Complex Systems" at 116th regular meeting of INCOSE Russian chapter, 14-Sep-2016
AI Data Summit 2019 Thompson Sampling - Thompson Sampling Tutorial “The redis...Hanan Shteingart
AI Data Summit 2019 Thompson Sampling - Thompson Sampling Tutorial “The rediscovery of a swiss army knife”
aka “Squeezing a 96-page review into a 25 min talk”
Based on Russo, D. J., Van Roy, B., Kazerouni, A., Osband, I., & Wen, Z. (2018). A tutorial on Thompson sampling. Foundations and Trends® inMachine Learning, 11(1), 1-96.
1. Learning to Search Henry Kautz University of Washington joint work with Dimitri Achlioptas, Carla Gomes, Eric Horvitz, Don Patterson, Yongshao Ruan, Bart Selman CORE – MSR, Cornell, UW
2.
3.
4.
5.
6.
7. Big Picture Problem Instances Solver static features runtime Learning / Analysis Predictive Model dynamic features resource allocation / reformulation control / policy
8. Case Study 1: Beyond 4.25 Problem Instances Solver static features runtime Learning / Analysis Predictive Model
9.
10.
11. Phase Transition Almost all unsolvable area Fraction of pre-assignment Fraction of unsolvable cases Almost all solvable area Phase transition Underconstrained area Critically constrained area Overconstrained area Complexity Graph 42% 50% 20% 42% 50% 20%
12. Easy-Hard-Easy pattern in local search % holes Computational Cost “ Over” constrained area Underconstrained area Walksat Order 30, 33, 36
13.
14. Deep structural features Hardness is also controlled by structure of constraints, not just the fraction of holes Rectangular Pattern Aligned Pattern Balanced Pattern Tractable Very hard
41. Case Study 3: Restart Policies Problem Instances Solver static features runtime Learning / Analysis Predictive Model dynamic features resource allocation / reformulation control / policy
Stochastic solver Sound, but not complete Governed by two key parameters “ Max Flips” “ Noise” Different variations of Walksat apply different heuristics in place of the red text This is the original variation sometimes called Walksat-SKC
Per cent chance of Walksat finding a solution in 100000 flips
Contribution: “ to turn the observation of the relationship of the invariant ratio into an effective procedure for estimating optimal noise level”
20 20 Incomplete nature of local search procedures: they can show consistency of a set of constraints and find a solution or model that satisfies those constraints but they cannot prove inconsistency, i.e., they cannot prove that a solution satisfying those constraints does not exist.
31
Choice points are states in search procedures where the algorithm assigns value to variables where that assignment is not forced via propagation of previous set values, as occurs with unit propagation, backracking,, lookahad, or forward checking. These are points at which an assignment is chosen heuristically per the policies implemented in the problem solver.
Choice points are states in search procedures where the algorithm assigns value to variables where that assignment is not forced via propagation of previous set values, as occurs with unit propagation, backtracking,, lookahead, or forward checking. These are points at which an assignment is chosen heuristically per the policies implemented in the problem solver.
Even better restart policies should be achievable by considering a range of different statistical properties of the search space.