The document discusses hyperparameter tuning in machine learning, specifically using Azure Machine Learning to optimize model performance through multiple training runs with varying hyperparameters. It explains defining search spaces for discrete and continuous hyperparameters, and emphasizes the importance of configuring sampling and early termination policies to improve efficiency during the tuning process. It also provides links and references for further information.

1. Training Optimal
Models

2. Eng Teong Cheah
Microsoft MVP

3. Hyperparameter Tuning
Hyperparameter tuning is accomplished by training the multiple models, using the same
algorithm and training data but different hyperparameter values. The resulting model
from each training run is then evaluated to determine the performance metric for which
you want to optimize (for example, accuracy), and the best-performing model is selected.
In Azure Machine Learning, you achieve this through an experiment that consists of
a hyperdrive run, which initiates a child run for each hyperparameter combination to be
tested. Each child run uses a training script with parameterized hyperparameter values to
train a model, and logs the target performance metric achieved by the trained model.

4. Defining a search space
The set of hyperparameter values tried during hyperparameter tuning is known as
the search space. The definition of the range of possible values that can be chosen
depends on the type of hyperparameter.
Discrete hyperparameters
Some hyperparameters require discrete values - in other words, you must select the value
from a particular set of possibilities. You can define a search space for a discrete
parameter using a choice from a list of explicit values, which you can define as a
Python list (choice([10,20,30])), a range (choice(range(1,10))), or an arbitrary set of comma-
separated values (choice(30,50,100))

5. Defining a search space
Continuous hyperparameters
Some hyperparameters are continuous - in other words you can use any value along a
scale. To define a search space for these kinds of value, you can use any of the following
distribution types:
•normal
•uniform
•lognormal
•loguniform

6. Configuring sampling

7. Configuring early termination
With a sufficiently large hyperparameter search space, it could take many iterations (child
runs) to try every possible combination. Typically, you set a maximum number of
iterations, but this could still result in a large number of runs that don't result in a better
model than a combination that has already been tried.
To help prevent wasting time, you can set an early termination policy that abandons runs
that are unlikely to produce a better result than previously completed runs. The policy is
evaluated at an evaluation_interval you specify, based on each time the target
performance metric is logged. You can also set a delay_evaluation parameter to avoid
evaluating the policy until a minimum number of iterations have been completed.

8. Tune Hyperparameters
https://ceteongvanness.wordpress.com/?p=441
2

9. References
Microsoft Docs