Selection of the optimal parameters for machine learning tasks is challenging. Some results may be bad not because the data is noisy or the used learning algorithm is weak, but due to the bad selection of the parameters values. This presentation gives a brief introduction about evolutionary algorithms (EAs) and describes genetic algorithm (GA) which is one of the simplest random-based EAs. A step-by-step example is given in addition to its implementation in Python 3.5.
---------------------------------
Read more about GA:
Yu, Xinjie, and Mitsuo Gen. Introduction to evolutionary algorithms. Springer Science & Business Media, 2010.
https://www.kdnuggets.com/2018/03/introduction-optimization-with-genetic-algorithm.html
https://www.linkedin.com/pulse/introduction-optimization-genetic-algorithm-ahmed-gad
Genetic Algorithm (GA) Optimization - Step-by-Step Example
1. Genetic Algorithm (GA) Optimization –
Step-by-Step Example with Python
Implementation
Ahmed Fawzy Gad
ahmed.fawzy@ci.menofia.edu.eg
MENOUFIA UNIVERSITY
FACULTY OF COMPUTERS AND INFORMATION
ARTIFICIAL INTELLIGENCE
ALL DEPARTMENTS
المنوفية جامعة
الحاسبات كليةوالمعلومات
اإلصطناعي الذكاء
األقسام جميعالمنوفية جامعة
2. Training a Machine Learning Model
• Goal is to find the set of parameters (𝒘 𝟏:𝒘 𝟔) that maps the following
input to its output.
𝒚 = 𝒘 𝟏 𝒙 𝟏+𝒘 𝟐 𝒙 𝟐+𝒘 𝟑 𝒙 𝟑+𝒘 𝟒 𝒙 𝟒+𝒘 𝟓 𝒙 𝟓+𝒘 𝟔 𝒙 𝟔
𝒙 𝟏 𝒙 𝟐 𝒙 𝟑 𝒙 𝟒 𝒙 𝟓 𝒙 𝟔 𝒚
4 -2 7 5 11 1 44.1
𝒚′
= 𝟒𝒘 𝟏 − 𝟐𝒘 𝟐 + 𝟕𝒘 𝟑 + 𝟓𝒘 𝟒 + 𝟏𝟏𝒘 𝟓 + 𝒘 𝟔
Data
6. 𝒘 𝟏 𝒘 𝟐 𝒘 𝟑 𝒘 𝟒 𝒘 𝟓 𝒘 𝟔
-1 2 2 -3 2 0.9
Solution 3
𝒚′
= 𝟏𝟑. 𝟗
Absolute Error 𝒆𝒓𝒓𝒐𝒓 = |𝒚 − 𝒚′
|
𝒆𝒓𝒓𝒐𝒓 = 𝟑𝟎. 𝟐
𝒆𝒓𝒓𝒐𝒓 = 𝟒𝟒. 𝟏 − 𝟏𝟑. 𝟗
Difficult to find the best solution manually.
Use optimization technique such as Genetic Algorithm (GA).
7. • Genetic algorithm is based on natural evolution of organisms.
• A brief biological background will be helpful in understanding GA.
Genetic Algorithm (GA)
8. • Genetic algorithm is based on natural evolution of organisms.
• A brief biological background will be helpful in understanding GA.
Chromosomes
Genes
Eye
Color
Dimples Freckles
Organism
Cells
Genetic Algorithm (GA)
9. • Genetic algorithm is based on natural evolution of organisms.
• A brief biological background will be helpful in understanding GA.
Chromosomes
Genes
Eye
Color
Dimples Freckles
Organism
Cells
Genetic Algorithm (GA)
https://www.icr.org/article/myth-human-evolution
10. • Genetic algorithm is based on natural evolution of organisms.
• A brief biological background will be helpful in understanding GA.
Chromosome
Genes
GA
Individuals/Solutions
Genetic Algorithm (GA)
Chromosomes
Genes
Eye
Color
Dimples Freckles
Organism
Cells
11. What are the Genes?
• Gene is anything that is able to enhance the results when changed.
• By exploring the following model, the 6 weights are able to enhance
the results. Thus each weight will represent a gene in GA.
𝒚 = 𝒘 𝟏 𝒙 𝟏+𝒘 𝟐 𝒙 𝟐+𝒘 𝟑 𝒙 𝟑+𝒘 𝟒 𝒙 𝟒+𝒘 𝟓 𝒙 𝟓+𝒘 𝟔 𝒙 𝟔
Gene 0 Gene 1 Gene 2 Gene 3 Gene 4 Gene 5
𝒘 𝟏 𝒘 𝟐 𝒘 𝟑 𝒘 𝟒 𝒘 𝟓 𝒘 𝟔
40. -1 2 2 -3 2 0.9
3.1 4 0 2.4 4.8 0
-2 3 -7 6 3 3
-2 3 -7 -3 1 0.9
3.1 4 0 6 1.5 3
-1 2 2 2.4 2.4 0
Old
Individuals
New
Individuals
New Population (Generation 1)
Why Reusing Old Individuals?
41. -1 2 2 -3 2 0.9
3.1 4 0 2.4 4.8 0
-2 3 -7 6 3 3
-2 3 -7 -3 1 0.9
3.1 4 0 6 1.5 3
-1 2 2 2.4 2.4 0
Old
Individuals
New
Individuals
New Population (Generation 1)
Why Reusing Old Individuals?
GA is a random-based optimization technique. There is no guarantee that the new individuals will be better
than the previous individuals. Keeping the old individuals at least saves the results from getting worse.
42. New Population (Generation 1)
Old
Individuals
New
Individuals
Population Mating Pool Crossover MutationFitness Value
-1 2 2 -3 2 0.9
3.1 4 0 2.4 4.8 0
-2 3 -7 6 3 3
-2 3 -7 -3 1 0.9
3.1 4 0 6 1.5 3
-1 2 2 2.4 2.4 0