This is for the iMLSE dicussions.

1. iMLSE Discussions
Hiroshi Maruyama
PFN Fellow

2. 2
What is Deep Learning? – A (Stateless) Function
Y = f(X)X Y
Very high-
dimensional, any
combination of
continuous and
categorial variables
Low-dimensional for
classification, very
high-dimensional
for generation

3. 3
Example: Converting Celsius to Fahrenheit
Hiroshi Maruyama
double c2f(double c) {
return 1.8*c + 32.0;
}
Input: C
Output: F
Where F is Fahrenheit
equivalent of C in Celsius
Requirements
Algorithm
F = 1.8 * C + 32Model
A Priori
Knowledge
Model must be know in advance, and
Algorithm must be constructible

4. Training Data Set
Observation
Training（search for parameter θ）
No knowledge on model or algorithm is required!
Alternative Approach – Data-Driven, Inductive Programming
(aka Statistical Modeling)

5. 5
Deep Neural Net as a Universal Computing Mechanism
⚫ Very large number of parameters
⚫ Can approximate ANY high-
dimensional function*
➔ Pseudo Turing Complete!
Output
Input
* G. Cybenko. Approximations by superpositions
of sigmoidal functions. Mathematics of Control,
Signals, and Systems, 2(4):303–314, 1989.

6. Maruyama’s Conjecture:
In 2020, more than half of newly developed software have
inductively-trained components
This is the largest paradigm shift since the inventin of digital computer!

7. https://medium.com/@mijordan3/artificial-intelligence-the-revolution-hasnt-happened-yet-5e1d5812e1e7
The need for new engineering descipline

8. What is engineering?
Theories（e.g.,
structure）
＊ Safety Factor
New technology is accepted by the society only after it becomes engineering descipline
Civil Engineering Handbook, p999
Why do we trust bridges? Because of the accumulated knowledge
called Civil Engineering

9. 9
What are the SE concepts that can be brought into the ML world?
⚫ Testing
— Covarage
— Regression
⚫ Invariance
⚫ Reuse
⚫ :

10. 10
Thank You
Twitter: @maruyama