The document discusses machine learning concepts and the ml-agents package in Unity. It provides an overview of machine learning types including reinforcement learning, curriculum learning, and imitation learning. It also describes key ml-agents elements such as the academy, brain, agents, observations, state, actions, and reward. The presentation includes demos of reinforcement and imitation learning using the Unity Balance Ball environment.

1. Using ml-agents package v0.5
Machine learning with Unity

2. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
Presentation plan
1. What’s machine learning.
2. Types of machine learning available in Unity.
3. Machine learning elements:
a. Academy/Brains/Agents.
b. State/Observations/Actions/Reward.
4. Technically talking.
5. Demo time.

3. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
What’s machine learning
“Field of artificial intelligence based on
statistic and probabilistic models given
to machines (computers) in order to
give them the capacity to learn.
Mainly three types.“
- Wikipedia French translated

4. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
Reinforcement learning
One of the machine learning methods based on trial and error
iterations.

5. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
Machine learning
“More specifically, the goal of reinforcement
learning is to learn a policy, which is essentially a
mapping from observations to actions. An
observation is what the robot can measure from its
environment (in this case, all its sensory inputs)
and an action, in its most raw form, is a change to
the configuration of the robot (e.g. position of its
base, position of its water hose and whether the
hose is on or off).”
- Unity ml-agents documentation

6. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
Machine learning
“More specifically, the goal of reinforcement
learning is to learn a policy, which is essentially a
mapping from observations to actions. An
observation is what the robot can measure from its
environment (in this case, all its sensory inputs)
and an action, in its most raw form, is a change to
the configuration of the robot (e.g. position of its
base, position of its water hose and whether the
hose is on or off).”
- Unity ml-agents documentation

7. Training vs. Inference

8. Bournane Abdelkrim
Unity developer at Tricksept studio
I also do witchcraft in my free time.
Types of machine learning available in Unity.
1.Reinforcement learning.
2.Curriculum learning.
3.Imitation learning.

9. Example of reinforcement learning

10. Example of Imitation learning

11. ML-agents elements
Academy
• Communicate with external
Python API
• Contains the brains to train.
• Configuration of inference
and training

12. ML-agents elements
Brain
• Contains the model.
• Asks the agents for
observations
• Makes decisions
• Pass actions to agents

13. ML-agents elements
Agent
• Collect observations when
asked by the brain,
• Executes actions given by the
brain,
• Calculate its reward after
every action.
• And sometimes contains the
reset parameters to reset
every episode

14. ML-agents elements
Observations
Parameters that are needed to
make a decision.
Vector Observations: Transform,
velocity, raycast distance…
Visual Observations: Camera
images

15. ML-agents elements
State
Set of observations of
an agent.

16. ML-agents elements
Action
A value given by the brain to the
agent. The agent will decide
what will do with it.

17. ML-agents elements
Reward
The value calculated by the agent
every action.

18. Demo time
Unity Balance Ball

19. Thank you! Any questions?
Unity Balance Ball