Unity ML-Agents enables developers to create intelligent agents capable of learning through various methods such as reinforcement and imitation learning within dynamic environments. The toolkit has evolved through several versions, adding components like curriculum learning, multi-agent training, and on-demand decision-making. It supports deployment across multiple platforms, allowing seamless integration into Unity projects.

2. Unity for Deep Learning:
ML-Agents Explained

3. Mike Geig
Head of Global Evangelism Content

4. Let’s start with one important
question...

5. Why program system to complete
a specific task when you can
design it to learn?
Let’s start with one important question...

8. Visual Complexity Cognitive ComplexityPhysical Complexity
ML Training Environment Requirements

9. The Unity
Ecosystem

10. ML-Agents v0.1
Components
● Learning Environments
● Flexible training scenarios (single
agent, simultaneous single agent,
adversarial self-play, cooperative
multi-agent, competitive multi-
agent, ecosystem
● Monitoring agent’s decision
making
● Complex Visual observations
ML-Agents v0.2
Components
● Additional environments (two
new continuous control
environments, plus two
platforming environments)
● Curriculum Learning
● Broadcasting
● Flexible monitor
ML-Agents v0.3
Components
● Imitation Learning
● Multi-Brain training
● On-demand decision-making
● Memory-enhanced agents

11. How does it work?

12. Unity ML-Agents Workflow
Create Environment Train Agents Embed Agents

13. Create Environment (Unity)
Observe & Act
Decide
Coordinate

14. Unity ML-Agents Workflow
Create Environment Train Agents Embed Agents

15. Training Methods
Reinforcement Learning
● Learn through rewards
● Trial-and-error
● Super-speed simulation
● Agent becomes “optimal” at task
Imitation Learning
● Learn through demonstrations
● No rewards necessary
● Real-time interaction
● Agent becomes “human-like” at task

16. Unity ML-Agents Workflow
Create Environment Train Agents Embed Agents

17. Embed Agents (Unity)
● Simply import a .bytes file (trained
brain) into Unity project
● Set corresponding brain component
to “Internal” mode.
● Support for Mac, Windows, Linux,
iOS, and Android.

18. Let’s see it in action!

19. Learning Scenarios

20. Goal Balance ball as long as possible
Observations Platform rotation, ball position and
rotation
Actions Platform rotation (in x and z)
Rewards Bonus for keeping ball up
Twelve Agents, One Brain,
Independent Rewards

22. Goal Keep ball up as long as possible
Observations Positions and velocities of racket and ball
Actions Forward, backward, and upward movement
Rewards +0.1 when sent over net by agent
-0.1 when ball falls because of agent
Two Agents, One Brain,
Cooperative Rewards

24. Striker Goal Get the ball into the opponents goal
Goalie Goal Defend own goal from opponents
Observations Local ray-cast perception on nearby objects
Actions Movement and rotation in x, z plane
Striker Rewards +1 when its team scores goal
-0.1 when opponent scores goal
Goalie Rewards -1 when opponent scores goal
+0.1 when its team scores goal
Four Agents, Multi-Brain,
Competitive Rewards

25. Multi-Stage Soccer Training
Defense
Train one brain with
negative reward for
ball entering their goal
Offense
Train one brain with
positive reward for ball
entering opponents goal
Combined
Train both brains
together to play against
opponent team

27. Learning Methods

28. Curriculum Learning

29. Curriculum Learning
● Bootstrap learning of difficult
task with simpler task
● Utilize custom reset parameters
● Change environment task based
on reward or fixed progress
Easy
Difficult

31. Imitation Learning

32. Imitation Learning
Collect demonstrations
from a teacher
Learn policy via imitation

33. ML-Agents v0.1
Components
● Learning Environments
● Flexible training scenarios (single
agent, simultaneous single agent,
adversarial self-play, cooperative
multi-agent, competitive multi-
agent, ecosystem
● Monitoring agent’s decision
making
● Complex Visual observations
ML-Agents v0.2
Components
● Additional environments (two
new continuous control
environments, plus two
platforming environments)
● Curriculum Learning
● Broadcasting
● Flexible monitor
ML-Agents v0.3
Components
● Imitation Learning
● Multi-Brain training
● On-demand decision-making
● Memory-enhanced agents

34. We are hiring!

35. Get it Now
github.com/Unity-Technologies/ml-agents
Contact us
https://unity3d.ai
ML-Agents@Unity3d.com

36. Thank you!
Mike Geig
Mike@unity3d.com
@MikeGeig