Building a Brain with Raspberry Pi and Zulu Embedded JVM

© Copyright Azul Systems 2017
@speakjava
Building A Brain With
Raspberry Pi And
Zulu Embedded JVM
Simon Ritter
Deputy CTO, Azul Systems
1

Introduction

Inspiration
3

Human Brain Neocortex

Hierarchical Temporal Memory
 Theoretical framework for how the neocortex works
– Models neurons and synapses and how they interact
– Also includes dendrites and precise synapses
 This is different to basic neural network topologies
 HTM simulated neuron far more complex
– More adaptable than AI neuron
– links decay over time if not used (temporal)
 Potentially much better at learning
5

Introduction To Machine Learning
6

Learning For Computers
 Machine Learning
– Giving “computers the ability to learn without being
explicitly programmed” - Arthur Samuel
– Requires training data to predict future values
7

Learning For Computers
 Deep Learning
– “Neural networks with more than two layers”
– Not necessary to have training data
 Deep Reinforcement Learning
– No training data
– Learn by experience
– Much like a human: trial and error
8

Markov Decision Process
 Mathematical model of decision making
– Outcome partly random, partly controlled by decision
 5-tuple
– S: State
– A: Action
– P: Probablility that action a leads to state s
– R: Reward for transitioning
– γ: Discount factor
 Difference in importance between future and present rewards
9

Markov Decision Process
10
By waldoalvarez - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=59364518

Deep Q Network
 Q: Quantity of a state-action combination
– Learning rate
– Discount factor
– Initial conditions
Used by Deep Mind
to learn to play Atari
games with no previous
experience

Project Hardware

Raspberry Pi
 Fantastically successful development board
– Cheap, powerful, great connectivity
– Original boards $25, Pi Zero only $5!
 Officially launched on Febuary 29th 2012
– First production run was 10,000 boards
– RS reported over 100,000 pre-orders in one day
 Crashed servers
– Over 10 million boards have been sold (Sept, 2016)
13

Raspberry Pi Variations
14
Model A Model B Pi Zero
ARM v6
700 MHz
ARMv6/7/8
700/900/1200 MHz
ARM v6
1 GHz
1 core 1 or 4 core 1 core
256 Mb
512 Mb
512 Mb
1 Gb
512 Mb
1 x USB
No Network
2/4 USB
100 Mb Ethernet
WiFi/Bluetooth (v3)
1 micro USB
No ethernet
WiFi/Bluetooth (Zero W)

Pi Brain Architecture
Network
Client
Laptop
Raspberry Pi
Cluster
(Emulating HTM)

Raspberry Pi Cluster (Mk 1)
16

Raspberry Pi Cluster (Mk 2)
17

Cluster Hat
18
USB GPIO
Raspberry Pi 3
Master
PORT 1
PORT 2
PORT 3
PORT 4
Pi Zero W
Slaves
GPIO 22
GPIO 23
GPIO 24
GPIO 25
Alert LED
GPIO 21

Java For Embedded
Applications

Why Java?
 “Write once, run anywhere”
– Easy to move code between different platforms
 Wide range of libraries and frameworks
– Takes a lot of hard work out of app development
 Simple interfacing
– Pi4J and DIO libraries for Raspberry Pi
 Simple concurrency
– Comprehensive multi-threading support
 Native code integration for complex situations
– JNI
20

Azul Zulu Java For Embedded
 Built from OpenJDK code base
 Passes all TCK tests
 Ports for Intel, ARM, PowerPC
 ARM v6, v7, v8
– Soft and hard float
– 64 bit close to release
– C2 and C1+C2 compiler enhancements
 Drop in replacement for other JVMs
 No licensing restrictions (FoU)
21

Machine Learning In Java

Deep Learning Basics
 Neural network nodes
– Neuron takes weighted inputs that can change
 Organised as layers
– Non-linear processing
– Different levels of problem abstraction
– Good analogy to HTM model
 Unsupervised learning
– No initial data set
– Learn by experience
23

Neural Network Elements: Node
24
Σ λ
1
X1
X2
X3
W0
W1
W2
W3

Neural Network Elements: Layers
25
Output Layer
Hidden Layer
Input Layer

DeepLearning4J (DL4J)
 Toolkit for building, training and deploying Neural Networks
– configure neural networks and build computation graphs
 ND4J: N-dimensional arrays for Java
– Core technology for deep learning
 DataVec
– Transform data into feature vectors
 Arbiter
– Used to evaluate and tune ML models
– Hyperparameter optimisation functionality
26

RL4J Basics
 Low-dimensional (state) and high-dimensional (pixels)
processing
– MDP class (Markov Decision Process)
– DQN class (Deep Q Network)
– Policy abstract class
 Choose the next action given a state
 ACpolicy (Actor Critic)
 BoltzmannQ (Probablilty distribution)
 DQNPolicy (Action with maximum Q value)
 EpsGreedy (Epsilon Greedy, partially random)
28

Minecraft And Project Malmo

Minecraft
 Immensly popular ‘construction’ game
– Especially amongst children
– Different modes: creative, survival, hardcore
 PC (first) version written in Java
– 121 million copies sold worldwide (all versions)
– Probably most successful Java desktop app ever
– Mojang aquired by Microsoft in 2014
30

Minecraft

Project Malmo
 Microsoft Labs project
 AI experimentation platform built on top of Minecraft
– Mod for Java version of Minecraft
– Libraries to write AI agents to act as player in game
 Including Java
32
Image courtesy of Microsoft

Malmo Example Code (1)
33
AgentHost host = new AgentHost();
ClientPool clientPool = new ClientPool();
clientPool.add(new ClientInfo("10.0.0.2"));
MissionSpec mission = new MissionSpec();
mission.timeLimitInSeconds(120);
mission.requestVideo(640, 480);

Malmo Example Code (2)
34
MissionRecordSpec missionRecord =
new MissionRecordSpec("./saved_data.tgz");
missionRecord.recordCommands();
missionRecord.recordMP4(20, 400000);
missionRecord.recordRewards();
missionRecord.recordObservations();
host.startMission(mission, clientPool,
missionRecord, 0, "TEST");

Minecraft Interaction
 MissionSpec
– setModeToCreative
– startAt/endAt
– drawBlock/drawLine/drawSphere
 AgentHost
– startMission
– peekWorldState
– sendCommand
35

Putting It All Together

Demo Software Architecture
37
Laptop
Minecraft with
Malmo mod
Pi Zero 1
(Input)
SS RL4J
Pi Zero 2
(Fliter 1)
RL4JSS
Pi Zero 3
(Fliter 2)
RL4JSS
Pi Zero 4
(Output)
RL4JSS
Raspberry Pi 3
Demo Controller
Malmo Library
TCP/I
P

Demo Scenario
 Original idea:
– Have agent build a house
– Turns out that’s really hard
 Demo revision 1
– Have the agent build a wall
– Simpler, but not much, than building a house
 Demo revision 2
– Have agent walk around without getting killed
– Easier reinforcement learning, reward is to stay alive

Applying ML To Minecraft
 DL4J/RL4J uses vectors to represent data
– Initially use Minecraft world state
– Long term goal is to use video frames
 Have neural network extract state from this
 Start with random moves
– Record success/failure (live/die)
 Use reinforcement to (hopefully) learn how to navigate
minecraft world safely
39

Conclusions & Future
Directions

Conclusions
 Embedded hardware can be used for AI/ML/DL
– Model on the structure of the human brain
– Cheap to build multi-core, multi-CPU cluster
 Java is an ideal language for this
– Easy to learn
– Simple threading model
– Many, many useful libraries
 Minecraft is a fun tool for AI testing and development
41

Future Directions
 More work on ML/DL for Minecraft
– Make a better player!
 Investigation of jCUDA
– Use NVidia Jetson TK1 board
 Use of machine vision
– OpenCV Java bindings
42

Further Information
 www.raspberrypi.org
 clusterhat.com
 deeplearning4j.org
 minecraft.net
 www.microsoft.com/en-us/research/project/project-malmo
 zulu.org
 Code to be published on Github after some cleaning up
– Will post a blog about this
43

Demo

Building a Brain with Raspberry Pi and Zulu Embedded JVM

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building a Brain with Raspberry Pi and Zulu Embedded JVM

Similar to Building a Brain with Raspberry Pi and Zulu Embedded JVM (20)

More from Simon Ritter

More from Simon Ritter (20)

Recently uploaded

Recently uploaded (20)

Building a Brain with Raspberry Pi and Zulu Embedded JVM

Editor's Notes