Java Carputer session from JMaghreb conference in Morocco.

1. Carjacking
With
Java
(Or
Building
A
Java
Carputer)
Simon
Ri>er
Head
of
Java
Evangelism,
Oracle
Twi>er:
@speakjava
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

2. Safe
Harbor
Statement
The
following
is
intended
to
outline
our
general
product
direcRon.
It
is
intended
for
informaRon
purposes
only,
and
may
not
be
incorporated
into
any
contract.
It
is
not
a
commitment
to
deliver
any
material,
code,
or
funcRonality,
and
should
not
be
relied
upon
in
making
purchasing
decisions.
The
development,
release,
and
Rming
of
any
features
or
funcRonality
described
for
Oracle’s
products
remains
at
the
sole
discreRon
of
Oracle.
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
2

3. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Program
Agenda
Cars
And
Computers
The
Java
Carputer
(Mark
1)
The
Java
Carputer
(Mark
2)
Heads
Up
Display/TelemaRcs
Data
Recording/Analysis
Conclusions,
Resources
and
Demos
1
2
3
4
5
6

4. Cars
And
Computers
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

5. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
My
First
Car:
1981
1971
Mini
Clubman
1000
• No
electronics
– Well,
it
had
a
radio
• Purely
electromechanical
– Points/Distributor
– Carbure>or/Manual
choke
– Drum
brakes
– Dynamo
– Lights,
horn,
wipers,
fan,
igniRon
switch

6. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Car
Wiring:
1970s
Image
coutesy
of
Haynes
manuals

7. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
My
Current
Car
• Lots
of
electronics
– Engine
Control
Unit
(ECU)
– Fuel
InjecRon/Electronic
Rming
– “Fly-­‐by-­‐wire”
thro>le
– AnR-­‐lock
Braking
System
(ABS)
– Electronic
Stability
Program
(ESP)
– Magnetorheological
Suspension
– Satellite
navigaRon
– Auto-­‐sensing
wipers
and
lights
2011
Audi
S3

8. Bus
architecture
means
substan>ally
less
wiring
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Car
Wiring:
2011
Convenience
Bus
Infotainment
Bus
Powertrain
Bus

9. The
Java
Carputer
(Mark
1)
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

10. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Carputer
Design
ObjecRves
• Display
realRme
data
• Engine
performance
(Power,
Torque,
Load)
• Driver
data
(Thro>le
posiRon,
steering
angle,
braking
force,
etc)
• G-­‐Forces
on
car
• Graphed
data
for
Rme-­‐based
analysis

11. Carputer
Hardware
Architecture
USB
WiFi
Dongle
Touch
Screen
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
HDMI
Accelerometer
USB
I2C
12V
Car
Converter
ELM327
Car
Service
Port
5V
USB

12. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
ELM327
Cheap
way
to
hack
your
car
• WiFi
or
Bluetooth
connecRon
to
OBD-­‐II
• Fixed
IP
address,
Ad-­‐hoc
networking
• Need
to
configure
Raspberry
Pi
– /etc/network/interfaces
• AT
style
commands
for
control
• Non-­‐AT
commands
are
assumed
to
be
OBD-­‐II
– Simple
request-­‐response
interacRon
– Easy
to
write
Java
code
to
handle
this

13. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Touchscreen
Lots
of
things
available
on
eBay
• 2
DIN
fiing
size
– Ideal
for
centre
console
• HDMI
input
– Specifically
marketed
for
Raspberry
Pi
• USB
connecRon
for
touch
screen

14. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Touchscreen
Hardware
FiJng
Challenges

15. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Accelerometer
• Sparkfun
breakout
board
MPU
9150
– 9
DoF
sensor
(accelerometer,
gyroscope,
compass)
• CommunicaRons
via
I2C
– Configure
Raspberry
Pi
/etc/modules
• i2c-­‐bcm2708,
i2c-­‐dev
• i2c-­‐detect
-­‐y
1
to
get
address
– Compass
communicaRon
is
a
bit
more
complicated
• Second
I2C
bus

16. Accelerometer
Code
Using
Pi4J
Library
on
Java
SE
Embedded
I2CBus
bus
=
I2CFactory.getInstance(I2CBus.BUS_1);
I2CDevice
device
=
bus.getDevice(0x68);
/*
Start
sensing
*/
device.write(0x6B,
(byte)0b00000000);
device.write(0x6C,
(byte)0b00000000);
/*
Set
configuration
*/
device.write(0x1B,
(byte)0b00011000);
//
Gyroscope
device.write(0x1C,
(byte)0b00000100);
//
Accelerometer
device.read(0x3B,
accelData,
0,
ACCEL_PACKET_SIZE);
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

17. CalculaRng
Power
And
Torque
Torque
(Nm)
=
Mass
x
Wheel
Radius
x
AcceleraRon
(in
G)
Torque
(lb/l)
=
Torque
in
Nm
x
0.73756
Power
(BHP)
=
Torque
(lb/l)
x
engine
RPM
/
5252
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
• Results
will
not
have
high
accuracy
– Values
in
red
are
not
precise
• Dynamometer
is
the
only
way
to
get
accurate
figures
• InteresRng
to
see
values
v.
manufacturers
figures

18. Carputer
Solware
Architecture
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
• Screens
based
– Splash
screen
– Basic
and
advanced
car
data
– G
forces
on
car
– Graphed
results
of
different
parameters
• Simple
UI
– Can’t
read
numbers
when
driving
• Touchscreen
to
switch
screens
– Repurpose
exisRng
car
controls
to
change
screen?
Real>me
Data

19. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Design
Ideas
Take
Inspira>on
From
Others
Renault
Megane
Sport

20. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Design
Ideas
Take
Inspira>on
From
Others
Tesla

21. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Design
Ideas
Take
Inspira>on
From
Others
Lamborghini
Reveton

22. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Simple
Data
Display
Control
• Only
uses
3
nodes
– Polygon
– Rectangle
– Line
– Labels
are
opRonal
• Displays
– Current
value
– Maximum
value
since
start
(resetable)
• Simple
to
understand
from
a
glance

23. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Screens
Splash
Screen

24. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Screens

25. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Screens

26. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
UI
Screens

27. Graph
Plots
Of
Thro>le
PosiRon
And
Power
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

28. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
JavaFX
Charts
And
Colours
Can
Only
Be
Changed
Using
CSS
.axis
{
-­‐fx-­‐tick-­‐label:
#FFFFFF;
}
.axis-­‐tick-­‐mark
{
-­‐fx-­‐stroke:
#FFFFFF;
}
.axis-­‐minor-­‐tick-­‐mark
{
-­‐fx-­‐stroke:
#000000;
}
.chart-­‐plot-­‐background
{
-­‐fx-­‐background-­‐color:
#000000;
}
.chart-­‐alternative-­‐row-­‐fill
{
-­‐fx-­‐fill:
#000000;
}
.default-­‐color0.chart-­‐series-­‐line
{
-­‐fx-­‐stroke:
#FFFF33;
}

29. The
Java
Carputer
(Mark
2)
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

30. Carputer
Hardware
Architecture
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
30
Crossover
ethernet
USB
via
FTDI
Infra
red
Thermal
sensor
CAN
High
CAN
Low
OBD2
Service
Port
Wi-­‐Fi
Accelerometer/
Gyro/Compass
I2C
USB
UART
Serial
Heart
Rate
Monitor
GPS
USB
Video
Camera
Raspberry
Pi
Model
B+
7”
Touch
secreen
USB
CAN
Bus
SDR/TPMS

31. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Tracking
PosiRon
With
GPS
• AdaFruit
UlRmate
GPS
breakout
– 10Hz
read
rate
– Serial
interface
• Very
simple
configuraRon
– Disable
/dev/ttyAMA0
on
Pi
as
console
– 9600
8N1
– Read
data
using
librxtx-­‐java
– Comma
separated
values
$GPGGA
and
$GPVTG
– Need
to
handle
frequent
data
corrupRon

32. Heart
Rate
Monitoring
of
the
Driver
• Sparkfun
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
board
– USB
interface,
serial
connecRon
• Similar
to
GPS
– Use
librxtx-­‐java
– 9600,
8N1
• More
complicated
coding
– Actually
need
to
send
a
read
request!
• Some
issues
with
wireless
connecRon
– 80cm
max,
doesn’t
work
well
through
a
Rapsberry
Pi

33. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
In-­‐car
Video
• Becoming
very
common
to
avoid
insurance
issues
• Transcend
DrivePro
200
– Built-­‐in
Wi-­‐Fi
– iPhone
app
• How
to
get
video
stream?
– Approached
Transcend
tech
support
– Found
VLC
code
– Wireshark
– SYN/ACK,
Yuck!
33
Camera
Applica>on
192.168.0.1
192.168.0.2
Port
9000
Port
43229
Port
9001
Port
42719
Port
9002
Port
58105
Port
8554
(RTSP)
Port
60583

34. The
Simplest
SoluRon
Is
Olen
The
Best
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
34
Push
bu>on
using
GPIO
lines
and
Pi4J

35. Thermal
Sensors
For
Brakes
and
Tyres
The
Goal
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

36. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Infra-­‐Red
Thermometer
• MLX90614
breakout
board
– I2C
interface
seemed
problemaRc
on
the
Pi
– Also
has
serial
interface
– Use
FTDI
breakout
board
– 4
wires
becomes
simple
serial
interface
36
The
Reality

37. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Thermal
Sensor
Data
• Provides
conRnuous
reading
as
lines
of
text
(e.g.
73.2F)
– Don’t
like
Fahrenheit,
wanted
Celcius
– Lambdas
and
Streams!
37
private
double
currentTemperature;
public
void
run()
{
thermalReader.lines()
.mapToDouble(s
-­‐>
Double.parseDouble(s.substring(0,
s.length()
-­‐
1)))
.map(t
-­‐>
((t
-­‐
32)
*
5
/
9))
.filter(t
-­‐>
t
!=
currentTemperature)
.peek(t
-­‐>
listener.ifPresent(l
-­‐>
l.temperatureChange(t)))
.forEach(t
-­‐>
currentTemperature
=
t);
}

38. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Thermal
Sensor
Data
• Provides
conRnuous
reading
as
lines
of
text
(e.g.
73.2F)
– Don’t
like
Fahrenheit,
wanted
Celcius
– Lambdas
and
Streams!
38
private
double
currentTemperature;
public
void
run()
{
thermalReader.lines()
.mapToDouble(s
-­‐>
Double.parseDouble(s.substring(0,
s.length()
-­‐
1)))
.map(t
-­‐>
((t
-­‐
32)
*
5
/
9))
.filter(t
-­‐>
t
!=
this.currentTemperature)
.peek(t
-­‐>
listener.ifPresent(l
-­‐>
l.temperatureChange(t)))
.forEach(t
-­‐>
this.currentTemperature
=
t);
}

39. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
39
Fiing
The
Thermal
Sensors
Tyre
Sensor

40. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
40
Fiing
The
Thermal
Sensors
Brake
Disk
Sensor

41. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
41
Fiing
The
Thermal
Sensors
Separate
Pi
in
Engine
Bay

42. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Fiing
The
Thermal
Sensors
• Simple
server
listens
on
port
• Carputer
requests
data
• Couple
of
issues
– Server
needed
reset
for
new
connecRon
(bug
to
be
fixed)
– How
to
tell
which
sensor
is
which
(>yUSB0/>yUSB1)
42
The
Networked
Car

43. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Fiing
The
Thermal
Sensors
• Rasperry
Pi
at
over
50C
– Aler
10
minute
drive
• New
approach
required
– Longer
USB
cables
– Put
the
Pi
in
the
car
43
Internal
Combus>on
Produces
Heat

44. Geing
Data
From
The
CAN
Bus
• Controller
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Area
Network
• Used
to
connect
many
sensors,
actuators
and
devices
– Without
huge
amounts
of
wiring
• Conveniently
the
CAN
bus
is
connected
to
the
OBD2
port
– Yes,
the
wiring
is
there
– No,
the
signals
are
not
– Presumably
security
measure
from
car
manufacturers
– Boo!

45. Geing
Data
From
The
CAN
Bus
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Where
There’s
A
Will
There’s
A
Way

46. Geing
Data
From
The
CAN
Bus
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Where
There’s
A
Will
There’s
A
Way
Success!
(And
the
car
sRll
works)

47. Geing
Data
From
The
CAN
Bus
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
• Lawicel
CAN
bus
to
USB
• Serial
device
• Sends
decoded
data
• Need
to
write
a
Java
library
– CAN
id
– Message
data
• Then
the
hard
part
starts
– Figuring
it
all
out
47
Electrical
Connec>vity
Is
Only
The
Start

48. Finished
Data
Capture
System
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
48

49. Heads
Up
Display/
TelemaRcs
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

50. In
Car
Heads
Up
Displays
Are
Becoming
A
Reality
Jaguar
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
ConRnental
Virtual
Windscreen
Mini
Cooper
NavTy

51. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Build
Your
Own
HUD
• Turns
out
it’s
not
that
simple
• First
iteraRon
– 3M
MPro110
Micro
Projector,
half
silvered
film
(from
a
cheap
in-­‐car
HUD)
• Didn’t
really
work
– Too
much
reflecRon
of
projector
lamp,
not
image

52. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
52
Build
Your
Own
HUD
Second
AWempt

53. Raspberry
Pi
Heads
Up
Display
• Cheap
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
screen
for
the
Pi
• Use
teleprompter
glass
rather
than
half-­‐silvered
perspex
– 70%
transmission,
30%
refecRon
much
be>er
• Issue:
How
to
produce
an
inverted
image?
• Answer:
Use
a
refecRon
effect
53
Reflection
ref
=
new
Reflection();
ref.setFraction(1.0);
ref.setBottomOpacity(1.0);
ref.setTopOpacity(1.0);

54. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Data
Display
On
HUD
• Speed
and
engine
RPM
– Use
OBD2
data
– Wi-­‐Fi
dongle
to
access
ELM327
• Convert
posiRon
(from
GPS)
into
current
speed
limit
– Bluetooth
dongle
for
pairing
with
mobile
phone
– Configure
Linux
to
provide
network
access
via
TCP/IP
– Transmit
posiRon
obtained
from
GPS
– Write
servlet
to
run
on
home
Linux
box
(open
port
on
firewall)
– Servlet
uses
posiRon
and
Ordanance
Survey
data
to
determine
speed
limit
– (Future
project)
54

55. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Hardware
In
Place
55

56. Data
Recording
and
Analysis
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

57. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Display
Collected
Data
• JavaFX
applicaRon
• Video
stream
– Synchronised
to
data
streams
• Data
recorded
as
sequence
of
text
records
– Comma
separated
values
– One
line
per
sensor
– Time
stamp
• Read
all
data
in
at
start
of
applicaRon
– 15
minutes
of
data
takes
about
100ms
57

58. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
• Video
(with
controls)
• G-­‐forces
• GPS
co-­‐ordinates/alRtude
• Map
of
current
posiRon
• Speed
(compare
car
and
GPS)
• Bearing
• Engine
load
• Fuel
pressure
• Air
intake
temperature
• Coolant
temperature
• Catalyst/exhaust
temperature
• Tyre
temperature
• Brake
temperature
• Thro>le
posiRon
• Engine
RPM
• Heart
rate
58
Data
To
Display

59. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
User
Interface
Design
• Use
JFXtras
library
(Gerrit
Grunwald’s
dials
and
LEDs)
• Formula
1
style
– Thro>le/brake/engine/gear
display
– LED
tachometer
• Heart
rate
number
is
not
too
exciRng
– AnimaRons
– ConRnuous
pulses
was
a
bit
of
a
challenge
(several
parallel,
offset
transiRons)
59
JavaFX

60. Conclusions
&
Resources
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

61. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Conclusions
• Cars
produce
lots
of
data
• Use
of
sensors
with
Java
is
simple
to
collect
(most)
data
• Playback
of
data
useful
for
insurance,
fleet
management,
training
• Java
and
JavaFX
make
UI
design
simple

62. Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.
Resources
and
Links
• www.sparkfun.com
• www.adafruit.com
• www.raspberrypi.org
• blogs.oracle.com/speakjava
• Twi>er:
@speakjava

63. Demos
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.

64. Carjacking
With
Java
(Or
Building
A
Java
Carputer)
Simon
Ri>er
Head
of
Java
Evangelism,
Oracle
Twi>er:
@speakjava
Copyright
©
2014,
Oracle
and/or
its
affiliates.
All
rights
reserved.