Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cars and Computers: Building a Java Carputer

3,444 views

Published on

The average family car of today has significantly more computing power than got the first astronauts to the moon and back. Modern cars contain more and more computers to monitor and control every aspect of driving, from anti-lock brakes to engine management to satellite navigation.

This session will look at how Java can (and is) used in cars to add more data collection. This will cover a project that was written to collect a variety of data from a car whilst driving (including video) and then play it back later so driving style and performance could be evaluated. There will be plenty of demonstrations.

Session at the IndicThreads.com Confence held in Pune, India on 27-28 Feb 2015

http://www.indicthreads.com
http://pune15.indicthreads.com

Published in: Software
  • Be the first to comment

Cars and Computers: Building a Java Carputer

  1. 1. Cars and Computers: Building a Java Carputer Simon Ritter Head of Java Technology Evangelism Oracle Corp. Twitter: @speakjava
  2. 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
  3. 3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Cars And Computers
  4. 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. My First Car: 1981 • No electronics – Well, it had a radio • Purely electromechanical – Points/Distributor – Carburettor/Manual choke – Drum brakes – Dynamo – Lights, horn, wipers, fan, ignition switch 1971 Mini Clubman 1000
  5. 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Car Wiring: 1970s Image coutesy of Haynes manuals
  6. 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. My Current Car • Lots of electronics – Engine Control Unit (ECU) – Fuel Injection/Electronic timing – “Fly-by-wire” throttle – Anti-lock Braking System (ABS) – Electronic Stability Program (ESP) – Magnetorheological Suspension – Satellite navigation – Auto-sensing wipers and lights 2011 Audi S3
  7. 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Bus architecture means substantially less wiring Car Wiring: 2011 Convenience Bus Infotainment Bus Powertrain Bus
  8. 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. The Java Carputer (Mark 1)
  9. 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Carputer Design Objectives • Display realtime data • Engine performance (Power, Torque, Load) • Driver data (Throttle position, steering angle, braking force, etc) • G-Forces on car • Graphed data for time-based analysis
  10. 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Carputer Hardware Architecture USB WiFi Dongle Touch Screen Accelerometer HDMI USB I2C 12V Car Converter ELM327 Car Service Port 5V USB
  11. 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. ELM327 • WiFi or Bluetooth connection to OBD-II • Fixed IP address, Ad-hoc networking • Need to configure Raspberry Pi – /etc/network/interfaces • AT style commands for control • Non-AT commands are assumed to be OBD-II – Simple request-response interaction – Easy to write Java code to handle this Cheap way to hack your car
  12. 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Touchscreen • 2 DIN fitting size – Ideal for centre console • HDMI input – Specifically marketed for Raspberry Pi • USB connection for touch screen Lots of things available on eBay
  13. 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Hardware Fitting Challenges Touchscreen
  14. 14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Accelerometer • Sparkfun breakout board MPU 9150 – 9 DoF sensor (accelerometer, gyroscope, compass) • Communications via I2C – Configure Raspberry Pi /etc/modules • i2c-bcm2708, i2c-dev • i2c-detect -y 1 to get address – Compass communication is a bit more complicated • Second I2C bus
  15. 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Accelerometer Code Using Pi4J Library on Java SE Embedded I2CBus bus = I2CFactory.getInstance(I2CBus.BUS_1); I2CDevice device = bus.getDevice(0x68); /* Start sensing */ device.write(0x6B, (byte)0b00000000); device.write(0x6C, (byte)0b00000000); /* Set configuration */ device.write(0x1B, (byte)0b00011000); // Gyroscope device.write(0x1C, (byte)0b00000100); // Accelerometer device.read(0x3B, accelData, 0, ACCEL_PACKET_SIZE);
  16. 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Carputer Software Architecture • Screens based – Splash screen – Basic and advanced car data – G forces on car – Graphed results of different parameters • Simple UI – Can’t read numbers when driving • Touchscreen to switch screens – Repurpose existing car controls to change screen? Realtime Data
  17. 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Take Inspiration From Others UI Design Ideas Renault Megane Sport
  18. 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Take Inspiration From Others UI Design Ideas Tesla
  19. 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Take Inspiration From Others UI Design Ideas Lamborghini Reveton
  20. 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Simple Data Display Control • Only uses 3 nodes – Polygon – Rectangle – Line – Labels are optional • Displays – Current value – Maximum value since start (resetable) • Simple to understand from a glance
  21. 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Splash Screen UI Screens
  22. 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. UI Screens
  23. 23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. UI Screens
  24. 24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. The Java Carputer (Mark 2)
  25. 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Carputer Hardware Architecture 25 USB via FTDICrossover ethernet Infra red Thermal sensor CAN High CAN Low OBD2 Service Port Wi-Fi Accelerometer/ Gyro/Compass I2C USB UART Serial GPSHeart Rate Monitor USB Video Camera Raspberry Pi Model B+ 7” Touch secreen USB SDR/TPMS CAN Bus
  26. 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Tracking Position With GPS • AdaFruit Ultimate GPS breakout – 10Hz read rate – Serial interface • Very simple configuration – Disable /dev/ttyAMA0 on Pi as console – 9600 8N1 – Read data using librxtx-java – Comma separated values $GPGGA and $GPVTG – Need to handle frequent data corruption
  27. 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Heart Rate Monitoring of the Driver • Sparkfun board – USB interface, serial connection • Similar to GPS – Use librxtx-java – 9600, 8N1 • More complicated coding – Actually need to send a read request! • Some issues with wireless connection – 80cm max, doesn’t work well through a Rapsberry Pi
  28. 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. In-car Video • Becoming very common to avoid insurance issues • Transcend DrivePro 200 – Built-in Wi-Fi – iPhone app • How to get video stream? – Approached Transcend tech support – Found VLC code – Wireshark – SYN/ACK, Yuck! 28 Camera Application 192.168.0.1 192.168.0.2 Port 9000 Port 43229 Port 9001 Port 42719 Port 9002 Port 58105 Port 8554 (RTSP) Port 60583
  29. 29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. The Simplest Solution Is Often The Best 29 Push button using GPIO lines and Pi4J
  30. 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Thermal Sensors For Brakes and Tyres The Goal
  31. 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Infra-Red Thermometer • MLX90614 breakout board – I2C interface seemed problematic on the Pi – Also has serial interface – Use FTDI breakout board – 4 wires becomes simple serial interface 31 The Reality
  32. 32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Thermal Sensor Data • Provides continuous reading as lines of text (e.g. 73.2F) – Don’t like Fahrenheit, wanted Celcius – Lambdas and Streams! 32 private double currentTemperature; public void run() { thermalReader.lines() .mapToDouble(s -> Double.parseDouble(s.substring(0, s.length() - 1))) .map(t -> ((t - 32) * 5 / 9)) .filter(t -> t != currentTemperature) .peek(t -> listener.ifPresent(l -> l.temperatureChange(t))) .forEach(t -> currentTemperature = t); }
  33. 33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Thermal Sensor Data • Provides continuous reading as lines of text (e.g. 73.2F) – Don’t like Fahrenheit, wanted Celcius – Lambdas and Streams! 33 private double currentTemperature; public void run() { thermalReader.lines() .mapToDouble(s -> Double.parseDouble(s.substring(0, s.length() - 1))) .map(t -> ((t - 32) * 5 / 9)) .filter(t -> t != this.currentTemperature) .peek(t -> listener.ifPresent(l -> l.temperatureChange(t))) .forEach(t -> this.currentTemperature = t); }
  34. 34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 34 Tyre Sensor Fitting The Thermal Sensors
  35. 35. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 35 Brake Disk Sensor Fitting The Thermal Sensors
  36. 36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 36 Separate Pi in Engine Bay Fitting The Thermal Sensors
  37. 37. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Fitting The Thermal Sensors • Simple server listens on port • Carputer requests data • Couple of issues – Server needed reset for new connection (bug to be fixed) – How to tell which sensor is which (ttyUSB0/ttyUSB1) 37 The Networked Car
  38. 38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Fitting The Thermal Sensors • Rasperry Pi at over 50C – After 10 minute drive • New approach required – Longer USB cables – Put the Pi in the car 38 Internal Combustion Produces Heat
  39. 39. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Getting Data From The CAN Bus • Controller Area Network • Used to connect many sensors, actuators and devices – Without huge amounts of wiring • Conveniently the CAN bus is connected to the OBD2 port – Yes, the wiring is there – No, the signals are not – Presumably security measure from car manufacturers – Boo!
  40. 40. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Where There’s A Will There’s A Way Getting Data From The CAN Bus
  41. 41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Where There’s A Will There’s A Way Getting Data From The CAN Bus Success! (And the car still works)
  42. 42. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Getting Data From The CAN Bus • Lawicel CAN bus to USB • Serial device • Sends decoded data • Need to write a Java library – CAN id – Message data • Then the hard part starts – Figuring it all out 42 Electrical Connectivity Is Only The Start
  43. 43. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 43 Finished Data Capture System
  44. 44. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Data Recording and Analysis
  45. 45. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Display Collected Data • JavaFX application • Video stream – Synchronised to data streams • Data recorded as sequence of text records – Comma separated values – One line per sensor – Time stamp • Read all data in at start of application – 15 minutes of data takes about 100ms 45
  46. 46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. • Video (with controls) • G-forces • GPS co-ordinates/altitude • Map of current position • Speed (compare car and GPS) • Bearing • Engine load • Fuel pressure • Air intake temperature • Coolant temperature • Catalyst/exhaust temperature • Tyre temperature • Brake temperature • Throttle position • Engine RPM • Heart rate 46 Data To Display
  47. 47. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Conclusions & Resources
  48. 48. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Conclusions • Cars produce lots of data • Use of sensors with Java is simple to collect (most) data • Playback of data useful for insurance, fleet management, training • Java and JavaFX make UI design simple
  49. 49. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Resources and Links • www.sparkfun.com • www.adafruit.com • www.raspberrypi.org • blogs.oracle.com/speakjava • Twitter: @speakjava
  50. 50. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Demos

×