Recommended
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to entenr_colloquium_v02
Similar to entenr_colloquium_v02 (20)
entenr_colloquium_v02
- 1. ENHANCING RESTRICTIVE COMPUTE DEVICES ROBERT ENTENMAN MAY 29 2015 UNIVERSITY OF WASHINGTON TACOMA ● SPRING COLLOQUIUM ENTENR@UW.EDU
- 2. WHAT IS A RESTRICTIVE COMPUTE DEVICE?
- 3. CAN WE MAKE THEM BETTER? Of course!
- 4. HOW CAN WE ENHANCE THESE DEVICES? PARROT AR.DRONE 2.0
- 5. HOW CAN WE ENHANCE THESE DEVICES? + • ARM-based • No room for more tasks in main control loop • X86-based • Easy to exploit “embarassingly parallel” operations • Comm. Expansions Available
- 6. SAMPLE CONTROL LOOP Motors Altimeter Gyros Pressure Ultrasonic Magnetometer Accelerometer MAIN SAMPLING LOOP Drone Status(TCP) Edison CPU Progressive Video Stream(UDP) Image Processing Location Update Remote IO Update MAIN UPDATE LOOP Control Commands(UDP)
- 8. EXISTING RESEARCH Munich Technical University – CV Group, MAV Project Credit: MTU http://vision.in.tum.de/research/quadcopter Credit: UWT Center for Data Science IEEE ICARCV 2014 Conference
- 10. PROOF OF CONCEPT AND STUDY • Evaluate: • Loss of Power • Loss of Communications • Analysis of outbound control feedback messaging • Project Scope and Phases • Control of Single Drone • Image Processing • Control of Multiple Drones
- 11. PHASE I – CONTROL OF SINGLE DRONE Edison CPU
- 12. PHASE II – IMAGE PROCESSING Edison CPU
- 13. THANK-YOU Faculty Sponsor: Dr. Matt Tolentino Additional Questions? ● entenr@uw.edu Image/Asset Credits: University of Washington Tacoma Center for Data Science Technische Universität München Wikipedia OpenCV Home Parrot Electronics Home Intel IoT Press Kit
Editor's Notes
- What are restrictive compute devices? Typically small form-factor or embedded devices Limited by size, maybe power, maybe programmatic limits. The unifying boundary here is that due to an external factor, processing time is limited in terms of complexity or operations per second. Perhaps the problem being solved is just computationally intensive. An image processing algorithm, or possibly even speech pattern recognition Here is an Auto ECU (Engine Control Uni)t from 2001– running a CPU of about 40 Mhz, and maybe 1 MB of program code to run a car. This TV set top box is also a restrictive compute device, mainly due to temperature constraints, by keeping noise level as passive as possible. Finally, we have a home power meter. This device has just enough intelligence to sample data, and forward it onward to a power utility. All of these devices are programmed to do simple tasks via deliberative control.
- Short answer: Of course! This is a Nest Thermostat, which leverages the power of the great Google in the sky to augment the base, simple, and restrictive functions that a thermostat would normally perform with the massive computing resources available in the cloud. This restrictive compute device’s functions have been augmented by wirelessly being able to receive and transmit data from the cloud (and all the big brother implications that this allows.)
- This is a Parrot AR.Drone 2.0. It uses 4 electric motors for actuation to provide it with 6 degrees of Freedom (DOF). 1 Ghz. 32-bit ARM Cortex A8, with an video DSP running at 800 Mhz. 1 GB DDR2 RAM @ 200 Mhz. 3-axis accelerometer, 3-axis magnetometer, 3-axis, gyroscope, Pressure sensor, and ultrasonic sensor. Camera operating at 60FPS Additionally, each motor has an 8 MIPS CPU per motor, listening to the ARM-based processor
- Let’s add on computation: Cost-Effective Source Exploit x86 instruction set for optimizations Wireless I/O Capable of even sending its commands out to a higher brain. The Intel Edison platform is all of these for about $50 today. The Edison platform is an x86 Atom-based dual-core chip. This die is attached to an Arduino package, enabling us to offer additional communication expansion such as Cellular, or Bluetooth LE 4.0.
- In this scenario, one Edison is established as Primary, and uses Bluetooth as a side-channel to communicate with the other Edison units which become “Secondary” units. This in theory allows for up to 32 discrete drone units in a single squad, assuming the Bluetooth communication side-channel loop could support it. Further study required here.
- There has been some research done in the area of drones and computer vision, as this is a popular topic that has captured the public’s imagination and excitement. A project from the Technical University of Munich has successfully demonstrated edge detection and pathfinding algorithms using a single drone. They too are leveraging OpenCV library, an open-source repository of Computer Vision API’s originally developed by Intel, and opened up for continued improvement. And more close to home, work undertaken here by former UWT Grad Student Kenny Kong, and Drs. Sheng and PHONETIC: “Tear-desci” involved writing a collision avoidance algorithm by sampling image data from camera sensors, and building a comprehensive set of polygon obstacles to avoid. Both of these solutions involved utilizing external CPU processing from laptops or nearby desktops. To date no one has leveraged small form-factor CPU’s that could be physically attached to the drones themselves. Phonetic: Tear-desci
- This is the National Palace in Haiti And this is after. A drone that could negotiate obstacles, control other drones, or relay information back to outside personnel would be incredibly useful during an emergency or disaster scenario. This is just one of many possible applications include security/defense, search and rescue, and…also, isn’t the idea of a fleet of drones kind of cool?!
- During the POC phase I evaluated: How the drone handles interruption of primary channel and side channel functions How the drone handles loss of power How does the drone communicate status updates over the TCP channel? What are the detail messages available How resilient is the UDP channel? (detail: Obviously you end retransmitting “LEFT, LEFT, LEFT” over a UDP channel which means you have to “react” to no longer needing to go Left via some other input data.
- Software Stack used to produce Phase I Edison stack supports NodeJS, and by extension the Node Package Manager NodePM. Cylon.JS is a framework used to support drones, and other robot packages out there. It provides a convenient interface for control. Note: We are not exploiting much of x86 in Phase 1, really just trying to get a feel for the hardware, tolerances with WiFi control distance, etc.
- In Phase II we’ll leverage the OpenCV libraries to begin performing image processing and edge detection on the video frames sent back from the drone. To begin with we’ll feed static images from a known source in order to perfect the algorithm. This implementation will likely be done in C or C++ for ease of adding in direct code written in assembly.