The document summarizes a mobile-controlled drone autonomous delivery system designed by a team from Harvard and HKUST. The system uses a DJI Flamewheel 450 drone modified with a Pixhawk autopilot to autonomously deliver packages within a 1 mile range and 20 minute flight time. Key features include proximity sensors for object avoidance, a servo-controlled gripper, and mobile/desktop applications for control and monitoring. Testing showed average speeds of 6.9 m/s and flight times of 20 minutes. Future work aims to increase payload, range, and accuracy while improving battery life and communication reliability.
Unified Systems Engineering feasibilityEric Verhulst
Is unified systems and safety engineering feasible?
This presentation introduces a new approach for developing composable systems with different SIL levels will be presented. It introduces the new notion of ARRL (Assured Reliability and Resilience Level).
Updating 50 year old infrastructure inspections with non destructive testing and robotics. Bridge inspection using nondestructive testing provides quantitative data to the department of transportation. This helps the DOT better allocate budgets and extend the service life of our critical infrastructure assets.
Final Year Engineering Project Seminar
For more information, check out my papers online:
Command controlled robot:
http://www.ijtre.com/manuscript/2014010976.pdf
Self controlled robot:
http://www.ijtre.com/manuscript/2014011008.pdf
Gesture controlled robot:
http://www.ijtre.com/manuscript/2014011107.pdf
Driving Efficiency with Splunk Cloud at Gatwick AirportSplunk
Gatwick Airport, the busiest single runway airport in the world, needed to ensure a high degree of efficiency for a record-breaking 925 daily flights and 38 million annual passengers. This presentation covers how they:
- Combine historical fact with "in the moment" data and events to predict success or failure, enabling the operation to prevent issues before they occur
- Support other organisations (e.g., airlines and ground handlers) with dashboards to improve their performance
- Moved from "how did we do?" to "how are we doing?" and are on the edge of answering "How will we do?”
- Plan to expand the use of Splunk Cloud in the future: tracking travel disruption, predicting passenger flow and getting real-time feedback via social media monitoring
Also, learn why a cloud solution gives Gatwick Airport the agility and scalability to achieve what they need.
Unified Systems Engineering feasibilityEric Verhulst
Is unified systems and safety engineering feasible?
This presentation introduces a new approach for developing composable systems with different SIL levels will be presented. It introduces the new notion of ARRL (Assured Reliability and Resilience Level).
Updating 50 year old infrastructure inspections with non destructive testing and robotics. Bridge inspection using nondestructive testing provides quantitative data to the department of transportation. This helps the DOT better allocate budgets and extend the service life of our critical infrastructure assets.
Final Year Engineering Project Seminar
For more information, check out my papers online:
Command controlled robot:
http://www.ijtre.com/manuscript/2014010976.pdf
Self controlled robot:
http://www.ijtre.com/manuscript/2014011008.pdf
Gesture controlled robot:
http://www.ijtre.com/manuscript/2014011107.pdf
Driving Efficiency with Splunk Cloud at Gatwick AirportSplunk
Gatwick Airport, the busiest single runway airport in the world, needed to ensure a high degree of efficiency for a record-breaking 925 daily flights and 38 million annual passengers. This presentation covers how they:
- Combine historical fact with "in the moment" data and events to predict success or failure, enabling the operation to prevent issues before they occur
- Support other organisations (e.g., airlines and ground handlers) with dashboards to improve their performance
- Moved from "how did we do?" to "how are we doing?" and are on the edge of answering "How will we do?”
- Plan to expand the use of Splunk Cloud in the future: tracking travel disruption, predicting passenger flow and getting real-time feedback via social media monitoring
Also, learn why a cloud solution gives Gatwick Airport the agility and scalability to achieve what they need.
Service Mesh - kilometer 30 in a microservice marathonMichael Hofmann
Distributed applications like microservices shift some of their complexities into the interaction of services. Such a service mesh, which can have hundreds of runtime instances, is very difficult to manage. You will be concerned with some of the following questions: Which service will be requested by which other services in which version and how often depending on the request content? How can you test the interaction and how can you replace single services with new ones?
These and other questions will be discussed in this session. Tools to make your live easier with a service mesh will also be introduced.
Making Runtime Data Useful for Incident Diagnosis: An Experience ReportQAware GmbH
QuASD/PROFES 2018, Wolfsburg: Talk by Marcus Ciolkowski (@M_Ciolkowski, Principal IT Consultant at QAware) and Florian Lautenschlager (@flolaut, Senior Software Engineer)
=== Please download slides if blurred! ===
Abstract: Important and critical aspects of technical debt often surface at runtime only and are difficult to measure statically.
This is a particular challenge for cloud applications because of their highly distributed nature.
Fortunately, mature frameworks for collecting runtime data exist but need to be integrated.
In this paper, we report an experience from a project that implements a cloud application within Kubernetes on Azure.
To analyze the runtime data of this software system, we instrumented our services with Zipkin for distributed tracing; with Prometheus and Grafana for analyzing metrics; and with fluentd, Elasticsearch and Kibana for collecting, storing and exploring log files.
However, project team members did not utilize these runtime data until we created a unified and simple access using a chat bot.
We argue that even though your project collects runtime data, this is not sufficient to guarantee its usage: In order to be useful, a simple, unified access to different data sources is required that should be integrated into tools that are commonly used by team members.
Get the research paper: http://bitly.com/2QmSNwl
The Internet of Things (IoT) can be considered as a modern manifestation of Mark Weiser’s classic vision of ubiquitous computing where tiny networked computers become part of everyday objects interweaving the virtual world and the physical world. The concept of the IoT originated some 15 years ago from linking real-world artifacts to virtual counterparts through radio-frequency identification (RFID) tags. More recently, environments have become ‘smart’ by augmenting physical objects with sensing or actuation capabilities and networking them with digital services. The ongoing standardization of Internet protocols for such IoT devices enables the seamless integration of smart things into the Internet. This trend is expected to eventually result in hundreds of billions of connected devices that need to be programmed, managed, and maintained. It has been shown that Web technology can significantly ease this process by providing well-known patterns and tools for developers and users. The existing solutions are, however, often too heavyweight for highly resource-constrained IoT devices. Indeed, most connected devices are expected to remain resource-constrained, as progress in technology witnessed by Moore’s Law is primarily leveraged to minimize dimensions, power consumption, and unit costs.
This dissertation presents a comprehensive solution for the seamless integration of highly resource-constrained IoT systems into the World Wide Web. Our thesis is that existing protocols and programming models do not effectually meet the needs of the IoT. We identify two key challenges for the vision to succeed: application-layer interoperability and improved usability for both developers and users. Both requirements can be met by an approach that amalgamates results from the field of Wireless Sensor Networks and the World Wide Web. This leads to the research questions (i) how to scale Web technology down to resource-constrained devices, (ii) how to scale it up to hundreds of billions of devices, and (iii) how to use it to improve the usability of the tiny networked computers. Our work addresses the resulting challenges with the following contributions: Being actively involved in the design and standardization of the Constrained Application Protocol (CoAP) within the Internet Engineering Task Force (IETF), we (i) evaluate the new Web protocol in the different components of IoT systems, namely resource-constrained devices, Cloud-based services, and user interaction. Based on this, we (ii) propose system architectures and guidelines for an optimal implementation and utilization of CoAP. Furthermore, we (iii) present concepts and tools for Web-like software development for the IoT. To supportour thesis, we also (iv) provide working open source implementations of our concepts, which build the basis for several IoT projects in academia and industry.
Building WebRTC based interesting features and services . WebRTC to stream from remote machine in IOT.
Details of Ramudroid a bot meant for cleaning outdoors uses webrtc stream for remote navigation .
What is a Service Mesh and what can it do for your MicroservicesMatt Turner
e’ll explore what a service mesh is and what it can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we’ll find out!
This session will also briefly cover how a service mesh works, giving us a mental model with which to explore and evaluate after the talk. Matt will show a simple installation and demo, giving us all the knowledge to go home and try for ourself.
5 Tips for Better Water Quality Monitoring with ASVsXylem Inc.
Automation is a disrupting force throughout the world, impacting our lives and creating a new normal for generations to come.
Join industry experts Kevin Simpson, Geoff Douglass, and Rob Porrett as they discuss automated vehicles for monitoring water quality and quantity – and the impact on the environmental community.
In this webinar, we provide a foundational understanding of Autonomous Surface Vehicles (ASV), share our top tips for better water monitoring, discuss the impact autonomous vehicles have on the environmental community, and introduce our latest and greatest water monitoring technology.
Watch this webinar to learn:
• Top tips to improve your water quality monitoring
• Best applications for an Autonomous Surface Vehicles (ASV)
• Real-world data examples and challenges
• A peek at our newest ASV technology—HYCAT
• And more!
Mobile IoT Middleware Interoperability & QoS Analysis - Eclipse IoT Day Paris...Nikolaos Georgantas
Research results by the Inria Paris MiMove Team on Mobile IoT Middleware Interoperability & QoS Analysis. Presentation at Eclipse IoT Day Paris Saclay 2019
Leveraging the strength of OSGi to deliver a convergent IoT Ecosystem - O Log...mfrancis
The “internet of things” is the next revolutionary wave following profound changes brought to us by Personal Computers (connecting places) and Mobile Phones (connecting people on the go). This third wave heralds the beginning of the new era of pervasive connectivity, embedded intelligence, and application convergence. It will be the world where smart things will communicate among themselves and with us enabling greener, more efficient, and at the same time more comfortable environment.
This talk will present a platform and products designed to serve the new markets enabled by the Internet of Things, with a particular focus on the value of the OSGi framework enabling convergence of Home Automation, Smart Energy, Electric Vehicle Charging, and e-health on a single remotely manageable platform. It will also provide insights on how the platform was developed leveraging the extensibility offered by the OSGi framework and ProSyst’s modular architecture.
The built-in OSGi stack provides Java-level abstraction of the network interfaces and Smart Energy Profile 2.0 stack as well as cloud integration features such as web server, web services and standards-based remote management. The OSGi framework is the key enabler of the product lifecycle and remote application management mandatory for service provider driven deployments. The Smart Energy 2.0 standard is a key element of the future smart grid. And the work presented in this talk describes the first platform integrating the SEP 2.0 protocol stack with an OSGi based middleware. The OSGi based solution also provides higher level of device security through the use of secure element. The UDK-21 is build around a System-on-Chip STreamPlug (ST2100), the solution features a fully integrated HomePlug PHY/MAC and Analog Front End combined with the ARM926EJ-S processor and a rich set of interfaces.
A demo showing Smart Energy Profile 2.0 use cases will outline these features. The demo will show how web based applications can interact with the OSGi stack on the already publicly available UDK-21 based gateway to control remote devices, such as a thermostat or an electric load. The access to SEP 2.0 devices will be done by the means of JSON-RPC based APIs, independent of the underlying device protocol, hence highlighting the benefits of a generic protocol agnostic architecture from the application standpoint. Other examples of the products that can be built around UDK-21 include Electric Vehicle Charger, Smart Meter, and a Basement Sensor Hub.
Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.
• Overall 5 years of IT Experience which include experience in the field of RBT in HSIT (Hardware software integration testing), SWI (Software integration testing), UT (Unit testing) Experienced in the Software Verification activities for safety critical systems in compliance with DO-178B standards
More Related Content
Similar to Harvard HKUST 2015 - Final Presentation
Service Mesh - kilometer 30 in a microservice marathonMichael Hofmann
Distributed applications like microservices shift some of their complexities into the interaction of services. Such a service mesh, which can have hundreds of runtime instances, is very difficult to manage. You will be concerned with some of the following questions: Which service will be requested by which other services in which version and how often depending on the request content? How can you test the interaction and how can you replace single services with new ones?
These and other questions will be discussed in this session. Tools to make your live easier with a service mesh will also be introduced.
Making Runtime Data Useful for Incident Diagnosis: An Experience ReportQAware GmbH
QuASD/PROFES 2018, Wolfsburg: Talk by Marcus Ciolkowski (@M_Ciolkowski, Principal IT Consultant at QAware) and Florian Lautenschlager (@flolaut, Senior Software Engineer)
=== Please download slides if blurred! ===
Abstract: Important and critical aspects of technical debt often surface at runtime only and are difficult to measure statically.
This is a particular challenge for cloud applications because of their highly distributed nature.
Fortunately, mature frameworks for collecting runtime data exist but need to be integrated.
In this paper, we report an experience from a project that implements a cloud application within Kubernetes on Azure.
To analyze the runtime data of this software system, we instrumented our services with Zipkin for distributed tracing; with Prometheus and Grafana for analyzing metrics; and with fluentd, Elasticsearch and Kibana for collecting, storing and exploring log files.
However, project team members did not utilize these runtime data until we created a unified and simple access using a chat bot.
We argue that even though your project collects runtime data, this is not sufficient to guarantee its usage: In order to be useful, a simple, unified access to different data sources is required that should be integrated into tools that are commonly used by team members.
Get the research paper: http://bitly.com/2QmSNwl
The Internet of Things (IoT) can be considered as a modern manifestation of Mark Weiser’s classic vision of ubiquitous computing where tiny networked computers become part of everyday objects interweaving the virtual world and the physical world. The concept of the IoT originated some 15 years ago from linking real-world artifacts to virtual counterparts through radio-frequency identification (RFID) tags. More recently, environments have become ‘smart’ by augmenting physical objects with sensing or actuation capabilities and networking them with digital services. The ongoing standardization of Internet protocols for such IoT devices enables the seamless integration of smart things into the Internet. This trend is expected to eventually result in hundreds of billions of connected devices that need to be programmed, managed, and maintained. It has been shown that Web technology can significantly ease this process by providing well-known patterns and tools for developers and users. The existing solutions are, however, often too heavyweight for highly resource-constrained IoT devices. Indeed, most connected devices are expected to remain resource-constrained, as progress in technology witnessed by Moore’s Law is primarily leveraged to minimize dimensions, power consumption, and unit costs.
This dissertation presents a comprehensive solution for the seamless integration of highly resource-constrained IoT systems into the World Wide Web. Our thesis is that existing protocols and programming models do not effectually meet the needs of the IoT. We identify two key challenges for the vision to succeed: application-layer interoperability and improved usability for both developers and users. Both requirements can be met by an approach that amalgamates results from the field of Wireless Sensor Networks and the World Wide Web. This leads to the research questions (i) how to scale Web technology down to resource-constrained devices, (ii) how to scale it up to hundreds of billions of devices, and (iii) how to use it to improve the usability of the tiny networked computers. Our work addresses the resulting challenges with the following contributions: Being actively involved in the design and standardization of the Constrained Application Protocol (CoAP) within the Internet Engineering Task Force (IETF), we (i) evaluate the new Web protocol in the different components of IoT systems, namely resource-constrained devices, Cloud-based services, and user interaction. Based on this, we (ii) propose system architectures and guidelines for an optimal implementation and utilization of CoAP. Furthermore, we (iii) present concepts and tools for Web-like software development for the IoT. To supportour thesis, we also (iv) provide working open source implementations of our concepts, which build the basis for several IoT projects in academia and industry.
Building WebRTC based interesting features and services . WebRTC to stream from remote machine in IOT.
Details of Ramudroid a bot meant for cleaning outdoors uses webrtc stream for remote navigation .
What is a Service Mesh and what can it do for your MicroservicesMatt Turner
e’ll explore what a service mesh is and what it can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we’ll find out!
This session will also briefly cover how a service mesh works, giving us a mental model with which to explore and evaluate after the talk. Matt will show a simple installation and demo, giving us all the knowledge to go home and try for ourself.
5 Tips for Better Water Quality Monitoring with ASVsXylem Inc.
Automation is a disrupting force throughout the world, impacting our lives and creating a new normal for generations to come.
Join industry experts Kevin Simpson, Geoff Douglass, and Rob Porrett as they discuss automated vehicles for monitoring water quality and quantity – and the impact on the environmental community.
In this webinar, we provide a foundational understanding of Autonomous Surface Vehicles (ASV), share our top tips for better water monitoring, discuss the impact autonomous vehicles have on the environmental community, and introduce our latest and greatest water monitoring technology.
Watch this webinar to learn:
• Top tips to improve your water quality monitoring
• Best applications for an Autonomous Surface Vehicles (ASV)
• Real-world data examples and challenges
• A peek at our newest ASV technology—HYCAT
• And more!
Mobile IoT Middleware Interoperability & QoS Analysis - Eclipse IoT Day Paris...Nikolaos Georgantas
Research results by the Inria Paris MiMove Team on Mobile IoT Middleware Interoperability & QoS Analysis. Presentation at Eclipse IoT Day Paris Saclay 2019
Leveraging the strength of OSGi to deliver a convergent IoT Ecosystem - O Log...mfrancis
The “internet of things” is the next revolutionary wave following profound changes brought to us by Personal Computers (connecting places) and Mobile Phones (connecting people on the go). This third wave heralds the beginning of the new era of pervasive connectivity, embedded intelligence, and application convergence. It will be the world where smart things will communicate among themselves and with us enabling greener, more efficient, and at the same time more comfortable environment.
This talk will present a platform and products designed to serve the new markets enabled by the Internet of Things, with a particular focus on the value of the OSGi framework enabling convergence of Home Automation, Smart Energy, Electric Vehicle Charging, and e-health on a single remotely manageable platform. It will also provide insights on how the platform was developed leveraging the extensibility offered by the OSGi framework and ProSyst’s modular architecture.
The built-in OSGi stack provides Java-level abstraction of the network interfaces and Smart Energy Profile 2.0 stack as well as cloud integration features such as web server, web services and standards-based remote management. The OSGi framework is the key enabler of the product lifecycle and remote application management mandatory for service provider driven deployments. The Smart Energy 2.0 standard is a key element of the future smart grid. And the work presented in this talk describes the first platform integrating the SEP 2.0 protocol stack with an OSGi based middleware. The OSGi based solution also provides higher level of device security through the use of secure element. The UDK-21 is build around a System-on-Chip STreamPlug (ST2100), the solution features a fully integrated HomePlug PHY/MAC and Analog Front End combined with the ARM926EJ-S processor and a rich set of interfaces.
A demo showing Smart Energy Profile 2.0 use cases will outline these features. The demo will show how web based applications can interact with the OSGi stack on the already publicly available UDK-21 based gateway to control remote devices, such as a thermostat or an electric load. The access to SEP 2.0 devices will be done by the means of JSON-RPC based APIs, independent of the underlying device protocol, hence highlighting the benefits of a generic protocol agnostic architecture from the application standpoint. Other examples of the products that can be built around UDK-21 include Electric Vehicle Charger, Smart Meter, and a Basement Sensor Hub.
Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.
• Overall 5 years of IT Experience which include experience in the field of RBT in HSIT (Hardware software integration testing), SWI (Software integration testing), UT (Unit testing) Experienced in the Software Verification activities for safety critical systems in compliance with DO-178B standards
Similar to Harvard HKUST 2015 - Final Presentation (20)
23. 23
Qt Creator - Powerful cross-platform GUI design tool
MAVLink - Micro Air Vehicle Communication Protocol
Deployable to Android devices and potentially to iOS
30. 30
MAVLink
To reduce latency, we tried to limit the transmission datastream by making smart
choices of messages. We picked 9 out of more than 200 MAVLink messages:
#1 heartbeat: time information
#2 system status: battery voltage and percentage
#30 attitude: row, pitch, yaw and attitude rates
#32 local position: x, y, z, 3-axis velocity and 3-axis acceleration
#33 global position: latitude, longitude, relative altitude
#105 high resolution IMU
#253 status text: drone flight log
#69 manual_control: direct flight commands and mode switch
#76 long command: arm and disarm
31. 31
Features
Computer Ground Control Station (GCS) has
● a serial port for radio telemetry to communicate with UAV
● a TCP server to connect the Android app and UAV together
● a flexible mode switch panel
● a display of real-time flight data
● a console to display flight log
● four sliders for manual control of row, pitch, yaw, throttle and fast key
Android Application has:
● a TCP client to connect to the server via IP address and port number
● functional manual control in the local area network (LAN)
34. 34
User Experience Survey (Application)
Summary of the User Experience Survey Results, Survey Size N=12
Average Score (out of 10) Standard Deviation
Ease of ordering (Customer) 8.8 1.1
Usefulness of the Map (Customer) 7.8 2.3
Ease of delivering (Vendor) 9.2 0.7
Ease of checking pending orders
(Vendor)
9.7 0.6
Ease of checking battery status
(Vendor)
8.6 1.4
Overall 8.8 /
38. Inter-Process Communication
——uORB
38
● Firmware onboard is consisted of multiple processes
● uORB stands for “Micro Object Request Broker”
● Is a mechanism for processes onboard to “communicate”
● Data packed into C structs and published/subscribed to
relevant “topics”
Sensor Process
Attitude Controller
Process
topic 3(irrelevant)
topic 2(manual control)
topic 1(sensor values )
40. 40
Controlling by Mobile/PC Terminal
How the Radio Controller works
Radio signal is decoded into several values including: Roll, Pitch, Yaw and Throttle
Then published to the MANUAL_CONTROL_SETPOINT topic
Subscribed by the Position Controller Process
Position Controller Process maneuvers the drone
Transmit Decode Roll
Pitch
Yaw
Throttle
Publish Position
Controller
Process
MANUAL_
CONTROL_
SETPOINT topic
Subscribe
41. 41
Controlling by Mobile/PC Terminal
How our PC/Mobile Control works:
Instead of going through the decoding procedure, we send desired Roll, Pitch, Yaw, and
Throttle information directly to the mainboard using MAVlink Protocol
Mavlink
Roll
Pitch
Yaw
Throttle
Position
Controller
Process
MANUAL_
CONTROL_
SETPOINT topic
SubscribePublish
42. Delivery Route
3. Descend
2. Fly to Destination
5. Return Home
6. Shutoff 4. Dropoff
Launch
Site
navigation_main.cpp
commander.cpp
MAIN_STATE_DELIVERY
NAVIGATION_STATE_DELIVERY
delivery.cpp
Delivery
Address
1. Preflight
43. Proximity Sensor Specs
Using the MB-1240 ultrasonic proximity sensor to detect barrier
Accurate sensing range from 20cm to 600cm
Sensor gives out analog output ranging from 0-3.3V
ADC sampling function runs in System Ticks
Sample rate of 100Hz
43
Barrier Avoidance Feature
49. Mechanical Design Evolution
49
Landing Gear: Design Analysis
More bending at all 4 corners
Cause UAV to topple
Redundant central part
Styrofoam supports at the four ends
Extended out for better balance
Easy attachment
50. Mechanical Design Evolution
50
Beautiful and robust
heavy: 600g
complex: hard to 3d print
and manufacture
Simpler, less complex,
still heavy: 250g
easy to manufacture but
fragile
Simpler, robust,
light (weight: 70g)
easy to manufacture,
And beautiful!
UAV Shell: Design Evolution
53. 53
Performance Testing
Flight Speeds on a 57 m Trajectory
Trial Time in Flight
(s)
Mean Speed
(m/s)
1 8.5 6.7
2 8.3 6.8
3 7.8 7.3
Average 8.2 6.9
Battery Life Testing
Trial 3300mAh
(320g)
5000mAh
(541g)
1 (Min:Sec) 15:20 20:43
2 (Min:Sec) 15:21 20:09
Average 15:21 20:26
57. Increasing payload to 1 kg (~ the weight of 15” pizza)
Optimizing battery usage for 30 minutes
Increasing delivery mileage to 3 km
Improving the communication system range
Improving accuracy of drone tracking for door deliveries
Developing more comprehensive object avoidance
More than just ascending
57
Future Work
58. Acknowledgements
Evelyn Hu, Ling Shi, and Kei May Lau, thank you for creating and nurturing the Harvard-HKUST
program. Without you, none of us would be here right now enjoying this enriching experience
of international collaboration.
Many thanks to our mentors Chris, Daniela, Avi, Lina, Xuan, and Bing, We greatly appreciated your
advice, guidance, and all those cross-time-zone Skype sessions. We couldn’t have done it
without your support!
Sarah and Patricia, thank you for ensuring all of our needs were met in Cambridge and Hong
Kong and going above and beyond in taking care of us.
The Harvard-HKUST Design Team is grateful for the financial support of Harvard alumnus Xiang
Dong “XD” Yang and Nancy Yang, the Harvard School of Engineering and Applied Sciences, the
Hong Kong University of Science and Technology, and the Harvard President’s Innovation
Fund for International Experiences.
58
66. Appendix II
http://www.gizmag.com/flirtey-drones-deliver-medicine-in-us-
first/38102/
The citizens of Wise County, Virginia lack access to proper healthcare
Previously, medical supplies were delivered each year to the area by truck
CEO Sweeney says, "they said that if there was a way they could receive the
medication quicker that would help"
Sweeney hopes this proof of concept whereby the benefits of drone delivery are
realized, albeit on a small scale, will act as a catalyst for the delivery of everyday
items.
66
68. 68
Feedback:
Evelyn:
-Move functional
requirements to the
beginning
-explain what we wanted to
create at the very beginning
-use more words
-lead up to the video, don’t
use it in the first slide
-tell audience what the hard
parts of the project were
-introduce very basics of
project at the start
-make it sound difficult
-slide 22 is a good slide,
but needs more time to
explain it
-audience needs a little
Chris
-switching speakers too
frequently
-group the slides so that one
person can talk for a longer
period of time
-loud background music took
away from narration
-explain what is impressive in
the videos
-talk to someone who has
not been involved with the
project
-slides at the end should
explain what we
accomplished relative to the
technical specs we listed at
Avi
-good groupings, but
missing out on
explaining context
-start with the why
-do not talk about
technical specifications
until slide 50
-how did we work in
existing firmware and
glob of code?
questions
-why does
only go up
reaches an
Could this
in areas lik
-why does
gear not g
height to a
payload?
-what is m
some text
written def
Editor's Notes
Erin Intro
Introduce ourselves
Hi, we're Harvard-HKUST Design Team and this summer we spent eight weeks building a drone delivery system.
Go around, do names quickly.
Erin
Erin
Erin
Ability to deliver an object with excellent stability and reliability
Low drop rate, sufficient load capacity
Efficient and convenient delivery system
High speed, user-friendly interface & operation
Low operation cost
Elvis
Load Capacity: ~ 400 grams
Maximum Delivery Range: ~ 1 mile (w/ 3 meters accuracy)
Average Flight Speed: 7 m/s
Maximum Flight Speed: ?? m/s
Positioning Accuracy (On App.): ~ 45 meters
Elvis
Maximum Communication Range: ?? meters
Note: the telemetry range is from 2km to 4km but the wifi connection really depends. I put the number 1.6km here only because the service is supposed to cover a circle with 1 mile radius. Please feel free to edit it
Battery Durability (At Full Load): ~ 20 minutes
Maximum Allowable Drop Rate: ~ 0.2% -> 1/500
Current UPS loss rate: 0.5 - 1.0%
Elvis
$3.66 - average price customers are willing to pay for delivery within 10 miles
63.6% of users order from within 2 miles
They expect to wait an average of 59.5 minutes for delivery within 10 miles
Many businesses don't offer delivery because of the low size and cost of their available inventory
They would like lower cost, more reliable options
Most make deliveries within 1 mile [Cambridge businesses]
Elvis
There are two main problems with current delivery systems:
Long waits for items ordered within short distances
Unknown wait times
Billie
Why did we choose to create a drone delivery system?
Billie
Today, almost 2 years after Amazon first revealed its project for drone delivery,
these systems are set to revolutionize commerce by lowering shipping costs and speeding up delivery for customers worldwide
but are also going to play a transformative role in fields ranging from
urban management to farming to coastal security to disaster relief
Recently on July 17th, the FAA, Federal Aviation Administration has approved Flirtey, a drone that delivers medical supplies to isolated coal mining regions in Virginia.
Although PrimeAir is still awaiting FAA approval, the path the industry has taken convinced us that it would be an interesting, relevant, and sufficiently complex a task to implement a drone delivery system for the smaller, local businesses that we know and interact with on a daily basis.
Brian
This is DJI Phantom 3. The DJI Phantom series have already been more or less unchallenged as the best all-in-one camera drone on the market.
It is small, light and safe. It can capture high-quality photos and videos.
It is stable in both indoor and outdoor environments, even in a fairly strong breeze.
It is a fully integrated, user-friendly, brilliant Product!
Brian
This brings us to our quadcopter design. We have not had the two plus years or resources that Amazon has had, but in our 2 months we have done some remarkable things in the ability to navigate and control the quadcopter from multiple platforms. In this growing market of quadcopter we envision ours as being the quadcopter for small businesses. It can revolutionize the way small businesses connect to their customers through a faster, more reliable, and very exciting delivery process.
Brian
To demonstrate a little bit of the flight capabilities of our quadcopter, we have put together this video.
Vinh
Now that we’ve talked a little bit about what we’ve been building and why we’re doing it, let’s talk about who this delivery system benefits.
-Vinh
Young professionals need an efficient, affordable delivery system to deliver goods from local stores due to a busy schedule. They don’t have time to go to the store to buy what the want, and can’t wait around all day for a delivery. They are willing to pay for convenience.
A tech-savvy business owner wants to target local professionals during the work day with a new delivery system with more predictable times and lower operation costs.
Vinh
With a clearer goal in mind, we set out on our design process.
First we did research on who was had the most cutting edge tehnology in drones and delivery (amazon, DJI)
Next, with our target customers in mind, we went out conducting surveys on real businesses and people matching our target persona to get a sense of what the valued in a delivery system.
Afterwards, we set out to build a prototype of a delivery system that met consumer need in a novel way.
We then proceeded to test and get user feedback on our system, and used those results to improve our system
Billie
Now we will delve further into the software side
Billie
We developed the application using MAVLink and Qt Creator.
The Pixhawk autopilot that came with our system uses MAVLink library
We wanted to use the same library as the on-board firmware so we choose to program in C/C++.
We chose to use Qt Creator ::::: a leading C++ development environment
Now let's go into a further breakdown of the application structure.
We built the user interface using Qt Creator which supports QML
a JavaScript-based, declarative language
We implemented tracking through the use of the QtLocation and Qt Positioning APIs
We designed the application responsively → so it would adapt to multiple screens and operating systems
The functions in the application are supported by C/C++ and a collection of page handlers that call MAVLink through serial port on the computer side / and through WiFi on the tablet side.
For the communication aspect, we used
MAVLink
Serial port, a form of radio telemetry
And upon discovering compatability issues with serial port and Android deployment, we also implemented WiFi
Eva will discuss this further later on.
Billie
This process was critical because it made us think about how to best implement drone control on a 2D tablet interface vs. the 3-D RC controller
To create the user interface, we went through a series of paper prototypes
sketched out how we wanted the interface to look
split the application into a vendor / customer side
mapped out the process of how our vendors / customers would walk through the application.
added error interfaces for potential flight issues, order issues, weight limits, etc.
added resolution interfaces.
Billie:
Here we have a short clip of the TRACKING FEATURE of the application that allows vendors and customers to see where the drone currently is.
Shown is a flight test on the lawn
The red circle shows the drone's current location.
Billie
Here are the predicted vs. actual flight paths from the video.
As you can see, the drone took more of a circuitous path from its starting point in blue to the destination point in red, probably due to wind conditions and also some GPS inaccuracies.
Billie
The tracking feature that you saw was developed using the QtLocation and QtPositioning APIs.
To walk you through how this happens:
Enter the address on the UI
GeoCodeModel that generates the coordinates for the address the customers type in.
The map object is created using Open Street Maps plugin
This map object is populated by MAPITEMVIEW which places the MapCircles and MapPolylines on the map
Now EVA will speak further on communication between the APPLICATION and DRONE
Eva:
Basically in the network, WiFi works between android app and computer while 433MHz radio telemetry works between computer and UAV.
Computer is critical in this system.
Firstly, it runs a TCP server for the android application. It answers data request and sends GPS and battery information to the app. Secondly, the computer is also a ground control station for the UAV.
Pixhawk board on the UAV can pack C-structs over serial channels with high efficiency and send these packets to the ground control station. The computer use MAVLink library to decode packets and extract information
Our original design is to connect the telemetry with phone directly. But in current stage QtSerialPort library only supports computer operating systems and currently is not workable for Android or iOS. We look forward to a newer version of Qt which officially supports serial port module. By that time, we can use the android app to communicate with the uav directly.
Now our design is to use computer as a signal transfer center. It has its own advantages:
1.as processing and analysing real-time data take serious power.
Computer uses radio telemetry to communicate with UAV. The android application, as a TCP client, connects to the TCP server running by computer and communicates with the drone indirectly.
In this stage, we let the computer work between the android and UAV
why do we use wifi
To use the same library as the on-board control firmware, we choose to programme in C/C++. Not java. We use Qt creator.Furthermore, computer has a bigger screen and can show us more data. Now we have monitor-only mode, computer control mode and app control mode. We can use keyboard and mouse at the same time in computer control, which is very convenient.
Eva
In human natural language communication, we care more about what we want to say
while in machine language communication, we have to decide what we can say.
To reduce transmission latency, we try to limit the datastream by making smart choice of messages.
We pick 9 messages out of more than 200 MAVLink messages and they are listed here
For example, let’s say message number 2 system status. We need to read the battery voltage and remaining battery percentage. We also try to design our own battery life estimation function to see how far it can fly.
Eva
Make sure all android tablets have the dronedeliveryapplication
3 modes: monitor-only, computer control, app control
Eva:
GCS:
The GCS has to be professional, as processing and analyzing flight data in real time takes serious power. The computer GCS collects data from the entire system, including GPS location, data from automatic sensors and command inputs. It cooperates with onboard control to make sure that the drone behaves well in all moments.
Another core function is communication. It runs a server to connect the app and drone together
Control: mouse and keyboard.
APP: in this clip of video, we run the android app on Nexus 9 device.
1. Customers can place an order and track the order
2. The vendor can view customer orders and send out the delivery drone after checking battery
3. Manual control is enabled after the phone joins local area network
Brian
Our main focus in hardware can be broken down into 3 areas . . .
To take a jump back before delving into each of these, I would like to introduce where we started off on the control of the quadcopter *Change to next slide*
Brian
To demonstrate where we started in our control of the quadcopter, we can talk about the difficulties we had flying the drone. Specifically, the incident with the tree.
To quickly summarize, we lost control of the copter and got it stuck in a tree ~60ft or 20m in the air. Now, our control of the drone has progressed to where we can fly it safely with a remote control, computer, or mobile device. It can perform automatic deliveries and detect objects in its trajectory. And now Gary will help introduce how we managed to accomplish all of this
Gary
To pave a way for the audience to better understand our control mechanism
The entire firmware onboard is running with multiple processes at the same time (e.g. LED, Motor control, etc)
A process might need information generated by another
Wrap up information by means of structs
A publish and subscribe mechanism
Gary
Two advantage: without global variable, not messy
allow for us to define owm topic, provide flexibility on app development
-Gary
In order for the user to control the drone conveniently, we wanna be able to control by a personal terminal
-Gary
Brian
We have walked through how the delivery works for both the customer and vendor, but a third equally important character in this process is the drone. The delivery route can be broken down into 6 different stages. The preflight stage sets the mission items for the route. This includes the GPS coordinates for the delivery address. The drone will then be flying to its destination, comparing its current location to the destination and flying in a straight line to the destination. At the destination it will descend to a low altitude to dropoff the package. This is done to softly drop the item and prevent breaking. After it drops the item off, the drone will return home, land, and finally disarm itself.
This is all accomplished with the flip of a switch on the RC or a click of a button on the app. This is done through the following code. There are two important set of states for the drone to be in. The first is the main state, which determines how the navigation states are set. The navigation states are a collection of navigation routines that the drone can move through. They set various flight requirements, like requiring GPS lock, grant permissions, and set control modes for the flight routine to run. The commander.cpp app controls how the main and navigation states are set, while navigation_main.cpp receives the navigation state and activates the delivery.cpp file. The delivery.cpp file is the implementation of the delivery class, which directs the quadcopter through the delivery process. It does this by setting a series of mission_items and polling location data to help it move through the 6 different stages outlined to the left.
Gary
System Ticks is kinda like a real time clock onboard
Gary
Using the MB-1240 ultrasonic proximity sensor to detect barrier
Accurate sensing range from 20cm to 600cm
Sensor gives out analog output ranging from 0-3.3V
ADC sample function runs in System Ticks
Sample rate of 100Hz
Vinh
To briefly describe the gripper, it is a servo connected to a 4cm plastic jaw that will hold a package in place during flight. On delivery, the servo will turn and release the package. It has a torque of 1.8 kg*cm, and so can carry a max of 450g
In terms of powering the gripper, it must be powered and controlled from the AUX output. The PixHawk has 8 MAIN pins and 6 AUX pins, which can send PWM signals out. However, it doesn’t output enough power to actually drive the motors. To do this we need a driver to power the motor’s motion. In this case, a voltage converter is used to take the battery’s voltage and output 5V to AUX. The servo is connected to another AUX pin. Positions shown in diagram, The code control process is shown to the right.
Now that we’ve talked about how the gripper operates, let’s go through the echanical design process that led to this gripper.
-Brian
Here is a video to visually demonstrate our drone delivery system
To start, a customer will place an order using our mobile app. They can select the items they want and then enter in their address and billing information
The vendor will then review this order, check the battery level, and be able to start the delivery with the app
When the drone has been prepared with a charged battery and the requested items, the delivery routine can be activated from the app
If the drone encounters an obstacle during its route, it will stop and ascend until it has cleared the obstacle before continuing on its mission
The progress of the drone can be tracked in the app by both the vendor and the customer while it is completing the delivery
When it reaches the delivery address, it will descend and activate the servo to release the package
The drone will then return home and the customer can retrieve their package once the drone is a safe distance away
Upon coming home, the vendor can get the drone ready with another battery for a new delivery
Some of you may have noticed that the drone flew backwards after the object avoidance part of the video. This was done because we have not actually run a delivery routine with an object to avoid. It is still very likely that the drone will damage itself if it is moving full speed towards an object, so that clip was from us testing the drone at a low velocity against the side of the building. Vinh will now walk us through, quantitatively, how we compared to our initial goals.
Vinh
Vinh
Load Capacity: ~ 400 grams
Maximum Delivery Range: ~ 1 mile (w/ 3 meters accuracy)
Average Flight Speed: 7 m/s
Maximum Flight Speed: ?? m/s
Positioning Accuracy (On App.): ~ 45 meters
Vinh
Maximum Communication Range: ?? meters
Note: the telemetry range is from 2km to 4km
Battery Durability (At Full Load): ~ 20 minutes
Maximum Allowable Drop Rate: ~ 0.2% -> 1/500
Current UPS loss rate: 0.5 - 1.0%
Erin
Erin
Erin
Now we will open up the floor to some questions.
Billie
And now, for a fun video of all our adventures over the summer.
Please follow us outside to the lawn for a live demonstration of delivery.