23. 23
Qt Creator - Powerful cross-platform GUI design tool
MAVLink - Micro Air Vehicle Communication Protocol
Deployable to Android devices and potentially to iOS
30. 30
MAVLink
To reduce latency, we tried to limit the transmission datastream by making smart
choices of messages. We picked 9 out of more than 200 MAVLink messages:
#1 heartbeat: time information
#2 system status: battery voltage and percentage
#30 attitude: row, pitch, yaw and attitude rates
#32 local position: x, y, z, 3-axis velocity and 3-axis acceleration
#33 global position: latitude, longitude, relative altitude
#105 high resolution IMU
#253 status text: drone flight log
#69 manual_control: direct flight commands and mode switch
#76 long command: arm and disarm
31. 31
Features
Computer Ground Control Station (GCS) has
● a serial port for radio telemetry to communicate with UAV
● a TCP server to connect the Android app and UAV together
● a flexible mode switch panel
● a display of real-time flight data
● a console to display flight log
● four sliders for manual control of row, pitch, yaw, throttle and fast key
Android Application has:
● a TCP client to connect to the server via IP address and port number
● functional manual control in the local area network (LAN)
34. 34
User Experience Survey (Application)
Summary of the User Experience Survey Results, Survey Size N=12
Average Score (out of 10) Standard Deviation
Ease of ordering (Customer) 8.8 1.1
Usefulness of the Map (Customer) 7.8 2.3
Ease of delivering (Vendor) 9.2 0.7
Ease of checking pending orders
(Vendor)
9.7 0.6
Ease of checking battery status
(Vendor)
8.6 1.4
Overall 8.8 /
38. Inter-Process Communication
——uORB
38
● Firmware onboard is consisted of multiple processes
● uORB stands for “Micro Object Request Broker”
● Is a mechanism for processes onboard to “communicate”
● Data packed into C structs and published/subscribed to
relevant “topics”
Sensor Process
Attitude Controller
Process
topic 3(irrelevant)
topic 2(manual control)
topic 1(sensor values )
40. 40
Controlling by Mobile/PC Terminal
How the Radio Controller works
Radio signal is decoded into several values including: Roll, Pitch, Yaw and Throttle
Then published to the MANUAL_CONTROL_SETPOINT topic
Subscribed by the Position Controller Process
Position Controller Process maneuvers the drone
Transmit Decode Roll
Pitch
Yaw
Throttle
Publish Position
Controller
Process
MANUAL_
CONTROL_
SETPOINT topic
Subscribe
41. 41
Controlling by Mobile/PC Terminal
How our PC/Mobile Control works:
Instead of going through the decoding procedure, we send desired Roll, Pitch, Yaw, and
Throttle information directly to the mainboard using MAVlink Protocol
Mavlink
Roll
Pitch
Yaw
Throttle
Position
Controller
Process
MANUAL_
CONTROL_
SETPOINT topic
SubscribePublish
42. Delivery Route
3. Descend
2. Fly to Destination
5. Return Home
6. Shutoff 4. Dropoff
Launch
Site
navigation_main.cpp
commander.cpp
MAIN_STATE_DELIVERY
NAVIGATION_STATE_DELIVERY
delivery.cpp
Delivery
Address
1. Preflight
43. Proximity Sensor Specs
Using the MB-1240 ultrasonic proximity sensor to detect barrier
Accurate sensing range from 20cm to 600cm
Sensor gives out analog output ranging from 0-3.3V
ADC sampling function runs in System Ticks
Sample rate of 100Hz
43
Barrier Avoidance Feature
49. Mechanical Design Evolution
49
Landing Gear: Design Analysis
More bending at all 4 corners
Cause UAV to topple
Redundant central part
Styrofoam supports at the four ends
Extended out for better balance
Easy attachment
50. Mechanical Design Evolution
50
Beautiful and robust
heavy: 600g
complex: hard to 3d print
and manufacture
Simpler, less complex,
still heavy: 250g
easy to manufacture but
fragile
Simpler, robust,
light (weight: 70g)
easy to manufacture,
And beautiful!
UAV Shell: Design Evolution
53. 53
Performance Testing
Flight Speeds on a 57 m Trajectory
Trial Time in Flight
(s)
Mean Speed
(m/s)
1 8.5 6.7
2 8.3 6.8
3 7.8 7.3
Average 8.2 6.9
Battery Life Testing
Trial 3300mAh
(320g)
5000mAh
(541g)
1 (Min:Sec) 15:20 20:43
2 (Min:Sec) 15:21 20:09
Average 15:21 20:26
57. Increasing payload to 1 kg (~ the weight of 15” pizza)
Optimizing battery usage for 30 minutes
Increasing delivery mileage to 3 km
Improving the communication system range
Improving accuracy of drone tracking for door deliveries
Developing more comprehensive object avoidance
More than just ascending
57
Future Work
58. Acknowledgements
Evelyn Hu, Ling Shi, and Kei May Lau, thank you for creating and nurturing the Harvard-HKUST
program. Without you, none of us would be here right now enjoying this enriching experience
of international collaboration.
Many thanks to our mentors Chris, Daniela, Avi, Lina, Xuan, and Bing, We greatly appreciated your
advice, guidance, and all those cross-time-zone Skype sessions. We couldn’t have done it
without your support!
Sarah and Patricia, thank you for ensuring all of our needs were met in Cambridge and Hong
Kong and going above and beyond in taking care of us.
The Harvard-HKUST Design Team is grateful for the financial support of Harvard alumnus Xiang
Dong “XD” Yang and Nancy Yang, the Harvard School of Engineering and Applied Sciences, the
Hong Kong University of Science and Technology, and the Harvard President’s Innovation
Fund for International Experiences.
58
66. Appendix II
http://www.gizmag.com/flirtey-drones-deliver-medicine-in-us-
first/38102/
The citizens of Wise County, Virginia lack access to proper healthcare
Previously, medical supplies were delivered each year to the area by truck
CEO Sweeney says, "they said that if there was a way they could receive the
medication quicker that would help"
Sweeney hopes this proof of concept whereby the benefits of drone delivery are
realized, albeit on a small scale, will act as a catalyst for the delivery of everyday
items.
66
68. 68
Feedback:
Evelyn:
-Move functional
requirements to the
beginning
-explain what we wanted to
create at the very beginning
-use more words
-lead up to the video, don’t
use it in the first slide
-tell audience what the hard
parts of the project were
-introduce very basics of
project at the start
-make it sound difficult
-slide 22 is a good slide,
but needs more time to
explain it
-audience needs a little
Chris
-switching speakers too
frequently
-group the slides so that one
person can talk for a longer
period of time
-loud background music took
away from narration
-explain what is impressive in
the videos
-talk to someone who has
not been involved with the
project
-slides at the end should
explain what we
accomplished relative to the
technical specs we listed at
Avi
-good groupings, but
missing out on
explaining context
-start with the why
-do not talk about
technical specifications
until slide 50
-how did we work in
existing firmware and
glob of code?
questions
-why does
only go up
reaches an
Could this
in areas lik
-why does
gear not g
height to a
payload?
-what is m
some text
written def
Editor's Notes
Erin Intro
Introduce ourselves
Hi, we're Harvard-HKUST Design Team and this summer we spent eight weeks building a drone delivery system.
Go around, do names quickly.
Erin
Erin
Erin
Ability to deliver an object with excellent stability and reliability
Low drop rate, sufficient load capacity
Efficient and convenient delivery system
High speed, user-friendly interface & operation
Low operation cost
Elvis
Load Capacity: ~ 400 grams
Maximum Delivery Range: ~ 1 mile (w/ 3 meters accuracy)
Average Flight Speed: 7 m/s
Maximum Flight Speed: ?? m/s
Positioning Accuracy (On App.): ~ 45 meters
Elvis
Maximum Communication Range: ?? meters
Note: the telemetry range is from 2km to 4km but the wifi connection really depends. I put the number 1.6km here only because the service is supposed to cover a circle with 1 mile radius. Please feel free to edit it
Battery Durability (At Full Load): ~ 20 minutes
Maximum Allowable Drop Rate: ~ 0.2% -> 1/500
Current UPS loss rate: 0.5 - 1.0%
Elvis
$3.66 - average price customers are willing to pay for delivery within 10 miles
63.6% of users order from within 2 miles
They expect to wait an average of 59.5 minutes for delivery within 10 miles
Many businesses don't offer delivery because of the low size and cost of their available inventory
They would like lower cost, more reliable options
Most make deliveries within 1 mile [Cambridge businesses]
Elvis
There are two main problems with current delivery systems:
Long waits for items ordered within short distances
Unknown wait times
Billie
Why did we choose to create a drone delivery system?
Billie
Today, almost 2 years after Amazon first revealed its project for drone delivery,
these systems are set to revolutionize commerce by lowering shipping costs and speeding up delivery for customers worldwide
but are also going to play a transformative role in fields ranging from
urban management to farming to coastal security to disaster relief
Recently on July 17th, the FAA, Federal Aviation Administration has approved Flirtey, a drone that delivers medical supplies to isolated coal mining regions in Virginia.
Although PrimeAir is still awaiting FAA approval, the path the industry has taken convinced us that it would be an interesting, relevant, and sufficiently complex a task to implement a drone delivery system for the smaller, local businesses that we know and interact with on a daily basis.
Brian
This is DJI Phantom 3. The DJI Phantom series have already been more or less unchallenged as the best all-in-one camera drone on the market.
It is small, light and safe. It can capture high-quality photos and videos.
It is stable in both indoor and outdoor environments, even in a fairly strong breeze.
It is a fully integrated, user-friendly, brilliant Product!
Brian
This brings us to our quadcopter design. We have not had the two plus years or resources that Amazon has had, but in our 2 months we have done some remarkable things in the ability to navigate and control the quadcopter from multiple platforms. In this growing market of quadcopter we envision ours as being the quadcopter for small businesses. It can revolutionize the way small businesses connect to their customers through a faster, more reliable, and very exciting delivery process.
Brian
To demonstrate a little bit of the flight capabilities of our quadcopter, we have put together this video.
Vinh
Now that we’ve talked a little bit about what we’ve been building and why we’re doing it, let’s talk about who this delivery system benefits.
-Vinh
Young professionals need an efficient, affordable delivery system to deliver goods from local stores due to a busy schedule. They don’t have time to go to the store to buy what the want, and can’t wait around all day for a delivery. They are willing to pay for convenience.
A tech-savvy business owner wants to target local professionals during the work day with a new delivery system with more predictable times and lower operation costs.
Vinh
With a clearer goal in mind, we set out on our design process.
First we did research on who was had the most cutting edge tehnology in drones and delivery (amazon, DJI)
Next, with our target customers in mind, we went out conducting surveys on real businesses and people matching our target persona to get a sense of what the valued in a delivery system.
Afterwards, we set out to build a prototype of a delivery system that met consumer need in a novel way.
We then proceeded to test and get user feedback on our system, and used those results to improve our system
Billie
Now we will delve further into the software side
Billie
We developed the application using MAVLink and Qt Creator.
The Pixhawk autopilot that came with our system uses MAVLink library
We wanted to use the same library as the on-board firmware so we choose to program in C/C++.
We chose to use Qt Creator ::::: a leading C++ development environment
Now let's go into a further breakdown of the application structure.
We built the user interface using Qt Creator which supports QML
a JavaScript-based, declarative language
We implemented tracking through the use of the QtLocation and Qt Positioning APIs
We designed the application responsively → so it would adapt to multiple screens and operating systems
The functions in the application are supported by C/C++ and a collection of page handlers that call MAVLink through serial port on the computer side / and through WiFi on the tablet side.
For the communication aspect, we used
MAVLink
Serial port, a form of radio telemetry
And upon discovering compatability issues with serial port and Android deployment, we also implemented WiFi
Eva will discuss this further later on.
Billie
This process was critical because it made us think about how to best implement drone control on a 2D tablet interface vs. the 3-D RC controller
To create the user interface, we went through a series of paper prototypes
sketched out how we wanted the interface to look
split the application into a vendor / customer side
mapped out the process of how our vendors / customers would walk through the application.
added error interfaces for potential flight issues, order issues, weight limits, etc.
added resolution interfaces.
Billie:
Here we have a short clip of the TRACKING FEATURE of the application that allows vendors and customers to see where the drone currently is.
Shown is a flight test on the lawn
The red circle shows the drone's current location.
Billie
Here are the predicted vs. actual flight paths from the video.
As you can see, the drone took more of a circuitous path from its starting point in blue to the destination point in red, probably due to wind conditions and also some GPS inaccuracies.
Billie
The tracking feature that you saw was developed using the QtLocation and QtPositioning APIs.
To walk you through how this happens:
Enter the address on the UI
GeoCodeModel that generates the coordinates for the address the customers type in.
The map object is created using Open Street Maps plugin
This map object is populated by MAPITEMVIEW which places the MapCircles and MapPolylines on the map
Now EVA will speak further on communication between the APPLICATION and DRONE
Eva:
Basically in the network, WiFi works between android app and computer while 433MHz radio telemetry works between computer and UAV.
Computer is critical in this system.
Firstly, it runs a TCP server for the android application. It answers data request and sends GPS and battery information to the app. Secondly, the computer is also a ground control station for the UAV.
Pixhawk board on the UAV can pack C-structs over serial channels with high efficiency and send these packets to the ground control station. The computer use MAVLink library to decode packets and extract information
Our original design is to connect the telemetry with phone directly. But in current stage QtSerialPort library only supports computer operating systems and currently is not workable for Android or iOS. We look forward to a newer version of Qt which officially supports serial port module. By that time, we can use the android app to communicate with the uav directly.
Now our design is to use computer as a signal transfer center. It has its own advantages:
1.as processing and analysing real-time data take serious power.
Computer uses radio telemetry to communicate with UAV. The android application, as a TCP client, connects to the TCP server running by computer and communicates with the drone indirectly.
In this stage, we let the computer work between the android and UAV
why do we use wifi
To use the same library as the on-board control firmware, we choose to programme in C/C++. Not java. We use Qt creator.Furthermore, computer has a bigger screen and can show us more data. Now we have monitor-only mode, computer control mode and app control mode. We can use keyboard and mouse at the same time in computer control, which is very convenient.
Eva
In human natural language communication, we care more about what we want to say
while in machine language communication, we have to decide what we can say.
To reduce transmission latency, we try to limit the datastream by making smart choice of messages.
We pick 9 messages out of more than 200 MAVLink messages and they are listed here
For example, let’s say message number 2 system status. We need to read the battery voltage and remaining battery percentage. We also try to design our own battery life estimation function to see how far it can fly.
Eva
Make sure all android tablets have the dronedeliveryapplication
3 modes: monitor-only, computer control, app control
Eva:
GCS:
The GCS has to be professional, as processing and analyzing flight data in real time takes serious power. The computer GCS collects data from the entire system, including GPS location, data from automatic sensors and command inputs. It cooperates with onboard control to make sure that the drone behaves well in all moments.
Another core function is communication. It runs a server to connect the app and drone together
Control: mouse and keyboard.
APP: in this clip of video, we run the android app on Nexus 9 device.
1. Customers can place an order and track the order
2. The vendor can view customer orders and send out the delivery drone after checking battery
3. Manual control is enabled after the phone joins local area network
Brian
Our main focus in hardware can be broken down into 3 areas . . .
To take a jump back before delving into each of these, I would like to introduce where we started off on the control of the quadcopter *Change to next slide*
Brian
To demonstrate where we started in our control of the quadcopter, we can talk about the difficulties we had flying the drone. Specifically, the incident with the tree.
To quickly summarize, we lost control of the copter and got it stuck in a tree ~60ft or 20m in the air. Now, our control of the drone has progressed to where we can fly it safely with a remote control, computer, or mobile device. It can perform automatic deliveries and detect objects in its trajectory. And now Gary will help introduce how we managed to accomplish all of this
Gary
To pave a way for the audience to better understand our control mechanism
The entire firmware onboard is running with multiple processes at the same time (e.g. LED, Motor control, etc)
A process might need information generated by another
Wrap up information by means of structs
A publish and subscribe mechanism
Gary
Two advantage: without global variable, not messy
allow for us to define owm topic, provide flexibility on app development
-Gary
In order for the user to control the drone conveniently, we wanna be able to control by a personal terminal
-Gary
Brian
We have walked through how the delivery works for both the customer and vendor, but a third equally important character in this process is the drone. The delivery route can be broken down into 6 different stages. The preflight stage sets the mission items for the route. This includes the GPS coordinates for the delivery address. The drone will then be flying to its destination, comparing its current location to the destination and flying in a straight line to the destination. At the destination it will descend to a low altitude to dropoff the package. This is done to softly drop the item and prevent breaking. After it drops the item off, the drone will return home, land, and finally disarm itself.
This is all accomplished with the flip of a switch on the RC or a click of a button on the app. This is done through the following code. There are two important set of states for the drone to be in. The first is the main state, which determines how the navigation states are set. The navigation states are a collection of navigation routines that the drone can move through. They set various flight requirements, like requiring GPS lock, grant permissions, and set control modes for the flight routine to run. The commander.cpp app controls how the main and navigation states are set, while navigation_main.cpp receives the navigation state and activates the delivery.cpp file. The delivery.cpp file is the implementation of the delivery class, which directs the quadcopter through the delivery process. It does this by setting a series of mission_items and polling location data to help it move through the 6 different stages outlined to the left.
Gary
System Ticks is kinda like a real time clock onboard
Gary
Using the MB-1240 ultrasonic proximity sensor to detect barrier
Accurate sensing range from 20cm to 600cm
Sensor gives out analog output ranging from 0-3.3V
ADC sample function runs in System Ticks
Sample rate of 100Hz
Vinh
To briefly describe the gripper, it is a servo connected to a 4cm plastic jaw that will hold a package in place during flight. On delivery, the servo will turn and release the package. It has a torque of 1.8 kg*cm, and so can carry a max of 450g
In terms of powering the gripper, it must be powered and controlled from the AUX output. The PixHawk has 8 MAIN pins and 6 AUX pins, which can send PWM signals out. However, it doesn’t output enough power to actually drive the motors. To do this we need a driver to power the motor’s motion. In this case, a voltage converter is used to take the battery’s voltage and output 5V to AUX. The servo is connected to another AUX pin. Positions shown in diagram, The code control process is shown to the right.
Now that we’ve talked about how the gripper operates, let’s go through the echanical design process that led to this gripper.
-Brian
Here is a video to visually demonstrate our drone delivery system
To start, a customer will place an order using our mobile app. They can select the items they want and then enter in their address and billing information
The vendor will then review this order, check the battery level, and be able to start the delivery with the app
When the drone has been prepared with a charged battery and the requested items, the delivery routine can be activated from the app
If the drone encounters an obstacle during its route, it will stop and ascend until it has cleared the obstacle before continuing on its mission
The progress of the drone can be tracked in the app by both the vendor and the customer while it is completing the delivery
When it reaches the delivery address, it will descend and activate the servo to release the package
The drone will then return home and the customer can retrieve their package once the drone is a safe distance away
Upon coming home, the vendor can get the drone ready with another battery for a new delivery
Some of you may have noticed that the drone flew backwards after the object avoidance part of the video. This was done because we have not actually run a delivery routine with an object to avoid. It is still very likely that the drone will damage itself if it is moving full speed towards an object, so that clip was from us testing the drone at a low velocity against the side of the building. Vinh will now walk us through, quantitatively, how we compared to our initial goals.
Vinh
Vinh
Load Capacity: ~ 400 grams
Maximum Delivery Range: ~ 1 mile (w/ 3 meters accuracy)
Average Flight Speed: 7 m/s
Maximum Flight Speed: ?? m/s
Positioning Accuracy (On App.): ~ 45 meters
Vinh
Maximum Communication Range: ?? meters
Note: the telemetry range is from 2km to 4km
Battery Durability (At Full Load): ~ 20 minutes
Maximum Allowable Drop Rate: ~ 0.2% -> 1/500
Current UPS loss rate: 0.5 - 1.0%
Erin
Erin
Erin
Now we will open up the floor to some questions.
Billie
And now, for a fun video of all our adventures over the summer.
Please follow us outside to the lawn for a live demonstration of delivery.
