Download to read offline
Uploaded byJuntima Nawilaijaroen
Touch Drive - A touch-based multi-function controller for autonomous driving
The document discusses the development of a touch-based multi-function controller for autonomous driving, aiming to enhance driver interaction and efficiency. It outlines research questions related to the design and functionality of the interface, user studies, and design iterations, highlighting the importance of feedback and usability. The findings emphasize the necessity of intuitive design and user involvement in creating effective interfaces for autonomous vehicle operation.
More Related Content
Similar to Touch Drive - A touch-based multi-function controller for autonomous driving
Touch Drive - A touch-based multi-function controller for autonomous driving
- 1. A T OU C H - B A S E D M U LT I - F U N C T I O N C O N T R O L L E R F O R A U T O N O M O U S D R I V I N G J U N T I M A N A W I L A I J A R O E N & VA S I L E I O S G O L E M A T I S I X D C T H C H A L M E R S U N I V E R S I T Y O F T E C H N O L O G Y - S E M C O N - A I M M I T TOUCH DRIVE
- 2. O U TL I N E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N We planned. We worked. Tirelessly.
- 3. R E SE A R C H Q U E S T I O N S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Problems were found. Hypothesis was then formed. Will a tablet-based tactical controller, augmented with an overlay cover increase the efficiency of the interface, providing a clear distinction between the input and output functions? How can a tablet interface be designed to keep the driver in the loop of the autonomous driving task by functioning as a tactical and tertiary controller, providing an easy switch between these modes?
- 4. S C OP E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Great work came with great responsibilities. Semi-autonomous Tactical features & Tertiary menu Focus on visual modality Driver as supervisory operator Given design problem
- 5. D I VE R G E N C E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Literature Review - Requirement Elicitation - User Studies
- 6. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N What are things we are going to deal with? + + Automation IVIS & ADAS Touch - Based Interface
- 7. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in automation Levels of Automation
- 8. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in automation Interaction between human & automation Levels of Automation
- 9. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in automation Interaction between human & automation Levels of Automation Trust and Controllability
- 10. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in automation Interaction between human & automation Levels of Automation Trust and Controllability Driver in the loop
- 11. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N In-Vehicle Information Systems (IVIS) and Advanced Driver Assistance Systems(ADAS) Interference & Distraction Complexity & Over-Reliance Loss of situation awareness
- 12. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Why touch - based interface?
- 13. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Why touch - based interface? why tablet?
- 14. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Why not?
- 15. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Rapidly increasing uses of tablet in everyday contexts
- 16. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Rapidly increasing uses of tablet in everyday contexts then, What is the problem of using tablet in car?
- 17. L I TE R AT U R E R E V I E W I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N design challenges and limitations in touch - based interfaces Utilization of tablet affordance Intuitiveness & efficiency Overlay cover for better affordance
- 18. R E QU I R E M E N T E L I C I TAT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N listed out system requirements and use cases
- 19. R E QU I R E M E N T E L I C I TAT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N listed out system requirements and use cases Highway Driving City Driving Change Lane Accelerate / Decelerate Stop Take next exit Turn left / right Roundabout Stop Park U - Turn
- 20. R E QU I R E M E N T E L I C I TAT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N listed out system requirements and use cases Input Turn left / right Change lane Action related information : available actions, confirmation of actions, executing action Output Speed (accerelate / decerelate / stop) Switching to infotainment Speedometer Alert Notifications (fuel and oil level)
- 21. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express'
- 22. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express' Build Try Express
- 23. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express'
- 24. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express' What did we get from this workshop?
- 25. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express' Patterns were found Blank space to draw a gesture Needs of additional info. - ETA - Environmental Hazards
- 26. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express' There are also interesting findings
- 27. U S ER S T U D I E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N participatory design - 'Build - Try - Express' Highway Driving Participants say: There's nothing much to do here. I will trust the automation. Facing obstacle Participants say: I concern about my safety. I need do something. I want more control. I want to know everything! Then, participants also say
- 28. R E QU I R E M E N T E L I C I TAT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N we then realized that... Feedback is vital. Design solution therefore should provide a sufficient amount of feedback.
- 29. T R AN S F O R M AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Design Iterations - Expert Evaluation
- 30. A F FI N I T Y D I A G R A M I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Beginning of 1st design iteration. An emergence of dominant ideas.
- 31. 1 S TP R O T O T Y P E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N first batch of prototypes were crafted.
- 32. 1 S TP R O T O T Y P E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N first batch of prototypes were crafted.
- 33. E X PE RT E VA L U AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Explore the possibilities with the help from experts. ฉ พ ร ะCriteria ฉ พ ร ะLayout ฉ พ ร ะContent ฉ พ ร ะAccessbility ฉ พ ร ะNavigation ฉ พ ร ะSatisfaction ฉ พ ร ะLearnability ฉ พ ร ะProgress and Interaction
- 34. E X PE RT E VA L U AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Explore the possibilities with the help from experts. ฉ พ ร ะAlignment is very subjective 5IxD experts ฉ พ ร ะ Combination of both menus can be a good thing to consider ฉ พ ร ะMore visual affordances! ฉ พ ร ะRevisit drag & drop gestures
- 35. 2 N DP R O T O T Y P E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N time passed. things changed.
- 36. 2 N DP R O T O T Y P E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N time passed. things changed.
- 37. C O NV E R G E N C E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Implementation - Validation - Result Analysis
- 38. I M PL E M E N TAT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N We designed. And we crafted thoroughly. + + Android Studio Sketch 3 Git & Bitbucket
- 39. F I NA L P R O T O T Y P E I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N We designed. And we crafted thoroughly.
- 40. TA C TI C A L C O N T R O L I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N
- 41. S P EE D S L I D E R I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N The evolution in designing for speed slider Operational Tactical Tactical & Strategic
- 42. T E RTI A RY M E N U I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N We designed. And we crafted thoroughly.
- 43. O V ER L AY C O V E R I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Tacticle feedback and better affordances
- 44. ฆ VA L ID AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Wizard and his magic Speed Disabled Commands Processing Commands Environmental Alerts Issued Commands
- 45. M O RE VA L I D AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N What to validate? U SUsability Efficiency Effectiveness Satisfaction Moreover,
- 46. VA L ID AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N What to validate? U SUsability Efficiency Effectiveness Satisfaction U SSemi-Autonomous Driving Experience Trust Controllability
- 47. VA L ID AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Simulate the semi-autonomous driving experience
- 48. VA L ID AT I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Simulate the semi-autonomous driving experience Questionnaire Manual Driving Semi-Autonomous Driving Free drive Drive with given tasks System Usability Scales Interview
- 49. R E SU LT S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Rumors had it all... Both system were efficient. There is no overall significant difference between two system.
- 50. R E SU LT S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N Rumors had it all. Pros Easy to use Functions are useful Some functions are hard to access (Eco-slider, 2nd turn) Cons Easy and fast switching between modes Misinterpretation of alerts Unintuitive disabled commands Divided attention Fast to learn
- 51. D I SC U S S I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N There's always a room for improvement. Driver in the loop Driver as a supervisory operator Gradual transition to autonomous cars Tertiary control Multimodalities
- 52. C O NC L U S I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N conclusion - ethical issues - future work
- 53. C O NC L U S I O N I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N To sum it up... Feedback is vital. Mutimodalities is a must. (1) Dedicated tactical touch-based interface with overlay cover. (2) Multi-function touch-based interface
- 54. E T HI C A L I S S U E S I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N There always two sided of a coin Deskilling Misplacement of trust Replacement of human
- 55. F U TU R E W O R K I N T R O D U C T I O N D I V E R G E N C E T R A N S F O R M AT I O N C O N V E R G E N C E C O N C L U S I O N One another iteration Tablet placement Audio & Haptic Feedback More strategic functions
- 56. T H AN K Y O U Are there any question?
Editor's Notes
- #2 Touch drive, a touch based multi-function controller for autonomous driving. Right now, we can see that there’s a competition over autonomous car technology. Car companies like Volvo, BMW and Audi, even Google are competing over fin`ding best solution for the car that drive on its own. In this thesis, with the help from Semcon and Volvo Car Company, we were able to design multi-function controller using tablet. This tablet device is expected to used instead of a steering wheel.
- #3 In our thesis work, we followed traditional Jone’s model closely. We divided our work into three phases namely: Divergence, transformation and convergence. Hence, in this presentation, in the next 25 mins, we going to talk briefly about our work scope, the three phases as showed here and ending with conclusion, future works and some ethical issues that may arise.
- #4 Before we begin, we want to show you our formal research questions. You will find an answer to these question through our presentation. We first started off with some literature reviews combined with design problems given by Semcon and Volvo, we were able to form these research questions. Our problem is that how can a tablet interface be design to keep the driver in the loop and also function as both tactical and tertiary controller? And also, will tablet that augmented with overlay cover increase the efficiency of the tablet interface? Since comparing a tablet with steering wheel, it obvious that tablet has less affordances. So we wish to investigate that tablet would keep the driver in the loop of driving or not? and also if we put some highlighter like overlay cover, would it help in increasing affordance?
- #5 From the research question and given design problems, we set up our scope of work as listed here. First, we considered our work to be used in ‘semi-autonomous’ situation. Second, the tablet should be driver as a supervisory operator which means the car does most of the task like adjusting the wheel. Role of the driver is that giving more advance command like changing lanes, turn left or right. Third, we wish to focus on tactical features. As I said earlier, like change to left lane. Not like setting up destination and go. We wish also to investigate the possibility of including tertiary feature like playing music and movies. Lastly, we will focus only in visual modality as it is the strength of tablet device modality.
- #6 After you know a little bit of our project. I will begin with our first phase, Divergence.
- #7 To answer the research question we’ve set. We realized that we need to deal with these things. Automation, IVIS & ADAS and design principle for touch-based interface.
- #8 Here, we want to give you a very short description on what exactly we are dealing with. Firstly, an automation part. Before we begin the project. We need to know on what level we are going to design for. You can see here in this figure that there are several levels of automation. Starting from manual to fully automated. On manual side, human driver would be a driver as we’ve seen today. On the other side, fully automated, human would be just a passenger, just like riding a taxi. You set the destination and go. Our work will lie on these two levels, operator and supervisor. Human driver still in charge of controlling the car by giving a set of commands. But the car will judge if the command is valid or not in terms of safety.
- #9 Next, we need to concern also an interaction between human and automation. Normally, we will give out command to machine like do this and that. And the machine will give us back an information if command is successfully performed or not. These interaction are essential as it lead to next issue. Trust and controllability.
- #10 As machine speak different language. We can not talk to machine like we talk to human. So we had an issue of should we trust a machine or not. And how much should we control it. Will it go mad if we sent out a command this way. This issue need to be solve in a good way.
- #11 Moving to the next topic, IVIS and ADAS. IVIS stands for in vehicle information system which is the screen as you can see here. GPS, music, secondary feature like temperature control and stuff. ADAS is advanced driver assistance system. I don’t know if you guys have tried this system before or not. But car nowadays had some sort of auto pilot mode where it could park on its own. But why these are problems? Because we had these issues. Distraction when interact with the screen. over-reliance when on auto-pilot mode. and as a consequence loss of situation awareness.
- #12 Now, we are going to deal with touch based interface. You may ask why touch based interface over steering wheel?
- #13 even more, why tablet?
- #14 Then we asked, why not?
- #15 Since people nowadays get so used to with mobile technology. The uses of tablet is rapidly increasing.
- #16 So, what’s the problem of using tablet.
- #17 We learned that tablet, comparing to steering wheel, has less affordance. Here’s we need to utilize the tablet affordance, As well as increase its intuitiveness and efficiency. Lastly, we wish to test if tablet affordance could be increase by using overlay cover.
- #18 Then we run a first round of system requirement elicitation. We listed interesting findings from literature. Combined with our own experience in driving. We then got…
- #19 We set a participatory design workshop where participant immerse themselves into semi-autonomous driving situation and help us design the tablet interface. We divided the workshop into three phases.
- #20 This is a sample of the interface design from participant.
- #21 And what did we ger from this workshop.
- #23 what we’ve got from ‘try’ section: Observe that, in highway in facing obstacle
- #24 Then we realized that feedback is vital. Participant want to know everything. car status. speed. environmental information. Design solution…
- #25 After that, we get into transformation. We tranform data into ideas.
- #26 ideate through affinity diagram.
- #28 tablet orientation
- #31 main difference is icon representing available commands.
- #32 Putting more strategic functions such as
- #34 Android Programming Graphical Design Update repository (keep track of progress/revert)
- #35 Tactical Map Strategic Functions (Involve route planning) Automation status current task performed/Car Mode (By default) Car Status Speed/Fuel Environmental alerts signs/ obstacles Route info Destination / ETA (PD workshop)
- #36 Confirm/Cancel command
- #37 Set a mode, let the automation handle the speed according to environment/laws Less control, more convenience
- #38 switch to tertiary (full screen considered - receive output through audio/haptic cues)
- #39 3 layers - opaque, transparent, exposed: distinguish input-output
- #40 wizard app - disabled commands/current performed tasks by automation , alerts
- #41 Usability criteria
- #42 Trust-Controllability
- #43 Safer Simulator - Science Park 1 pilot (10 subjects) 1hour sessions
- #44 Questionnaire Manual driving - Get the feeling of the steering wheel Freeway driving, Think Aloud - Get familiar with interface Instructed driving, SUS x2 (Usability) Closing interview (Experience)
- #45 No significant difference, but tablet with overlay performed better overall with less error rate and TCT. Felt more dedicated to tactical control
- #47 Driver in the loop - driver controls and knows Tactical level of control - automation handles tedious tasks Not Replacement of steering wheel! But complement for convenience Separation of devices or multimodalities (for tactical/tertiary)
- #49 2 paths Not just sufficient feedback, but also proper feedback! Multimodality (relieve visual load, reduce divided attention)
- #50 Placement (in the front, share between passengers) Audio & Haptic (tailor to specific functions) More strategic (navigation map)
- #51 Deskilling (reduce human skill in driving, inability to be efficient in manual driving) Misplacement of trust (blame automation/car industry/manufacturer) Replacement of human (professional drivers/taxi/chauffer etc.)
- #53 this list of requirement. We separated driving experience into two main phase. Highway driving. Nothing much to do here except for changing lanes, take next exit, speed up and down and stop the car. City driving is more complex, you need to make a turn, driving along roundabout, stop and parking lot. So, from here, we see that driver need to provide input to the car.
- #54 As you can see here, these are input commands that was expected from driver. And the car will start an interaction with human driver by providing these output. We felt this list of system requirements is not enough. But we can not observe and find more requirements from real situation as the technology doesn’t exit yet. So, we run a user studies.
- #55 Build try and express. We started off by describing our work scope and the idea of semi-autonomous car. We then asked participant to filled out questionnaire as we wish to collect the ethnography data as well. After that, we begin build. Participant were asked to used design tools given. as you can see here. to build up interface on a tablet. Next, participant were asked to try out their design and see if anything need to be fix or not. In here, we set up the situation of combination between highway driving and city driving. Participant were asked what if there is traffic light, what they are going to do. Lastly, express, in here we capture the thought behind their interface design. we asked the reason why they design like this. We gathered a very valuable data from here.