Recommended
More Related Content
Similar to Design Issues for Real-Time Remote Robotic Science Operations Support Tools: Observations from the field
Similar to Design Issues for Real-Time Remote Robotic Science Operations Support Tools: Observations from the field (20)
Recently uploaded
Recently uploaded (20)
Design Issues for Real-Time Remote Robotic Science Operations Support Tools: Observations from the field
- 1. KEEP CALM AND TELE-OP Design Issues for Real-Time Remote Robotic Science Operations Support Tools: Observations from the Field Hyunjung Kim*, Young-Woo Park, Electa Baker, Julie Adams, and Terry Fong
- 2. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 3. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 4. Real-Time Remote Robotic Science Operations Rely on diverse, emerging information & require fast, effective decision-making Operations software that supports efficient monitoring of science data and planning is essential!! Science teams Rover operators Planetary Rover Define the science plan & analyze real-time data collected as a result of executing that plan Perform navigation of robots & ensure system functionality Explore remote locations
- 5. Real-Time Remote Robotic Science Operations Support Tools • Rarely tested in practice: the tasks and activities that need to be supported are not yet well understood • Hierarchical, and involve professionals from multiple disciplines, resulting in a diversity of information needs • Deal with incomplete information and unpredictable problems 1 2 Real-Time Science Operations Science Teams The information to be delivered and its representation should be carefully designed!! Design Challenges
- 6. Summary To better understand real-time remote robotic science operations1 To provide practical guidance for improving the design of operations support tools 2 In-field observations of a team of 14 scientists remotely operating a planetary rover during a five-day prospecting mission Three characteristics of real-time science operations in a short- duration robotic prospecting mission 1 Identified challenges, opportunities, and guidelines associated with improving the design of support tools 2 GoalsApproachContributions
- 7. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 8. The Mojave Volatiles Prospector (MVP) Analog Mission
- 9. The Mojave Volatiles Prospector (MVP) Analog Mission • October 20–24, 2014, with the rover operating remotely for 5 hrs/day • To perform remote prospecting, using rover-mounted instruments (e.g., GroundCam, Hazcams, NIRVSS, NSS), to investigate the water content in the Mojave Desert as an analog for subsurface volatiles on the Moon Science Team Rover Operators Rover and payload instruments SOC (Science Operations Center) @ NASA ARC MROC (Mojave Rover Operations Center)
- 10. Science Operations • Nominally covers the entire test duration • Sets of regions of interest (ROIs) • Created prior to the start of the field test • Updated every few days as the mission progresses • The MVP science operations consist of three main processes, each having a characteristic time span and update time: • Covers one day of operations • Traverse plans for prospecting, predesignated stops, area of interest mappings (AIMs), predicted power usage an margin, part of the next plan to support get-ahead activities • Created 1 or 2 days before use, updated 1 day prior to execution • Conducting activities in the tactical plan, with decisions based on science data returned and assessed during the activities 1 Strategic Planning 2 Tactical Planning 3 Real-tine Execution
- 11. Science Lead Sci Ops Mngr (SOM) NIRVSS Science NSS Science Timeliner Traverse Planner SciCom Camera Science Stenographer Science Team Sci Lead (Responsible for science part of tactical plan) Sci Ops Mgr (Responsible for strategic plan and mission goals) SciCom (single interface to Real Time Sci) Stenographer Sci Analysis Team (analysis and recommendations) NIRVSS Sci (monitor sci & hk) NSS Sci (monitor sci & hk) Camera Sci (monitor sci & hk) Traverse planner (create/update traverse plan) Time-liner (create/update timeline plan) @ Science Backroom Rover Team @ MROC
- 12. Science Operations Support Tools NSS Science Camera Science NIRVSS Science Science Lead Science Ops Manager Timeliner Traverse Planner Science Communications Google Earth map xGDS InstrumentsxGDS PlansxGDS Images 3DDisplay(Verve) Timeline(Playbook) Steno- grapher
- 13. Science Operations Support Tools ImagesPlans Raster MapsInstruments
- 14. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 15. Data Collection • To understand how the science team members experience real-time remote science operations • To collect their feedback and suggestions on the use of science operation support tools • 4 Sessions of semi-structured interviews, 30-50 minutes/session • To observe and investigate how the science team used operation support tools for science activities and operational tasks • To identify inadequacies of the tools and opportunities for improvement General Observation, Focused Observation, Voice loop conversations, Videos 1 Interview 2 Observation Participants 14 scientists and mission specialists (8 females and 6 males, ranging in age from 18 to 65, with experience of involvement in a series of NASA robotic field tests)
- 16. Debrief Interview Science Lead and Traverse Planner File: Oct-23_SciLead/TraversePlanner.m4a Duration: 00:46:40 Date: Oct/23/2014 M: Moderator (Electa Baker) O: Observer (Julie Adams) SL: Science Lead TP: Traverse Planner [00:00:27] M: Did you feel you could monitor the voice loops of RT science well? [00:00:32] SL: Voice loops of RT Science. Yes, pretty well. When he was on his phone, I could hear him fine. [00:00:56] M: Did you feel like you could ascertain like the general consensus of the science team, and then put that together to something cohesive to send to the robot team? [00:01:09] SL: That was more difficult. Herding cats. What was really difficult is we had a back room of scientists who don't have as much insight what is going on. But also they definitely don't have the same familiarity with the systems that the front team has. And that's an issue because there's a lot of requests or comments or discussions just about what systems can do. Other times too, there has been I think sometimes it was really good one Darlene showed the goals the other day because I think a lot of other team members get into their own set of expectations or goals or pet projects within the project if you will. Often the science team members had different activities or measurements they wanted to pursue that were not necessarily in line with the mission goals. That's normal. Scientists do that, right? I think at times it took the science lead and the SOM say no, come back, this is what we are doing, we Debrief Interview Science Lead and Traverse Planner File: Oct-23_SciLead/TraversePlanner.m4a Duration: 00:46:40 Date: Oct/23/2014 M: Moderator (Electa Baker) O: Observer (Julie Adams) SL: Science Lead TP: Traverse Planner [00:00:27] M: Did you feel you could monitor the voice loops of RT science well? [00:00:32] SL: Voice loops of RT Science. Yes, pretty well. When he was on his phone, I could hear him fine. [00:00:56] M: Did you feel like you could ascertain like the general consensus of the science team, and then put that together to something cohesive to send to the robot team? [00:01:09] SL: That was more difficult. Herding cats. What was really difficult is we had a back room of scientists who don't have as much insight what is going on. But also they definitely don't have the same familiarity with the systems that the front team has. And that's an issue because there's a lot of requests or comments or discussions just about what systems can do. Other times too, there has been I think sometimes it was really good one Darlene showed the goals the other day because I think a lot of other team members get into their own set of expectations or goals or pet projects within the project if you will. Often the science team members had different activities or measurements they wanted to pursue that were not necessarily in line with the mission goals. That's normal. Scientists do that, right? I think at times it took the science lead and the SOM say no, come back, this is what we are doing, we are not going to do that because that's not addressing the basic goals of the project. Example Interview Transcript
- 17. Example Observation Data General Observation (Team-level activities) Focused Observation (Ethnographic observation on each console position) Voice loop Annotation (Direction of execution & re- planning, internal science team conversations, conversations btw the science team & robot operators)
- 18. Data Analysis Observation data General obs logs Focused obs notes Voice loop annotations Interview data Interview transcriptions (i) The tools involved (ii) The type of activity or task supported (iii) The type of related awareness (iv) How well the concerned tools supported users’ goals or tasks (v) If the statement or incident involve users’ design suggestions (i) Science downlink assessment (ii) Balancing science desires (iii) Optimizing science activities vs. controlling operational complexity (The common elements of science operations identified by Cheng et al. 2008)
- 19. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 20. Real-Time Sci Ops in a Short-Duration Robotic Prospecting Mission • Discovery-based approach • Science as the team’s highest priority “For this [mission], being able to really work with the rover operators to maximize what you can do in a very small amount of time [is unique].” – Science Lead “We have the instrument leads that aren’t monitoring the instruments which is typically what you see in mission operation, they are actually observing the science out of the instruments.” - Timeliner 1 Closeness of the scientists to the operation decision-making • The ability to react to the realities “…a several hour turnaround and science planning, and then the reaction-all stop, do this-handed over to the rover driver to accomplish a certain goal of science,…” – Science Lead “… It takes time just get to know your instrument and the real environment. You won’t really know how the system is going to behave.” – NSS Science 2 Plan revision during execution • The traverse provides the data that the scientists need “…This is a very unique simulation in which the traverse is actually part of the science. So the scientists are very interested in the traverse… Usually, in other robotic missions, you are interested in the destination, and you do science at the destination.” – Timeliner 3 A Traverse-based, not a destination-based approach
- 21. Identified Challenges for Real-Time Remote Robotic Sci Ops Support Tools • Support real-time analyses that will actually promote immediate plan decisions “When you are sitting in a console position, you’re to monitor inconsistencies, you can point things out, but when I analyze it, I’m going to crunch a bunch of numbers on it, and if I’m doing that, then I’m not doing my console position.”– NSS Science • Enable scientists to quickly identify trends and correlations within and across different data sources by allowing the rapid and precise manipulation of data and its settings • Help scientists easily return to a specific time or area of interest and extract the data needed more efficiently 1 Facilitate Science for Operations, Not Science Itself • Maximizing the utilization of robots means maximizing rover traverse distance “They [the scientists] intend to rather have more than you could do than less. That was intentional to have no margins. Well, we can stop and start and give up whenever we want to do. That’s the direction I went.” – Traverse Planner • Consider and facilitate collaborative tactical planning activities • Help the science team identify the availability of resources easily 2 Optimize Rover Traverse to Maximize Science Returns
- 22. Design Opportunities to Improve Spatial Awareness 2 Geospatial information-based science activities • Comparing multiple traverse paths on a single map, or different sets of data collected at a certain location should be better supported “We are having a discussion about ‘where we have been’ and ‘where we wanted to go.’ The questions I had to go through with the team yesterday were ‘what knowledge have you garnered?’ ‘How do you expand or deepen that breath of the knowledge, and then how do you all test that knowledge and based on those answers, where do you want to go?’”– Science Operations Manager “Camera Sci wants to see TextureCam from the exact same spot. I think that’s hard” - General obs note (Oct 21 10:21) • No means of coordinating visual attention or support for deictic communication • Unclear which specific area on the map is being referred to and whether others are following the conversations or not 1 Communication of spatial info within and btw teams “[The scientists] Said ‘We want this. Do this area, and try to avoid what looks like an obstacle’. I am not a scientist so I don’t know exactly what they want me to cover…” – Traverse Planner
- 23. Design Opportunities to Improve Temporal Awareness • Time stamps based on absolute time, time since task performance, time until the next required tasks “If you are going to change a certain path or certain segments [of the traversal plan], you need to bring it in and do your whole planning process in 5 minutes. They [the scientists] spend so much time discussing if they will do or not, then they run out of the time to actually do it, approve it, send it”– Timeliner “Science Lead is talking about collecting data and the time available to develop a plan of what to do next. How do we integrate a timer so that we can see? - Global observation note (Oct 21 15:16:42)” 1 Offer scientists the temporal information needed at a glance 2 Time-based navigation and search • Support to easily and quickly navigate and search past instrument plots, images, and raster map data “I am quantifying real time in a tactical sense, being able to augment the plan significantly or make a reaction to what you are seeing within an hour or two.”– Science Lead * “Real time” in actual operations “I don’t think they [the scientists] are looking at it [the timeline] that much. When I call their attention to it, they do.” - Timeliner * Significantly less interested in temporal info than spatial info
- 24. I Introduction II Analog Mission Overview III Methods IV Results and Findings V Discussion and Conclusion
- 25. Suggestions and Guidelines 1 Support efficient time-based and geolocation-based navigation & search • Indicate operational events on the time axes of the strip charts • For viewing location-related data, allow users to input locations easily, by, e.g., clicking directly on the position of interest on the rover traverse • Permit direct tagging in the plots • Add shortcuts to facilitate the extraction of necessary data 2 Support communication of spatial information • An implementation of screen sharing that allows users to indicate, highlight, draw, and write on top of the underlying visual information would be useful • Enable scientists to leave graphical instructions and comments directly on the map and allow the Traverse Planner to import this info into the planning tool as a layer for reference use
- 26. Suggestions and Guidelines • A clock displays the absolute time, which is important for coordinating work within and between teams • A critical reference for time-based navigation and search • Must be consistent across all different tools used 3 Integrate a clock, a timer, and a stopclock as temporal representations until… from… • A timer that counts down from a specified amount of time is necessary for deadline-driven plan revision • A stopclock that shows the time since the task performance is necessary to present temporal context of science operation • e.g., NIRVSS reference spectra collection, taking panorama images
- 27. Conclusion
Editor's Notes
- Thank you for your introduction. My name is Hyunjung Kim. I’m here today to talk about “Design Issues for real-time remote robotic science operations support tools”
- I’d like to give you a brief outline of my presentation. The presentation is divided into five main sections: introduction, analog mission overview, data collection and analysis methods, results and findings, and discussion and conclusion.
- Let’s start with what real-time remote robotic science operations is. As the picture describes, real-time remote robotic science operations require highly effective coordination and collaboration btw: Science teams, who define the science plan and analyze real-time data collected as a result of executing that plan, Robot operators, who perform navigation of robots and ensure system functionality, and Planetary rovers, which explore remote locations Since real-time remote robotic science operations rely on diverse, emerging information and require fast, effective decision-making, operations software that supports efficient monitoring of science data and planning is essential.
- However, in order to design operations software to support real-time remote robotic science operations, several challenges must be addressed. First, as the concept of real-time science operations is relatively new and has been rarely tested in practice, the tasks and activities that need to be supported are not yet well understood. Second, science teams, the main users, are hierarchical, and involve professionals from multiple disciplines, resulting in a diversity of information needs. In addition, they deal with incomplete information and unpredictable problems. Therefore, the information to be delivered and its representation should be carefully designed.
- This study has two goals: First, to better understand real-time remote robotic science operations, and second, to provide practical guidance for improving the design of operations support tools. To achieve these goals, we conducted in-field observations of a team of 14 scientists remotely operating a planetary rover during a five-day prospecting mission. Our contribution is twofold: First, three characteristics of real-time science operations in a short duration robotic prospecting mission, and second, the identified challenges, opportunities and guidelines associated with improving the design of support tools.
- Let me show you a 3-minute video about the Mojave volatiles prospector analog mission during which we conducted in-field observations.
- The MVP analog mission took place on October 20 – 24 , 2014, with the rover operating remotely for five hours a day. The team’s science goal for the mission was to perform remote prospecting, using rover-mounted instruments, to investigate the water content in the Mojave desert as an analog for subsurface volatiles on the Moon. The science team was located in the science operations center at the NASA Ames, approximately 400 miles from the Mojave rover operations center, where the rover operators were located.
- The MVP science operations consist of three main processes, each having a characteristic time span and update time. First, strategic planning nominally covers the entire test duration. It contains sets of regions of interest areas identified to potentially have characteristics relevant to the science goals and objectives. The strategic plan is created prior to the start of the field test, and the plan is updated every few days as the mission progresses. Second, tactical planning covers one day of operations. Tactical plans include traverse plans for prospecting, predesignated stops, and search locations, allocations for discretionary activities such as area of interest mapping, predicted power usage and margin, and part of the next plan to support get-ahead activities. The plans are created one or two days before use, and updated one day prior to execution. Finally, real-time execution refers to conducting activities in the tactical plan, with decisions based on science data returned and assessed, during the activities. Observation in this study was specifically focused on tactical re-planning and real-time execution processes during remote rover operations.
- The science team consisted of scientists and mission specialists with different roles, responsibilities, knowledge, and expertise. The team includes the Science Operations Manager, the science lead, science communications, the traverse planner, the timeliner, the three science payload leads, and the stenographer. The science analysis team who focuses on more in-depth data analysis was located to a separate room called the science backroom.
- This is the layout of the science operations center. The layout is designed to enable face-to-face communications between certain console roles. For example, the science operations manger and the science lead, who are responsible for strategic and tactical decision-making, are adjacent to each other and are located behind the science payload leads to allow observation of their individual displays. The Traverse Planer and Timeliner, who are in charge of spatial and temporal planning, are adjacent to each other as well. The science operations center had six wall-mounted shared displays.
- In terms of operations support software, the NASA exploration ground data system, xGDS, provided four displays: Plans, Images, Instruments, and Raster maps. In addition, the 3D Display showed a three-dimensional representation of the rover and the terrain. The timeline display showed the NASA Playbook software, which tracks the timeline.
- To understand real-time remote robotic science operations, and to assess how well the tools support science operations, we gathered data by means of semi-structured interviews, direct observation, and audio and video recordings of science operations. The participants consist of 14 scientists and mission specialists. Many of them had experience of involvement in a series of NASA mission operations, such as the MER mission, the LCROSS mission, the LADEE mission, and the Mars Science Laboratory prelaunch. First, we conducted four sessions of semi-structured interviews To understand how the science team members experience real-time remote science operations To collect their feedback and suggestions on the use of science operation support tools Second, during the five-day period of the mission, for five hours each day, we conducted observations within the science operations center: To observe and investigate how the science team used operation support tools for science activities and operational tasks To identify inadequacies of the tools and opportunities for improvement
- This is an example interview transcript
- And these are example general observation logs, focused observation notes, and voice loop annotation. General observation focused on how the science team used the tools for team-level activities. On the other hand, focused Observation focused on the individual use of the tools. In addition, voice loop conversations were recorded and annotated as well.
- To analyze the collected data, we used the framework of common elements of science operations identified by cheng’s paper presented in 2008: science downlink assessment, balancing science desires, and optimizing science activities versus controlling operational complexity. In addition, we identified statements and incidents related to the use of support tools. During the analysis we considered: the tools involved, the type of activity or task supported, the type of related awareness, how well the concerned tools supported users’ goals or tasks, and if the statement or incidents involve users’ design suggestions.
- Let’s move on to the results and findings.
- From the interviews and observations, we identified three characteristics of real-time science operations in a short-duration robotic prospecting mission. First, as this mission takes a discovery-based approach, the closeness of the scientist to the operation decision-making is unique compared to other missions. Second, most participants agreed that plan revision during execution in reaction to real-time data is a critical feature of real-time science operations in a robotic prospecting mission. They emphasized that the ability to react to the realities is invaluable. Third, a science-driven robotic prospecting mission takes a traverse-based approach, not a destination-based approach. The traverse is critical because it must provide the data that the scientists need.
- Our major findings include two design challenges for improving monitoring, analysis, and planning tools for real-time remote robotic science operations: First, is to facilitate science for operations, not science itself. It is important to support real-time analyses that will actually promote immediate plan decisions. This means that the tools should enable scientists to quickly identify trends and correlations within and across different data sources by allowing the rapid and precise manipulation of data and its settings. In addition, the tools should help scientists easily return to a specific time or area of interest and extract the data needed more efficiently. Second, is to optimize rover traverse to maximize the science returns. In this kind of cases, for scientists, maximizing the utilization of the robots means maximizing rover traverse distance. Traverse planning and re-planning is about optimization of rover traverse in reaction to the reality. In practice, the major bottleneck in traverse re-planning was in collaborative activities, for example, collection of information and operational recommendations, delivery of instructions to the Traverse Planner, and collaborative reviewing. Therefore the tools should consider and facilitate collaborative tactical planning activities. In addition, the tools should help the science team identify the availability of resources easily, such as the time remaining.
- One of the main objectives of remote robotic operations support tools is to provide users with a sense of awareness. In this study, we particularly focused on spatial awareness and temporal awareness, as they are directly related to the key question: “How can we make a traverse that meets our science goals but doesn’t make the durations so long that we cannot actually finish it?” To better support spatial awareness, First, communication of spatial information within and between teams should be improved. Although the scientists often reference the maps and telemetry plots displayed on shared screens, there is no means of coordinating visual attention or support for deictic communication. It is sometimes unclear which specific area on the map is being referred to, and whether others are following the conversations or not. As shown in the pictures, the science lead and science operations manager use laser pointers to point to certain areas or draw paths on the map. This is not precise, and the trace disappears instantly. Especially, the Traverse Planner commented that having a clear consensus of what scientists want to do was difficult. Second, geospatial information-based science activities, for example, comparing multiple traverse paths on a single map or different sets of data collected at a certain location, should be better supported. Although the current tool allows stacking of multiple layers of plans, the plans all appear in the same color and become cluttered by the labels of all of the stations.
- Regarding temporal awareness of the team, it is noteworthy that real-time in actual operations had a longer time frame than we initially expected. In addition, there was less time pressure than we predicted, as well. It was also interesting to observe that the scientists were significantly less interested in temporal information than spatial information. To better support temporal awareness, First, shared information displays should be redesigned to offer scientists the temporal information needed at a glance. During the observation, it was observed that the science team did not know if they were ahead of or behind the schedule unless the Timeliner announced it. In addition, there was no support for measurement of the execution time of ad hoc science activities, such as NIRVSS reference spectra collection. Second, time-based navigation and search should be improved to support scientists to easily and quickly navigate and search past instrument plots, images, and raster map data.
- Here are some suggestions and guidelines for the design of real-time remote robotic science operations support tools. First, the tools should support efficient time-based and geolocation-based navigation and search. One of our suggestions for more efficient data navigation is to indicate operational events, for example, starting and finishing times for the plan or reference data collection, on the time axes of the strip charts. In addition, for viewing location-related data, allowing users to input locations easily, by, for example, clicking directly on the position of interest on the rover traverse would help. Other suggestions include permitting direct tagging in the plots and adding shortcuts to facilitate the extraction of necessary data. Second, the tools should better support communication of spatial information. An implementation of screen sharing that allows users to indicate, highlight, draw, and write on top of the underlying visual information, such as maps, plots, and images, would be useful, as has been proved in many other collaborative support tools. For example, to enable scientists to give precise instructions and guides to the Traverse Planner, scientists should be enable to leave graphical instructions and comments directly on the map, and allow the Traverse Planner to import this information into the planning tool as a layer for reference use.
- Finally, the tools, particularly shared information displays, should integrate a clock, a timer, and a stopclock as temporal representations. First, a clock displays the absolute time, which is important for coordinating work within and between teams. Time stamps based on absolute time are also a critical reference for time-based navigation and search of archived data. So, they must be consistent across all different tools used. Second, a timer that counts down from a specified amount of time is necessary for deadline-driven plan revision, to notify science teams the time available to develop a plan of what to do next. Lastly, a stopclock that shows the time since the task performance, in this case, the time since NIRVSS reference spectra collection or taking panorama images, is necessary to present the temporal context of the science operation. Importantly, the information for updating temporal awareness needs to be accessible and absorbable at a glance.
- Let’s summarize briefly what we’ve looked at. Through and in situ observation study, we investigated how science teams monitor, analyze, and plan during real-time remote robot operations and evaluated how the current tools support teams’ goals, tasks, and activities. On the basis of our results, we identified the characteristics of real-time science operations in a short duration robotic prospecting mission. In addition, we discussed the identified challenges, opportunities, and guidelines associated with improving the design of operations support tools. Although our study was limited to observation of a single analog mission, we expect that our findings will inform future research on robotic operations support tools and expand the scope of inquiry to several different use cases, including a future lunar volatile prospecting mission. Thank you for your attention.