Invited talk for AI-HRI '19 (AAAI Fall Symposium Series)

4. `

15. Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2017. A Motion Retargeting Method
for Effective Mimicry-based Teleoperation of Robot Arms. HRI ’17.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2018. RelaxedIK: Real-time Synthesis
of Accurate and Feasible Robot Arm Motion. RSS ’18.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2019. Effects of Onset Latency and
Robot Speed Delays on Mimicry-Control Teleoperation. Submitted for publication.

24. Mimicry-based
Teleoperation

25. Approach Overview
Hand
6-DOF Space
End Effector
6-DOF Space
Robot Joint
Angle Space
Spatial
Mapping
Inverse
Kinematics

27. 𝐟 𝜃 = 𝑤 𝑝 𝑓𝑝 𝜃 + 𝑢𝑤 𝑜 𝑓𝑜 𝜃 + 𝑤𝑗 𝑓𝑗 𝜃 + 𝑤𝑒 𝑓𝑒 𝜃
𝑤 𝑝 low 𝑤 𝑝 medium 𝑤 𝑝 high
Position Tracking

28. 𝑤 𝑜 low 𝑤 𝑜 medium 𝑤 𝑜 high
𝐟 𝜃 = 𝑤 𝑝 𝑓𝑝 𝜃 + 𝑢𝑤 𝑜 𝑓𝑜 𝜃 + 𝑤𝑗 𝑓𝑗 𝜃 + 𝑤𝑒 𝑓𝑒 𝜃
Orientation Tracking

29. 𝑤𝑗 low 𝑤𝑗 medium 𝑤𝑗 high
𝐟 𝜃 = 𝑤 𝑝 𝑓𝑝 𝜃 + 𝑢𝑤 𝑜 𝑓𝑜 𝜃 + 𝑤𝑗 𝑓𝑗 𝜃 + 𝑤𝑒 𝑓𝑒 𝜃
Minimized Joint Velocities

30. 𝑤𝑒 low 𝑤𝑒 medium 𝑤𝑒 high
𝐟 𝜃 = 𝑤 𝑝 𝑓𝑝 𝜃 + 𝑢𝑤 𝑜 𝑓𝑜 𝜃 + 𝑤𝑗 𝑓𝑗 𝜃 + 𝑤𝑒 𝑓𝑒 𝜃
Minimized End Effector Cartesian Velocity

31. TRAC-IK
Relaxed-Mimicry
(Ours)
Singularity Avoidance
𝐂 𝜃 ∶= |det 𝑱 𝜃 𝑱 𝜃 𝑇 | − 𝑠 𝑚𝑖𝑛 ≥ 0

32. Self-Collision Avoidance
TRAC-IK
Relaxed-
Mimicry (Ours)
𝑙𝑖 ≤ 𝜃𝑖 ≤ 𝑢𝑖

34. Importance-Based
Inverse Kinematics

35. Hand Velocity High
Importance-Based
Inverse Kinematics

36. Hand Velocity Low
Importance-Based
Inverse Kinematics

43. Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2017. A Motion Retargeting Method
for Effective Mimicry-based Teleoperation of Robot Arms. HRI ’17.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2018. RelaxedIK: Real-time Synthesis
of Accurate and Feasible Robot Arm Motion. RSS ’18.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2019. Effects of Onset Latency and
Robot Speed Delays on Mimicry-Control Teleoperation. Submitted for publication.

45. Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2018. An Autonomous Dynamic
Camera Method for Effective Remote Teleoperation. HRI ’18.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2019. Remote Telemanipulation with
Adapting Viewpoints in Visually Complex Environments. RSS ’19.
Daniel Rakita, Bilge Mutlu, Michael Gleicher, and Laura M. Hiatt. 2019. Shared control–
based bimanual robot manipulation. Science Robotics 4, 30 (May 2019).

47. It takes more than 2 one-handed systems
We must help with coordination

49. Laura will say more
tomorrow

52. Array of Static Cameras
Camera ViewsCameras

54. End Effector Camera
Camera
Point of View

57. No context of environment
Participant Video

61. Motion Retargeting
Optimization
User Motion Input
Manipulation Robot
Configuration
(per update)
Camera Robot
Motion Optimization
Live Video Stream
Camera Robot
Configuration
(per update)

62. Camera Distance
Camera should move in for detail when user is exhibiting slow control

63. Camera Distance
Camera should move out for context when user moves robot quickly

64. Absolute Frame Relative Frame
Control Frame

65. ParticipantRemote
Workspace
Experimenter

66. Organize Pills

69. Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2018. An Autonomous Dynamic
Camera Method for Effective Remote Teleoperation. HRI ’18.
Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2019. Remote Telemanipulation with
Adapting Viewpoints in Visually Complex Environments. RSS ’19.
Daniel Rakita, Bilge Mutlu, Michael Gleicher, and Laura M. Hiatt. 2019. Shared control–
based bimanual robot manipulation. Science Robotics 4, 30 (May 2019).

71. Pragathi Praveena, Guru Subramani, Bilge Mutlu, and Michael Gleicher. 2019.
Characterizing Input Methods for Human-to-Robot Demonstrations. HRI ‘19.
Guru Subramani, Michael Zinn, and Michael Gleicher. 2018. Inferring geometric
constraints in human demonstrations. Conference on Robot Learning ‘18.
Guru Subramani, Michael Hagenow, Michael Zinn, and Michael Gleicher. Constraint
Inference Using Pose and Wrench Measurements. Submitted for publication.
Guru Subramani, Michael Hagenow, Bolun Zhang, Michael Zinn and Michael Gleicher.
Robust Replay of Human Demonstrations using Identified Constraints. Submitted for
publication.

73. Kinesthetic Teaching

74. Kinesthetic Teaching

75. Photo by Meghan Schiereck on Unsplash
Photo by Mat Reding on Unsplash
Photo from Robotiq Website

76. Efficient – Prevent wasteful use of resource
Subjective performance – Demonstrator perception
Facile – Demonstrator ease of use
Amenable to analysis – Post-processing
Desired demonstrations – Experimenter objectives
Affords quality demonstrations – Data quality
Easy to learn – Process to proficiency
Preference – Demonstrator liking
Feedback – Access to demonstration performance
Plausibility – Equivalent strategies
Feasibility – Correspondence
Instrumentable – Measurement capabilities
Desirable properties in a demonstration method

77. • Easy for demonstrator
• High quality demonstrations
• Hard to instrument
• Hard to map to robot
Kinesthetic Teaching
• Hard for demonstrator
• Poor quality demonstrations
• Easy to instrument (with caveat)
• Feasible mappings (with caveats)
Showing tasks to robots (common approaches)
Hand Demonstrations

79. https://commons.wikimedia.org/wiki/File:Salad_bar-02.jpg Photo by Dan Gold on Unsplash

100. Pragathi Praveena, Guru Subramani, Bilge Mutlu, and Michael Gleicher. 2019.
Characterizing Input Methods for Human-to-Robot Demonstrations. HRI ‘19.
Guru Subramani, Michael Zinn, and Michael Gleicher. 2018. Inferring geometric
constraints in human demonstrations. Conference on Robot Learning ‘18.
Guru Subramani, Michael Hagenow, Michael Zinn, and Michael Gleicher. Constraint
Inference Using Pose and Wrench Measurements. Submitted for publication.
Guru Subramani, Michael Hagenow, Bolun Zhang, Michael Zinn and Michael Gleicher.
Robust Replay of Human Demonstrations using Identified Constraints. Submitted for
publication.

105. gleicher@cs.wisc.edu