The document discusses a new approach for robots to learn grasping without human supervision. The robot collects its own training data by executing random grasps on objects and using force sensors to recognize successful grasps. As of now, the robot has executed over 17,000 random grasps, with around 1,300 successful grasps recorded along with image and depth data. This dataset is larger than existing human-labeled datasets and also includes information on failed grasps. The goal is for the robot to autonomously learn knowledge and understanding of grasping from its own experiences rather than relying on human knowledge transfers.

1. Learning to grasp without
human supervision

2. Motivation
• Current methods in grasp learning for previously unseen objects
involve the transfer of features and primitives from humans to
robots.
• These transferred knowledge includes memory of previous
grasps[], human labelled datasets[], pre-defined visual/depth
features[]
• But these previous methods are inherently biased towards a
human’s knowledge and intuition
• For truly autonomous manipulators, we need a framework that
allows the robot itself to learn the knowledge and understanding
for grasping

3. Our Approach
• We follow a learning by training approach where we have a dataset of images (and
depth point clouds) of objects along with positive and negative grasp points
• Since we need to eliminate the human in the loop, the robot itself should be able to
collect training data
• For this we have a robot execute random grasps on a wide variety of objects. The
robot recognizes successful grasps using its inbuilt gripper force sensors

4. Grasp Data collected
• Currently our robot is capable of executing about 80 random grasps
an hour per arm. Which on a typical day(running for about 8 hours)
yields about 80*8*2 = 1280 grasps a day.
• As of now, the robot has executed about 17,000 random grasps and
has around 1300 successful grasps. This is already larger than
Cornell Grasp Dataset that is hand labelled and doesn't contain
negative examples (failure grasps)
• Associated with each grasp we record data from 2 RGB image
cameras on the arm, 1 Depth camera (Creative Senz3D) and Kinect
V2 attached to the head of the robot. The recorded streams include
the entire motion of the arm in the grasp sequence

5. Grasp Data collected