Your SlideShare is downloading. ×
0
A Cheap Chess Robot: PerceptionBilly Okal, Oliver Dunkley, Andreas N¨chter                                     u       Aut...
Introduction Methodology Experiments Conclusion and Future Work Outline      1   Introduction             Motivation      ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work                                                           ...
Introduction Methodology Experiments Conclusion and Future Work scene to 2D view Detect Move and Query Chess Engine       ...
Introduction Methodology Experiments Conclusion and Future Work scene to 2D view Detect Move and Query Chess Engine       ...
Introduction Methodology Experiments Conclusion and Future Work Conclusion and Future Work             Conclusions        ...
Introduction Methodology Experiments Conclusion and Future Work Conclusion and Future Work             Conclusions        ...
Introduction Methodology Experiments Conclusion and Future Work The End                                   Thank you for yo...
Upcoming SlideShare
Loading in...5
×

Development of A cheap chess robot: Planning and Perception

1,719

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,719
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Development of A cheap chess robot: Planning and Perception"

  1. 1. A Cheap Chess Robot: PerceptionBilly Okal, Oliver Dunkley, Andreas N¨chter u Automation and Machine Vision Lab Jacobs University Bremen May 13, 2011
  2. 2. Introduction Methodology Experiments Conclusion and Future Work Outline 1 Introduction Motivation Previous/Related Work Research Agenda 2 Methodology Finding the chessboard Detecting Changes Determining Moves 3 Experiments From scene to 2D view Detect Move and Query Chess Engine 4 Conclusion and Future Work Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  3. 3. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Motivation What is Robot Chess? Robot playing a game of chess against another robot or human opponent. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  4. 4. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Motivation What is Robot Chess? Robot playing a game of chess against another robot or human opponent. We focus on the case of a Robot vs a Human Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  5. 5. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Motivation What is Robot Chess? Robot playing a game of chess against another robot or human opponent. We focus on the case of a Robot vs a Human What is Robot Chess Perception? Robot Perception generally enables the robot to perceive the changes in the environment using its sensors and interpret sensory input to make decisions about its next actions. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  6. 6. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Motivation What is Robot Chess? Robot playing a game of chess against another robot or human opponent. We focus on the case of a Robot vs a Human What is Robot Chess Perception? Robot Perception generally enables the robot to perceive the changes in the environment using its sensors and interpret sensory input to make decisions about its next actions. In our case, the sensor is a monocular camera, the environment is a chess game and actions include moving the chess pieces on the chessboard. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  7. 7. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  8. 8. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; The AAAI small scale manipulation(table top chess) challenge during AAAI2010 and AAAI2011 Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  9. 9. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; The AAAI small scale manipulation(table top chess) challenge during AAAI2010 and AAAI2011 Modified Chessboard [http://chiara-robot.org/] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  10. 10. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; The AAAI small scale manipulation(table top chess) challenge during AAAI2010 and AAAI2011 Carefully chosen colors for pieces and board Non-standard chessboards Non standard sizes Modified Chessboard [http://chiara-robot.org/] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  11. 11. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; The AAAI small scale manipulation(table top chess) challenge during AAAI2010 and AAAI2011 Carefully chosen colors for pieces and board Non-standard chessboards Non standard sizes Modified Chessboard [http://chiara-robot.org/] Systems that involved mounting the camera directly above the chessboard as in [Urting:2003] and [Groen:1992] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  12. 12. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Previous/Related Work We highlight a few interesting/related cases; The AAAI small scale manipulation(table top chess) challenge during AAAI2010 and AAAI2011 Carefully chosen colors for pieces and board Non-standard chessboards Non standard sizes Modified Chessboard [http://chiara-robot.org/] Systems that involved mounting the camera directly above the chessboard as in [Urting:2003] and [Groen:1992] A fews humanoids like REEM-A(30 dof, stereo-cam) by Pal Robotics. etc Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  13. 13. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Research Agenda Common Feature of Previous Work Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  14. 14. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Research Agenda Common Feature of Previous Work Constraints imposed on the chessboard and pieces to simplify the problem. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  15. 15. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Research Agenda Common Feature of Previous Work Constraints imposed on the chessboard and pieces to simplify the problem. Which of these constraints can we relax/get rid of? Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  16. 16. Introduction Methodology Experiments Conclusion and Future Work Motivation Previous/Related Work Research Agenda Research Agenda Common Feature of Previous Work Constraints imposed on the chessboard and pieces to simplify the problem. Which of these constraints can we relax/get rid of? Our Attempt Allow for camera to move relative to chessboard (including moving out of the scene) Use a un-modified chessboard Use standard chess piece set (Staunton standard tournament pieces No 5 - from Amazon) All the above using cheap camera (a USB webcam) and basic hardware (4 dof + 1 for gripper). Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  17. 17. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Methodology Finding the chessboard in the scene Detecting Changes on the chessboard (moves) Decoding the changes to identify user moves Generating new moves and transform these into free space poses for planning Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  18. 18. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Finding the chessboard in the scene Algorithm 1: Detect Board in the Scene Data: Raw Scene Image (img) - 800x600 Result: 4 Corners of the chessboard begin img edges ←− EdgeDetect(img ); all lines ←− LineDetect(img edges); MergeCloseLines(all lines); bounding lines ←− FindBoundingLines(all lines); corners ←− ComputerIntersections(bounding lines); return corners; end Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  19. 19. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Finding the chessboard in the scene Algorithm 2: Detect Board in the Scene Data: Raw Scene Image (img) - 800x600 Result: 4 Corners of the chessboard begin img edges ←− EdgeDetect(img ); all lines ←− LineDetect(img edges); MergeCloseLines(all lines); bounding lines ←− FindBoundingLines(all lines); corners ←− ComputerIntersections(bounding lines); return corners; end Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  20. 20. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Finding the chessboard in the scene Algorithm 3: Detect Board in the Scene Data: Raw Scene Image (img) - 800x600 Result: 4 Corners of the chessboard begin img edges ←− EdgeDetect(img ); all lines ←− LineDetect(img edges); MergeCloseLines(all lines); bounding lines ←− FindBoundingLines(all lines); corners ←− ComputerIntersections(bounding lines); return corners; end Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  21. 21. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Project the chessboard to a 2D view Homography/Projective Transformation Given a 2D point p = (x, y ) on the image plane, find a matrix H such that the point p = (x , y ) given by p = Hp lies on a 2D plane P 2 of choice. In homogenous coordinates we have; a b p = (a, b, c), s.t x = , y= and c = 0 (1) c c a b p = (a , b , c ), s.t x = , y = and c = 0 (2) c c      a h1 h2 h3 a  b  =  h4 h5 h6   b  (3) c h7 h8 h9 c [Hartley:2000] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  22. 22. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Project the chessboard to a 2D view We plugin all the correspondences for the four corners and arrive at the following system;      a1 b1 1 0 0 0 −a1 a1 −b1 a1 h1 a1   a2 b2 1 0 0 0 −a2 a2 −b2 a2   h2     a2     a3 b3 1 0 0 0 −a3 a3 −b3 a3   h3     a3     a4 b4 1 0 0 0 −a4 a4 −b4 a4   h4   = a4   (4)   0 0 0 a1 b1 1 −a1 b1 −b1 b1   h5     b1     0 0 0 a2 b2 1 −a2 b2 −b2 b2   h6     b2    0 0 0 a3 b3 1 −a3 b3 −b3 b3  h7   b3  0 0 0 a4 b4 1 −a4 b4 −b4 b4 h8 b4 Which then can be solved using a number of well established algorithms. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  23. 23. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Project the chessboard to a 2D view We plugin all the correspondences for the four corners and arrive at the following system;      a1 b1 1 0 0 0 −a1 a1 −b1 a1 h1 a1   a2 b2 1 0 0 0 −a2 a2 −b2 a2   h2     a2     a3 b3 1 0 0 0 −a3 a3 −b3 a3   h3     a3     a4 b4 1 0 0 0 −a4 a4 −b4 a4   h4   = a4   (4)   0 0 0 a1 b1 1 −a1 b1 −b1 b1   h5     b1     0 0 0 a2 b2 1 −a2 b2 −b2 b2   h6     b2    0 0 0 a3 b3 1 −a3 b3 −b3 b3  h7   b3  0 0 0 a4 b4 1 −a4 b4 −b4 b4 h8 b4 Which then can be solved using a number of well established algorithms. In our implementation we make use of a OpenCV methods cv::getPerspectiveTransform and cv::warpPerspective. [Hartley:2000] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  24. 24. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Detecting Changes on the chessboard(moves) Locate blobs/change spots 1 Difference image (Given two images f (x, y ) at time t1 and f (x , y ) at time t2 , difference is g (x, y ) = |f − f |, [Farahat:2003] 2 Remove noise by erosion and dilation 3 Distance Transform (label pixels with distance to background) 4 Find the pixel whose label is largest 5 Flood fill the pixel above 6 Repeat steps 3,4,5 until threshold Map change spots to board changes. Use image dimensions and know chess square sizes to place changes on board squares. Generate change strings. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  25. 25. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Detecting Changes on the chessboard(moves) Locate blobs/change spots 1 Difference image (Given two images f (x, y ) at time t1 and f (x , y ) at time t2 , difference is g (x, y ) = |f − f |, [Farahat:2003] 2 Remove noise by erosion and dilation 3 Distance Transform (label pixels with distance to background) 4 Find the pixel whose label is largest 5 Flood fill the pixel above 6 Repeat steps 3,4,5 until threshold Map change spots to board changes. Use image dimensions and know chess square sizes to place changes on board squares. Generate change strings. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  26. 26. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Detecting Changes on the chessboard(moves) Locate blobs/change spots 1 Difference image (Given two images f (x, y ) at time t1 and f (x , y ) at time t2 , difference is g (x, y ) = |f − f |, [Farahat:2003] 2 Remove noise by erosion and dilation 3 Distance Transform (label pixels with distance to background) 4 Find the pixel whose label is largest 5 Flood fill the pixel above 6 Repeat steps 3,4,5 until threshold Map change spots to board changes. Use image dimensions and know chess square sizes to place changes on board squares. Generate change strings. Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  27. 27. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Determining chess moves from the changes Convert chessboard changes into moves Compare every pair of changes say a2b3 against an 8 × 8 state matrix and return a move string Repeat over the whole change string Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  28. 28. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Determining chess moves from the changes Convert chessboard changes into moves Compare every pair of changes say a2b3 against an 8 × 8 state matrix and return a move string Repeat over the whole change string Query chess engine for moves based on detected moves Using standard chess engine communication protocol (UCI - Universal Chess Interface). Communication handles via ROS messages Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  29. 29. Introduction Methodology Experiments Conclusion and Future Work Finding the chessboard Detecting Changes Determining Moves Determining chess moves from the changes Convert chessboard changes into moves Compare every pair of changes say a2b3 against an 8 × 8 state matrix and return a move string Repeat over the whole change string Query chess engine for moves based on detected moves Using standard chess engine communication protocol (UCI - Universal Chess Interface). Communication handles via ROS messages Convert suggested moves into poses in free space Poses are defines as a pair of Position p = (x, y , z)T and Orientation q = (x, y , z, w )T to allow for design freedom in planning. Using chessboard size, we interpret a move string say a2b3 into respective pose say p = (24, 14, 2)T q = (0, 0, 0, 1)T Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  30. 30. Introduction Methodology Experiments Conclusion and Future Work scene to 2D view Detect Move and Query Chess Engine From Find Board and project it to 2D View [[Movie]] Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  31. 31. Introduction Methodology Experiments Conclusion and Future Work scene to 2D view Detect Move and Query Chess Engine From Determine Move and query Chess Engine Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  32. 32. Introduction Methodology Experiments Conclusion and Future Work Conclusion and Future Work Conclusions We have demonstrated that the constraint of using a modified chessboard and/or pieces can be relaxed without compromising performance We demonstrated on very minimalistic hardware, use of better hardware should produce even better performance Camera need not be fixed Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  33. 33. Introduction Methodology Experiments Conclusion and Future Work Conclusion and Future Work Conclusions We have demonstrated that the constraint of using a modified chessboard and/or pieces can be relaxed without compromising performance We demonstrated on very minimalistic hardware, use of better hardware should produce even better performance Camera need not be fixed Future Work Investigate how the task could be solved multiple cameras (stereo vision) Adding other types of sensors (e.g. Kinect) Add object recognition modules Improve the blob detection module Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  34. 34. Introduction Methodology Experiments Conclusion and Future Work The End Thank you for your attention Questions/Comments? Special Thanks DAAD/PROFIN, Automation Group Members For further information check B.Okal and O.Dunkley, ”A Cheap Chess Robot: Planning and Perception”, Bachelor Thesis 2011. Software Tools http://www.ros.org http://opencv.willowgarage.com http://openrave.programmingvision.com Billy Okal, Oliver Dunkley, Andreas N¨chter u A Cheap Chess Robot: Perception
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×