This document provides an overview of swarm robotics. It begins with examples of decentralized control and self-organization in natural swarms like ants and bees. It then discusses how swarm robotics takes inspiration from these systems, using local control methods, local communication, and self-organization to complete collective tasks without centralized control. The rest of the document focuses on a proposed system for gesture recognition to allow human control of swarm robots. It describes hand detection, feature extraction, and hardware implementation using three foot-bot robots. It concludes with potential applications of swarm robotics and areas for future work.

1. Swarm robotics

2. PRESENTATION OVERVIEW
 Introduction To Gestures
 Design Of The Proposed System
 Hand Detection
 Hardware Implementation
 Conclusion / Future Work

3. Real World
Insect
Examples

4. Natural swarms
 Decentralised – no-one in control
 Individuals are simple and autonomous
 Local communication and control
 Cooperative behaviours emerge through self-
organisation
e.g. repairing damage to nest, foraging for food,
caring for brood

5. Bees

6. An In-depth Look at Real
Ant Behaviour
Interrupt The Flow

7. The Path Thickens!
Adapting to Environment Changes

8. The New Shortest Path
Adapting to Environment Changes

9. Welcome to the
Real World

10. Robots
• Collective task completion
• No need for overly complex
algorithms
• Adaptable to changing environment

11. Swarm robotics
 Inspired by self-organisation of social insects
 Using local methods of control and communication
Local control: autonomous operation
Local communication: avoids bottlenecks
Scalable – new robots can be added, or fail without need for
recalibration
Simplicity – cheap, expendable robots
 Self-organisation
 Decentralisation

12. Collective Robotics
Swarm Robotics

13. Introduction To Gestures
 Gestures can originate from any body
motion.
 Commonly from face/hand.
 Gesture recognition-understand human
body language.
 Help human to interact with machines
without any mechanical devices.

14. Disadvantages of centralised
control and communication.
 Central control: failure of controller implies
failure of whole system
 Robot to robot communication becomes very
complex as number of robots increases.
 Communication bottlenecks
 Adding new robots means changing the
communication and control system

15. Design Of The Proposed System
 Hand detection
 Feature extraction
 Gesture recognition
 Goal directed navigation of swarm
robots.

16. Hand Detection
 Hand detection
– Detection of hand in an image, background objects are
avoided for feature extraction.
– Skin color is the key component.
– Detecting skin and non-skin.
– Detecting image pixels and regions that contains skin-
tone color.
– Background is controlled.
– Appearance depends on illumination conditions.
 Two phases
– Training phase
– Detection phase

17. Training Phase
 Three steps
– Collecting a database of skin patches from different
images
– Choosing a suitable color space
– Learning the parameters of skin classifier

18. Detection Phase
 Two steps
Converting the image into some color space that was
used in training phase.
Classifying each pixel using the skin classifier to either
a skin or non-skin.
 RGB color space
 Skin classifier
Variety of classification techniques
Any pixel which color falls inside the skin color class
boundary is labeled as skin.

19. Feature Extraction
 Feature-An interesting part of an image.
 No exact definition.
 Depends on the problem.
 Transforming the input data into set of features.
 Result is a feature vector.
 Features extracted are invariant to image scaling,
rotation and less affected to changes in
illumination.
 SIFT feature extraction.

20.  Hand tracking and Feature extraction.

21. Image frame
from webcam
Tracking
hand by
skin detection
SIFT feature
extraction
Finding
match
Gesture
database
Gesture1 Gesture2 Gesture3
Action1 Action2 Action3
Navigation of
swarm

22. Hardware Implementation
 Foot-bot robot

24. Applications Of The System
 Three foot-bots.

25. Applications of swarm approach
Some tasks are particularly suited to group of expendable
simple robots
e.g. - cleaning up toxic waste
- exploring an unknown planet
- pushing large objects
- surveillance and other military applications

26. Conclusion
 Hand detection and feature extraction removes
noise from the image.
 System performance and accuracy will
increase.
 Swarm robots movement can be controlled
through gestures.
Future Work
Speech recognition.

27. Dumb parts, properly
connected into a swarm,
yield smart results.
Kevin Kelly

28. Satellite
Maintenance
The Future?
Medical
Interacting
Chips in
Mundane Objects
Cleaning Ship
Hulls
Pipe
Inspection
Pest Eradication
M
iniaturization
EngineMaintenance
Telecommunications
Self-Assem
bling
Robots
Job Scheduling
Vehicle
Routing
Data
Clustering
Distributed
M
ail
System
s
O
ptim
al
Resource
Allocation
Combinatorial
Optimization

29. 1. C.C.Wang, K.C.Wang.: Hand Posture Recognition Using Adaboost with SIFT For Human
Robot Interaction, in Robotics: Viable Robotic Service to Human, Springer-2009.
2. M. Kolsch and M. Turk.: Robust hand detection, in IEEE International Conference on
Automatic Face and Gesture Recognition, 2004.
3. Cristina Manresa, Javier Varona, Ramon Mas and Francisco J.Perales,.: Hand Tracking
and Gesture Recognition for Human-Computer Interaction , Electronic Letters on
Computer Vision and Image Analysis 5(3):96-104, 2005.
4. Alessandro Giusti, Jawad Nagi, Luca M. Gambardella, Gianni A. Di Caro : Distributed
Consensus for Interaction between Humans and Mobile Robot Swarms (Demonstration).
5. Ihab Zaqout, Roziati Zainuddin, Sapian Baba,.: Pixel-Based Skin Color Detection
Technique, in Machine Graphics and Vision, 2005. FLEXChip Signal Processor
(MC68175/D), Motorola, 1996.
6. Qiu-yu Zhang, Mo-yi Zhang, Jian-qiang Hu,.: Hand Gesture Contour Tracking Based on
Skin Color Probability and State Estimation Model, Journal of Multimedia, Vol. 4, No. 6,
December 2009. A. Karnik, “Performance of TCP congestion control with rate feedback:
TCP/ABR and rate adaptive TCP/IP,” M. Eng. thesis, Indian Institute of Science,
Bangalore, India, Jan. 1999.
7. Lars Bretzner, Ivan Laptev, Tony Lindberg,.: Hand Gesture Recognition using Multi-Scale
Colour Features, Hierarchical Models and Particle Filtering , Proceedings of the Fifth
IEEE International Conference on Automatic Face and Gesture Recognition (2002).
Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification,
IEEE Std. 802.11, 1997.
References

30. Contd…
• G. Lee and N. Y. Chong, “Decentralized formation control for small scale
robot teams with anonymity,” Mechatronics, vol. 19, no. 1, pp.85–
105,2009.
• G. Lee and N. Y. Chong, “A geometric approach to deploying robot
swarms,” Ann. Math. Artif. Intell., vol. 52, no. 2–4, pp. 257–280, 2009.
• E. Sahin, “Swarm robotics: From sources of inspiration to domains of
• application,” in Proc. 8th Int. Conf. Simulation of Adaptive
Behavior(LNCS),2005,vol.3342,pp.10–20.
• H. Niwa, K. Kodaka, Y. Sakamoto, M. Otake, S. Kawaguchi, K. Fujii, Y.
Kanemori, and S. Sugano, “GPS-based indoor positioning system with
multi-channel pseudolite,” in Proc. IEEE Int. Conf. Robot. Autom., 2008,
pp. 905–910. Faraj Alhwarin, Chao Wang, Danijela Risti -Durrant, Axel
Graser, Improved SIFT-Features Matching for Object Recognition, BCS
International Academic Conference- Visions of Computer Science,2008.

31. Thank you