This document describes a proposed system to translate hand gestures and sign language into text in real time using Google's Mediapipe artificial intelligence tools. It would help improve communication for deaf or mute individuals. The system would use machine learning and deep learning neural networks to extract features and classify gestures captured by a camera in real time. Mediapipe is highlighted as an improvement over openCV for more reliable and accurate hand detection by overcoming issues like needing a clean background and proper lighting.

1. A Hand Gesture Sign
Language to Text
Real Time Interpreter
using Google Mediapipe
Artificial Intelligence

2. Contents
• Hand Gesture & Sign Language.
• Problem statement.
• Background work.
• Cons
• What is Mediapipe?
• How it Work
• Conclusion.

3. Hand Gesture & Sign Language

4. • Aimed at innovating a device to help those without the knowledge of sign
language to communicate with the group of people.
• How might we improve communication between the hearing-impaired or mute by
using a real time visual hand gesture/sign language to text interpreter?
Problem Statement

5. The proposed system is based on the concept of
• Machine learning.
• Artificial intelligence using the deep learning neural network for
feature extraction and classification.
• Google Mediapipe Hand Gesture Recognition API.
• OPenCV(Open Source Computer Vision Library).
Background Work

6. Drawback : With openCV
Hand gesture detection was too dependent on many factors like:
• clean background.
• proper lighting for an accurate hand detection.
How can we Overcome it?
This inconsistency lead the team to research on other possible ways to
make detection more reliable and accurate which lead us to Google’s
Mediapipe.

7. Mediapipe
Mediapipe is a graph-based framework for building multimodal
(video, audio, and sensor) applied machine learning pipelines.
Mediapipe Hands

8. Hand Landmark with Mediapipe

9. How it works?

10. Capturing Gesture Data

11. A system for real-time hand gesture sign language
recognition, which we can use to communicate with deaf
and mute people.
Conclusion