This document summarizes a senior design project to create a sign language teaching glove. The goal was to develop a portable device that can accurately detect gestures and provide feedback to users learning sign language. The system included sensors to detect gestures, software to analyze the data using Kalman filtering and pattern recognition, and LEDs for feedback. While some components like the MPU-6050 motion sensors worked well, other parts like the flex sensors and Bluetooth module failed. The project achieved 75% accuracy in recognizing gestures but aims to improve performance by addressing current limitations.

1. SP13 ECE 445: Senior Design
Sign Language Teaching
Glove
Project #29:
Reebbhaa Mehta, Daniel Fong, Mayapati Tiwari
TA: Igor Fedorov

2. Introduction
• Motivation
- No available portable devices to teach sign language
• Objective
- Sensing unit to detect gestures accurately
- Kalman Filter to reduce noise from data
- Program to check each gesture
- Provide feedback to user

3. American Sign Language

4. Overview

5. System Components
• Three Main Units
- Sensing unit (MPU 6050 & Flex Sensors)
- Software Component
- Feedback unit (LED’s)

6. System Components
• Hardware
- Power Supply – 9V
- Microcontroller Unit – Arduino Uno
- 5 flex Sensors & 5 MPU 6050
- Bluetooth
- 5 Sensors
- LED Driver
• Software
- Arduino Programming Environment
- Kalman Filter
- Perceptron Learning Algorithm

7. Power Supply – Actual Choice
• First Choice – Lithium Backpack
- Supplies 5V to power Arduino Uno
- Supplies 3.3V to power all other components
- Matches Arduino size and fits on the back
- Rechargeable via USB
- Shorted !!

8. Power Supply – Alternate
• Alternate – Energizer 9V
- Supplies 9V
- Arduino on-board regulator generate 5V to power
Arduino Uno
- Arduino on-board regulator generate 3.3V to power all
other components
- Not very useful for effective space
utilization
- Not rechargeable

9. Sensing Unit
Consists of:
- 5 accelerometers and gyroscopes (MPU 6050)
- 5 flex sensors (FLX-03)
- I2C Multiplexer (TCA9548A)

10. Accelerometer & Gyroscope
MPU-6050
- Accelerometer & gyroscope in one chip
- Helps to detect gestures
- Accelerometers detect tilt
- Gyroscopes detect angular velocity
- Better space utilization
- Placed near fingertips – PCB’s
need to be small

11. MPU-6050

12. MPU-6050 Data
• Orientation from gravity
•
•
• Gyroscope for change in orientation

13. MPU 6050 Schematic

14. MPU 6050 PCB Pic

15. Flex Sensors
• Uni-directional flex sensors (FLX-
03) used
• Help to provide more accurate data
• Range: 10kΩ to 40kΩ
- For unflexed hand: 10kΩ
- For completely flexed hand: 40kΩ

16. Flex Sensor Circuit

17. Flex Sensor – Test & Data
0.5
1.5
2.5
9 14 19 24
Voltage
(V)
Resistance (Kohm)
V vs R
R (Flex Sensor) Vout
9.48 kΩ 1.78 V
15.3 kΩ 1.69 V
17.0 kΩ 1.59 V
21.2 kΩ 1.36 V
22.7 kΩ 1.32 V

18. Flex Sensor - Problems
• Problem
- Broke due to heat
- Used copper tape to fix the
problem without success
- Only one working flex sensor
• Consequences
- Less data from sensing unit
- Reduced accuracy to differentiate between gestures
- Failed requirement

19. I2C Multiplexer (TCA9548A)
• 8 bi-directional translating switches
• I2C bus compatible
• Channel selection via I2C bus
• 8-channel I2C switch communicate
with up to 8 I2C devices which have
the same address

20. Feedback Circuit
• Controls 10 LEDs
using 8 bit shift
register
• 3 arduino output
pins.

21. Microcontroller
• Arduino Uno
- Serial Communication
- Easily programmable
- Attaches to Lithium Backpack
(better space utilization)
- Voltage regulators to provide
3.3V and 5V
- Works well with external battery
- I2C protocol MPU 6050 communication library

22. Bluetooth
• Bluetooth Shield
- Arduino Uno compatible
- UART communication
- Up to 10m communication
- Fits the back of Arduino Uno
Requirement Failed: No
connection with computer.
Reason: Broken antenna or
chip not programmable

23. Kalman Filtering
• F = state
• u = gyroscope data
• Var(w) = Q
• Var(v) = R
• H = [1;0]
• B = delta (T)

24. Kalman Filtering
• Estimation:

25. Kalman Filtering
• Update:

26. Kalman Filtering
Test A Sensor 1 – Little Finger

27. Kalman Filtering

28. Kalman Filtering

29. Kalman Filtering

30. Perceptron Learning Algorithm
•On-line, mistake driven algorithm.
•Linear classifier that updates the weight vector
incrementally when mistakes are made.
•Decision rule checks whether the dot product of the
weight vector with an input vector is greater that some
threshold .
•

31. Perceptron Learning Algorithm

32. Failed Verification
Problem Reason
Bluetooth Initialization code
stopped working
after a while
Defective piece
(maybe)
Flex Sensors Not within the
specified range
Broke due to heat –
resistive strip came
off
Perceptron Wanted – 98%
accuracy
Achieved – 75%
No data from flex
sensor.
Adding more
variables.
Feedback Unit Not integrated No real time data
processing

33. Accomplishments
• Optimum data from MPU-6050
• Kalman Filter
• Perceptron to check gestures
• Effectively differentiate between 5 gestures (A, B, L, V
and Y)
• LED for feedback
• Soldering very small components like MPU-6050

34. Future Steps
• Replacing broken flex sensors to get more
accurate data for each gesture
• Increase the accuracy of perceptron by adding more
features
• Real time implementation
• Better feedback unit with haptic feedback and LED’s
• With more accuracy and better detection words can be
added to the library and thus progress can be made

35. Acknowledgement
Prof. Scott Carney
Igor Fedorov
Mark Smart
Skot Wiedmann
Waltham Smith
Daniel Mast
Aadhar Jain
Joseph Shim

36. Questions?