TONGUE DRIVE SYSTEM

           Presented by:
                   Anagha Balakrishnan
           ER & DCI-IT,Vellayambalam
                           RollNo: 07
What is TDS?

TONGUE–OPERATED            assistive   technology for

 people    with   severe      disability   to   control

 their    environment



DEVELPOED by GT Bionics Lab
Assistive Technology ?

ASSISTIVE TECHNOLOGY is an umbrella term that

 includes assistive, adaptive, and rehabilitative devices

 for people   with disabilities and also        includes the

 process   used   in   selecting,   locating,    and using

 them.
Why Spinal Cord Injury is Fatal?

              •SPINAL CORD is located inside the spine


              •CONNECTS the brain to nerves in the

              body


              •MESSAGE travel in very high speed from


              brain to rest of the body
Cause of Spinal Cord Injury…
 SPINAL CORD
PARALYSIS          is squashed or blood circulation of

  the spinal cord is cut off, which will result in..
UNABLE to easy breath


UNABLE to feel pain or sensation


SORES on skin not being able to move around


SEXUAL dysfunction
Existing Assistive Technologies…


   Sip-and-Puff wheel chair

                               •USE air pressure to control
                               wheel chair by sipping and
                               puffing
                               •NOT good for people with
                               week breathing
   Voice activated powered wheelchair

                            •WHEN user speaks

                            commands, a microphone in a

                            throat detects the vibration of

                            vocal cord

                            •UNSTABLE for powered

                            wheelchair control in noisy
   Head control

•WHEELCHAIR controlled by

head control device

•NOT good for the

users with bad head movement

ability
Why Tongue?...
   TONGUE and mouth occupies the major part



   MUSCLE fibers in tongue is similar to heart muscle fibers



   LOW rate of Perceived Exertion



   DIRECTLY connected to the brain



   HIDDEN inside mouth will give a certain degree of privacy
TDS – Overview
TDS –Overview…
   SMALL permanent magnet pierced on the tongue


   ARRAY of Hall-effect magnetic      sensors senses
    the magnetic field


   SENSORS mounted on a dental retainer and
    attached on the outside of the teeth


   SMALL batteries are intended to power
   POWER management circuitry scans through the
    sensors and turn on one at a time


   TDM analog output are digitized, modulated and
    transmitted to the external control unit through a
    wireless link


   FROM their signals are demodulated, demultiplexed to
    extract individual sensor output


   BY processing these output command is identified
TDS Prototypes
   BUILD on a face shield


   FUNCTION is to directly emulate mouse pointing and
    selection function with the tongue movement


   SIX COMMANDS : up, down, right, left, single click &
    double click


   SSP algorithm running in background
   MAGNETIC TRACER : small, cylindrical, rare-earth
    permanent magnet


   PAIR of two-axis magnetic field sensor modules- each
    contained a pair of orthogonal magneto-inductive sensor


   THREE AXIS MODULE: used as a reference electronic
    compass to minimize the effect of external magnetic field
    interference
   CONTROL unit & reference compass hidden under face
    shield cap


   ENTIRE system was powered by a 3.3V coin-sized
    battery(CR2023)
   SENSOR output where send serially to the ultralow-
    power MSP430 microcontroller


   MICROCONTROLLER took 11 samples from each sensor


   SAMPLES are arranged in a data frame and wirelessly
    transmitted to a PC across a 2.4 GHz wireless link
    established between nRF2401 transceivers
Sensor Signal Processing algorithm
   DEVELOPED in MATLAB environments


   2 phases: Training & Testing
      •TRAINING : USES principal components analysis

      (PCA) to extract the most important features of the

      sensor output waveforms for each specific command

      •             USER repeats each of the six designated

      commands 10 times in 3-second intervals
•                     TOTAL of 12 samples (3 per sensor)

are recorded in 12-variable vectors

•                     THE PCA-based feature-extraction

algorithm calculates the eigenvectors and Eigen values

based on the 12-variable vectors

•                 THREE eigenvectors with the largest

Eigen values are then chosen to set up the feature

matrix [v1, v2, v3]
•                    BY multiplying the training vectors
with the feature matrix, the SSP algorithm forms a
cluster (class) of 10 data points from training for each
specific command


•TESTING: k-nearest neighbour (kNN) classifier is then
used in real time to evaluate the proximity of the
incoming data points to the clusters formed earlier in
the training phase
•           kNN starts at the incoming new data point

and inflates an imaginary sphere around that data point

until it contains a certain number (k) of the nearest

training data points

•           IT associates the new data point to the

command that has the majority of the training data

points inside that spherical region.
AFTER finding the intended user command,

the mouse pointer starts moving slowly in the selected

direction

             FOR faster access the user can hold his or

her tongue in the position of the issued command and

the pointer will gradually accelerate until it reaches a

certain maximum velocity
Training Session

   GRAPHICAL user interface (GUI) prompted the user to

    define each command by moving his tongue from its

    resting position to the corresponding command position

    when the command light was on and returning it back to

    the resting position when the light went off
   THIS procedure was repeated 10 times for the entire set of

    six commands plus the tongue resting position, resulting

    in a total of 70 training data points.
Implementation

        - Controlling a Powered
Wheel Chair
Block diagram & hardware component

   SYSTEM consists of the hall effect sensors (A1231),
    MSP430 microcontroller, H-Bridge driver (SN754410)
   4 ADC channels of MSP430 convert analog signals from

    sensors to digital for processing



   BASED on the processed information microcontroller

    drives the driver IC



   DRIVER IC drives the DC powered wheelchair
   SENSOR- Linear Bipolar Hall effect sensor
             TRANSDUCER which varies its output voltage in

response to changes in magnetic field


•VOLTAGE:4.5V-5.5V

•OUTPUT is an analog

voltage that vary from 0-5V

•THAT is converted to a

digital value by ADC
   H-BRIDGE DRIVER : used to drive the wheelchair



   SN7454410:BidirectionalH-Bridge driver



   5V power supply is given



   SPEED of the wheelchair is controlled by the PWM signals

    from MSP430
•4 SWITCHES


•WHEN A and D are closed, a


positive voltage applied across the


motor


•WHEN B and C are closed , voltage

is reversed, allowing reverse operation


of the motor
   DC Motors and Gear Box
•5V DC motors with a gearbox for driving

the vehicle

•USED 2 motors: Left and right

•LEFT motor at rest and right in motion

turn vehicle left and vice versa

•FOR forward and backward motion both

the motors are driven in same direction

with same speed
Software Components
   ADC10(Analog to Digital converter)

    •MSP430G2252 has 8 ADC channels of 10 bit each


    •ADC10 convert the analog signal from the sensor into digital


    value


    •WE get the value ranging from 0-1023
   Pulse Width Modulation

     •MODULATING technique which generates variable width

     pulses is used to vary the speed of the motor
   Algorithm

     •ROUND ROBIN algorithm that implements polling technique

     •SENSOR values are always read one after other in a continuous

     loop

     •FEW threshold values set for each sensor

     •IF reading from sensors reaches the threshold value program

     triggers the change of state
   S1 & S2 front sensor , S3 & S4 back sensors



   WHITE circle in the middle- resting position of tongue



   DIFFERENT flag values for each sensor



   IF sensor is high, then the flag value remains high until

    the operation is performed, then it is reset again
   f1 - S1 > 750 // forward


   f2 - S2 > 600 // backward


   f3 - S3 > 700 // speed increase


   f4 - S4 > 700 // speed decrease


   f5 - S1 > 600 & S3 > 600 // turn left


   f6 - S2 > 600 & S4 > 600 // turn right
Advantages
   SIMPLE to implement , low cost, easy to operate
    flexible


   NO surgery needed


   OFFERS better privacy to the user
Drawbacks

   USERS should avoid inserting ferromagnetic objects in

    their mouth



   MAGNETIC tracer should be removed if the user is

    undergoing MRI
QUESTION TIME
Tongue drive system

Tongue drive system

  • 1.
    TONGUE DRIVE SYSTEM Presented by: Anagha Balakrishnan ER & DCI-IT,Vellayambalam RollNo: 07
  • 2.
    What is TDS? TONGUE–OPERATED assistive technology for people with severe disability to control their environment DEVELPOED by GT Bionics Lab
  • 3.
    Assistive Technology ? ASSISTIVETECHNOLOGY is an umbrella term that includes assistive, adaptive, and rehabilitative devices for people with disabilities and also includes the process used in selecting, locating, and using them.
  • 4.
    Why Spinal CordInjury is Fatal? •SPINAL CORD is located inside the spine •CONNECTS the brain to nerves in the body •MESSAGE travel in very high speed from brain to rest of the body
  • 5.
    Cause of SpinalCord Injury…  SPINAL CORD PARALYSIS is squashed or blood circulation of the spinal cord is cut off, which will result in.. UNABLE to easy breath UNABLE to feel pain or sensation SORES on skin not being able to move around SEXUAL dysfunction
  • 6.
    Existing Assistive Technologies…  Sip-and-Puff wheel chair •USE air pressure to control wheel chair by sipping and puffing •NOT good for people with week breathing
  • 7.
    Voice activated powered wheelchair •WHEN user speaks commands, a microphone in a throat detects the vibration of vocal cord •UNSTABLE for powered wheelchair control in noisy
  • 8.
    Head control •WHEELCHAIR controlled by head control device •NOT good for the users with bad head movement ability
  • 9.
  • 10.
    TONGUE and mouth occupies the major part  MUSCLE fibers in tongue is similar to heart muscle fibers  LOW rate of Perceived Exertion  DIRECTLY connected to the brain  HIDDEN inside mouth will give a certain degree of privacy
  • 11.
  • 12.
    TDS –Overview…  SMALL permanent magnet pierced on the tongue  ARRAY of Hall-effect magnetic sensors senses the magnetic field  SENSORS mounted on a dental retainer and attached on the outside of the teeth  SMALL batteries are intended to power
  • 13.
    POWER management circuitry scans through the sensors and turn on one at a time  TDM analog output are digitized, modulated and transmitted to the external control unit through a wireless link  FROM their signals are demodulated, demultiplexed to extract individual sensor output  BY processing these output command is identified
  • 14.
  • 15.
    BUILD on a face shield  FUNCTION is to directly emulate mouse pointing and selection function with the tongue movement  SIX COMMANDS : up, down, right, left, single click & double click  SSP algorithm running in background
  • 16.
    MAGNETIC TRACER : small, cylindrical, rare-earth permanent magnet  PAIR of two-axis magnetic field sensor modules- each contained a pair of orthogonal magneto-inductive sensor  THREE AXIS MODULE: used as a reference electronic compass to minimize the effect of external magnetic field interference
  • 17.
    CONTROL unit & reference compass hidden under face shield cap  ENTIRE system was powered by a 3.3V coin-sized battery(CR2023)
  • 19.
    SENSOR output where send serially to the ultralow- power MSP430 microcontroller  MICROCONTROLLER took 11 samples from each sensor  SAMPLES are arranged in a data frame and wirelessly transmitted to a PC across a 2.4 GHz wireless link established between nRF2401 transceivers
  • 20.
    Sensor Signal Processingalgorithm  DEVELOPED in MATLAB environments  2 phases: Training & Testing •TRAINING : USES principal components analysis (PCA) to extract the most important features of the sensor output waveforms for each specific command • USER repeats each of the six designated commands 10 times in 3-second intervals
  • 21.
    TOTAL of 12 samples (3 per sensor) are recorded in 12-variable vectors • THE PCA-based feature-extraction algorithm calculates the eigenvectors and Eigen values based on the 12-variable vectors • THREE eigenvectors with the largest Eigen values are then chosen to set up the feature matrix [v1, v2, v3]
  • 22.
    BY multiplying the training vectors with the feature matrix, the SSP algorithm forms a cluster (class) of 10 data points from training for each specific command •TESTING: k-nearest neighbour (kNN) classifier is then used in real time to evaluate the proximity of the incoming data points to the clusters formed earlier in the training phase
  • 23.
    kNN starts at the incoming new data point and inflates an imaginary sphere around that data point until it contains a certain number (k) of the nearest training data points • IT associates the new data point to the command that has the majority of the training data points inside that spherical region.
  • 24.
    AFTER finding theintended user command, the mouse pointer starts moving slowly in the selected direction FOR faster access the user can hold his or her tongue in the position of the issued command and the pointer will gradually accelerate until it reaches a certain maximum velocity
  • 25.
    Training Session  GRAPHICAL user interface (GUI) prompted the user to define each command by moving his tongue from its resting position to the corresponding command position when the command light was on and returning it back to the resting position when the light went off
  • 26.
    THIS procedure was repeated 10 times for the entire set of six commands plus the tongue resting position, resulting in a total of 70 training data points.
  • 27.
    Implementation - Controlling a Powered Wheel Chair
  • 30.
    Block diagram &hardware component  SYSTEM consists of the hall effect sensors (A1231), MSP430 microcontroller, H-Bridge driver (SN754410)
  • 31.
    4 ADC channels of MSP430 convert analog signals from sensors to digital for processing  BASED on the processed information microcontroller drives the driver IC  DRIVER IC drives the DC powered wheelchair
  • 32.
    SENSOR- Linear Bipolar Hall effect sensor TRANSDUCER which varies its output voltage in response to changes in magnetic field •VOLTAGE:4.5V-5.5V •OUTPUT is an analog voltage that vary from 0-5V •THAT is converted to a digital value by ADC
  • 33.
    H-BRIDGE DRIVER : used to drive the wheelchair  SN7454410:BidirectionalH-Bridge driver  5V power supply is given  SPEED of the wheelchair is controlled by the PWM signals from MSP430
  • 34.
    •4 SWITCHES •WHEN Aand D are closed, a positive voltage applied across the motor •WHEN B and C are closed , voltage is reversed, allowing reverse operation of the motor
  • 35.
    DC Motors and Gear Box •5V DC motors with a gearbox for driving the vehicle •USED 2 motors: Left and right •LEFT motor at rest and right in motion turn vehicle left and vice versa •FOR forward and backward motion both the motors are driven in same direction with same speed
  • 36.
    Software Components  ADC10(Analog to Digital converter) •MSP430G2252 has 8 ADC channels of 10 bit each •ADC10 convert the analog signal from the sensor into digital value •WE get the value ranging from 0-1023
  • 38.
    Pulse Width Modulation •MODULATING technique which generates variable width pulses is used to vary the speed of the motor
  • 39.
    Algorithm •ROUND ROBIN algorithm that implements polling technique •SENSOR values are always read one after other in a continuous loop •FEW threshold values set for each sensor •IF reading from sensors reaches the threshold value program triggers the change of state
  • 41.
    S1 & S2 front sensor , S3 & S4 back sensors  WHITE circle in the middle- resting position of tongue  DIFFERENT flag values for each sensor  IF sensor is high, then the flag value remains high until the operation is performed, then it is reset again
  • 42.
    f1 - S1 > 750 // forward  f2 - S2 > 600 // backward  f3 - S3 > 700 // speed increase  f4 - S4 > 700 // speed decrease  f5 - S1 > 600 & S3 > 600 // turn left  f6 - S2 > 600 & S4 > 600 // turn right
  • 43.
    Advantages  SIMPLE to implement , low cost, easy to operate flexible  NO surgery needed  OFFERS better privacy to the user
  • 44.
    Drawbacks  USERS should avoid inserting ferromagnetic objects in their mouth  MAGNETIC tracer should be removed if the user is undergoing MRI
  • 46.