Eye controlled HMICHAPTER 1                  INTRODUCTION TO EYE-HMI       Bio-based human computer interface (HCI) has th...
Eye controlled HMI      ELECTRO-OCULOGRAPHY (EOG) PRINCIPLE       Electro-oculography (EOG) is a new technology of placing...
Eye controlled HMIsurrounding iris and usually appears black in normal lighting conditions. Light raysentering through the...
Eye controlled HMIMETHODOLOGY       In our HCI system, four to five electrodes are employed to attain the EOG signals.Figu...
Eye controlled HMI                                     1 & 4 for detecting vertical movement                              ...
Eye controlled HMI• Head Movements are Permissible         The EOG has the advantage that the signal recorded is the actua...
Eye controlled HMI• Lighting Conditions          Variable lighting conditions may make some of the visual systems unsuitab...
Eye controlled HMI• Lighting Conditions       The DC level of the EOG signal varies with lighting conditions over long per...
Eye controlled HMI• Age and Sex        sex have a significant effect on baseline voltage levels, although this should notp...
Eye controlled HMI            BLOCK DIAGRAM AND DESCRIPTION Block Diagram of Eye controller Human Machine interface:      ...
Eye controlled HMIDESCRIPTIONAcquisition PartFor four-five different electrodes two separate acquisition electronics is re...
Eye controlled HMIhigh pass filter to block DC and frequencies upto 0.1-0.3Hz. These filters and gain blocksare implemente...
Eye controlled HMIApplication PartRF Receiver:         Wireless signals transmitted by our acquisition part are received i...
Eye controlled HMI              BIO-POTENTIALS & ELECTRODESBiopotentials        An electric potential that is measured bet...
Eye controlled HMIMechanism behind Bio Potentials    •   Concentration of potassium (K+) ions is 30-50 times higher inside...
Eye controlled HMI       As mentioned already, the function of the nerve cell is to transmit informationthroughout the bod...
Eye controlled HMI        As more Na+ channels open, more Na+ ions enter the cell and the inside of thecell membrane rapid...
Eye controlled HMIof propagation. The speed, or conduction velocity, at which the action potential travelsdown the nerve f...
Eye controlled HMI                              EOG ELECTRODES       Because of the very low amplitude of the EOG, the ele...
Eye controlled HMI       The desirable characteristics above can generally be satisfied by the use ofnonpolarisable electr...
Eye controlled HMI                    BIO-POTENTIAL AMPLIFIERS       Bio signals are recorded as potentials, voltages, and...
Eye controlled HMIbiopotentials, (3) a power line interference signal of 50 Hz and its harmonics, (4)interference signals ...
Eye controlled HMIincomplete rejection of common mode interference signals as a function of CMRR, andan undesired componen...
Eye controlled HMImeasured signal. The preamplifier represents the most critical part of the amplifier itselfsince it sets...
Eye controlled HMIFig :Circuit drawings for three different realizations of instrumentation amplifiers forbiomedical appli...
Eye controlled HMIfollowing amplifier stage can be low, and still the influence of interference signalscoupled into the tr...
Eye controlled HMI                      ACQUISITION FRONT END        Electrodes capture the biopotentials from the body bu...
Eye controlled HMI       In above circuit diagram IA AD620 is set for gain of 50, which is followed by lowpass filter and ...
Eye controlled HMI                     I2C Based serial ADC PCF8591Introduction to I2C Protocol:Name I2C is shorthand for ...
Eye controlled HMII2C is a two-wire serial bus, as shown in Figure 1. Theres no need for chip select orarbitration logic, ...
Eye controlled HMICommunicationAs you can see in Figure 2, the master begins the communication by issuing the startconditi...
Eye controlled HMIAdvantages of I2C   •   Only two bus lines are required to establish full-fledged bus.   •   Each slave ...
Eye controlled HMI   •   Accessing low speed DACs and ADCs.   •   Changing contrast, hue, and color balance settings in mo...
Eye controlled HMIStart and stop conditionsBoth data and clock lines remain HIGH when the bus is not busy. A HIGH-to-LOWtr...
Eye controlled HMIacknowledge on the last byte that has been clocked out of the slave. In this event thetransmitter must l...
Eye controlled HMI                    I2C Based Serial ADC PCF8591:       An analog-to-digital converter (ADC) is a device...
Eye controlled HMI   •   Low standby current   •   Serial input/output via I2C-bus   •   Address by 3 hardware address pin...
Eye controlled HMIBlock Diagram:Pin Description and Pin Diagram:ECE, SBMSIT                        38
Eye controlled HMIFUNCTIONAL DESCRIPTIONAddressing        Each PCF8591 device in an I2C-bus system is activated by sending...
Eye controlled HMI       The second byte sent to a PCF8591 device will be stored in its control register andis required to...
Eye controlled HMISamples picked up from differential inputs are converted to an 8-bit twos complementcode. The conversion...
Eye controlled HMI                MICROCONTROLLER P89V51RX2       Computer in its simplest form needs at least 3 basic blo...
Eye controlled HMI      Three 16-bit timers/counters      Programmable watchdog timer      Eight interrupt sources with...
Eye controlled HMIPin Diagram:Pin description of P89CV51RXXPort 0: Port 0 is an 8-bit open drain bidirectional I/O port. P...
Eye controlled HMIinputs in this state. As inputs, Port 1 pins that are externally pulled LOW will sourcecurrent (IIL) bec...
Eye controlled HMIAddress Latch Enable: ALE is the output signal for latching the low byte of theaddress during an access ...
Eye controlled HMICHAPTER                                RF INTERFACING       For transmission of data/Cmds from one point...
Eye controlled HMIApplications•   Car security system                      •   On-Site Paging•   Sensor reporting         ...
Eye controlled HMIFeatures•   Low Cost•   5V operation•   3.5mA current drain•   No External Parts are required•   Receive...
Eye controlled HMI       Note: 3pin RF Rx module is having helical wire antenna           Transmitter Encoder HT12E (212 s...
Eye controlled HMI•   Low standby current: 0.1_A (typ.) at VDD=5V•   HT12A with a 38kHz carrier for infrared transmission ...
Eye controlled HMIBlock diagram and Pin Diagram:ECE, SBMSIT                      52
Eye controlled HMIECE, SBMSIT          53
Eye controlled HMIECE, SBMSIT          54
Eye controlled HMIFlow chart and application circuit of HT12E working:ECE, SBMSIT                                         ...
Eye controlled HMIReceiver Decoder HT12D (212 series)       The 212 decoders are a series of CMOS LSIs for remote control ...
Eye controlled HMIApplications•   Burglar alarm system•   Smoke and fire alarm system•   Garage door controllers•   Car do...
Eye controlled HMIECE, SBMSIT          58
Eye controlled HMIOperation       The 212 series of decoders provides various combinations of addresses and datapins in di...
Eye controlled HMIFlow Chart and Application Circuit       The oscillator is disabled in the standby state and activated w...
Eye controlled HMIFlow Chart:Transmitter Module Interfacing       This is complete data transmission module, Microcontroll...
Eye controlled HMIcontains module address and sync bits, this signal is fed to RF Tx module which doesOOK modulation and t...
Eye controlled HMIACQUISITION AND PROCESSING SYSTEM       Acquisition front end system will interface to the body get the ...
Eye controlled HMIProcessing of data to decode the eye movements:       Basically we get digital data from ADC for each ch...
Eye controlled HMIWe have looked in straight direction and signals are stabilized, now its turn to givecommands using eye ...
Eye controlled HMI              Figure: Flow chart for up-down and blink detections.       These commands are sent via RF ...
Eye controlled HMI                         Graphics LCD JHD12864        JHD12864J is a light weight, low power consumption...
Eye controlled HMI    DB3     H/L     Data bit 3    DB4     H/L     Data bit 4    DB5     H/L     Data bit 5    DB6     H/...
Eye controlled HMIBlock Diagram of GLCD:ECE, SBMSIT              69
Eye controlled HMIFollowing are display control Instruction which MCU has to give whileinterfacing to GLCD module:ECE, SBM...
Eye controlled HMIBasic interfacing diagram with MCU:ECE, SBMSIT                           71
Eye controlled HMIAlgorithm and flowchart for GLCD interfacing1.Send the display off command 3eh2. Send the display on com...
Eye controlled HMI                                  Application Part          Application part consists of Graphics LCD, P...
Eye controlled HMICircuit diagram and flowcharts are given in following pagesFlow chart for ApplicationECE, SBMSIT        ...
Eye controlled HMIECE, SBMSIT          75
Eye controlled HMICircuit for Application part:ECE, SBMSIT                     76
Eye controlled HMI               Software and hardware requirementsSoftware and Hardware requirements along with component...
Eye controlled HMI                         Applications and Future work                                         Applicatio...
Eye controlled HMI    •   US army is doing research on eye movement based Air craft and tank controls    •   Hands free mo...
Eye controlled HMI                                    conclusion            EXPECTED OUTPUT AND CONCLUSIONIntermediate Out...
Eye controlled HMI                                  REFERENCES   •   http://en.wikipedia.org/wiki/Eye_tracking   •   The 8...
Upcoming SlideShare
Loading in …5
×

Modified report

1,981 views

Published on

Published in: Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,981
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
96
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Modified report

  1. 1. Eye controlled HMICHAPTER 1 INTRODUCTION TO EYE-HMI Bio-based human computer interface (HCI) has the potential to enable severelydisabled people to drive computers directly by bioelectricity rather than by physicalmeans. A study on the group of persons with severe disabilities shows that many of themhave the ability to control their eye movements, which could be used to develop newhuman computer interface systems to help them communicate with other persons orcontrol some special instruments. Furthermore, this application of EOG-based HCI could be extended to the groupof normal persons for game or other entertainments. Nowadays, some methods whichattain user’s eye movements are developed. In this project our objective is to design a Human Machine interface, which can becontrolled using EOG Signals and final output is to be used to move cursor on theGraphic Display which has several buttons and each button on clicking by blinking ofeyes activated corresponding appliance or action. We will provide RF interface betweenacquisition/processing part and application so that it’s easy to handle and easy to install inhomes and hospitals.ECE, SBMSIT 1
  2. 2. Eye controlled HMI ELECTRO-OCULOGRAPHY (EOG) PRINCIPLE Electro-oculography (EOG) is a new technology of placing electrodes on user’sforehead around the eyes to record eye movements. EOG is a very small electricalpotential that can be detected using electrodes. Compared with the EEG, EOG signalshave the characteristics as follows: the amplitude is relatively high (15-200uV), therelationship between EOG and eye movements is linear, and the waveform is easy todetect. Considering the characteristics of EOG mentioned above, EOG based HCI isbecoming the hotspot of bio-based HCI research in recent years. Basically EOG is a bio-electrical skin potential measured around the eyes but firstwe have to understand eye itself:Anatomy of the Eye The main features visible at the front of the eye are shown in Figure .The lens,directly behind the pupil, focuses light coming in through the opening in the centre of theeye, the pupil, onto the light sensitive tissue at the back of the eye, the retina. The iris isthe coloured part of the eye and it controls the amount of light that can enter the eye bychanging the size of the pupil, contracting the pupil in bright light and expanding thepupil in darker conditions. The pupil has very different reflectance properties than theECE, SBMSIT 2
  3. 3. Eye controlled HMIsurrounding iris and usually appears black in normal lighting conditions. Light raysentering through the pupil first pass through the cornea, the clear tissue covering the frontof the eye. The cornea and vitreous fluid in the eye bend and refract this light. Theconjuctiva is a membrane that lines the eyelids and covers the sclera, the white part of theeye. The boundary between the iris and the sclera is known as the limbus, and is oftenused in eye tracking. The light rays falling on the retina cause chemical changes in the photosensitivecells of the retina. These cells convert the light rays to electrical impulses which aretransmitted to the brain via the optic nerve. There are two types of photosensitive cells inthe retina, cones and rods. The rods are extremely sensitive to light allowing the eye torespond to light in dimly lit environments. They do not distinguish between colours,however, and have low visual acuity, or attention to detail. The cones are much lessresponsive to light but have a much higher visual acuity. Different cones respond todifferent wavelengths of light, enabling colour vision. The fovea is an area of the retina ofparticular importance. It is a dip in the retina directly opposite the lens and is denselypacked with cone cells, allowing humans to see fine detail, such as small print. Thehuman eye is capable of moving in a number of different manners to observe, read orexamine the world in front of them.The Electrooculogram The electrooculogram (EOG) is the electrical signal produced by the potentialdifference between the retina and the cornea of the eye. This difference is due to the largepresence of electrically active nerves in the retina compared to the front of the eye. Manyexperiments show that the corneal part is a positive pole and the retina part is a negativepole in the eyeball. Eye movement will respectively generates voltage up to 16uV and14uV per 1° in horizontal and vertical way. The typical EOG waveforms generated byeye movements are shown in Figure 1. In Figure 1, positive or negative pulses will be generated when the eyes rollingupward or downward. The amplitude of pulse will be increased with the increment ofrolling angle, and the width of the positive (negative) pulse is proportional to the durationof the eyeball rolling process.ECE, SBMSIT 3
  4. 4. Eye controlled HMIMETHODOLOGY In our HCI system, four to five electrodes are employed to attain the EOG signals.Figure 2 shows the electrode placement.ECE, SBMSIT 4
  5. 5. Eye controlled HMI 1 & 4 for detecting vertical movement 2 & 3 for detecting horizontal movement 5 is for reference(can be omitted or place at forehead). Blink detection is by separate algorithm based on EOG signalsFigure 2: electrode placementsAdvantages of the EOG over other methods The visual systems mentioned in our projet offer robust methods of eye tracking,usually with very good accuracy. While in certain circumstances, visual methods may bemore appropriate, the electrooculogram offers a number of advantages. Some of thereasons for favouring the EOG over other options• Range The EOG typically has a larger range than visual methods which are constrainedfor large vertical rotations where the cornea and iris tend to disappear behind the eyelid.Angular deviations of up to 80◦ can be recorded along both the horizontal and verticalplanes of rotation using electrooculography.• Linearity The reflective properties of ocular structures used to calculate eye position invisual methods are linear only for a restricted range, compared to the EOG where thevoltage difference is essentially linearly related to the angle of gaze for ±30◦ and to thesine of the angle for ±30◦ to ±60◦ECE, SBMSIT 5
  6. 6. Eye controlled HMI• Head Movements are Permissible The EOG has the advantage that the signal recorded is the actual eyeball positionwith respect to the head. Thus for systems designed to measure relative eyeball positionto control switches (e.g. looking up, down, left and right could translate to four separateswitch presses) head movements will not hinder accurate recording.• Non-invasive Unlike techniques such as the magnetic search coil technique, EOG recordings donot require anything to be fixed to the eye which might cause discomfort or interfere withnormal vision. EOG recording only requires three electrodes (for one channel recording),or five electrodes (for two channel recording), which are affixed externally to the skin.• Obstacles in front of the eye In visual methods, measurements may be interfered with by scratches on thecornea or by contact lenses. Bifocal glasses and hard contact lenses seem to causeparticular problems for these systems. EOG measurements are not affected by theseobstacles.• Cost EOG based recordings are typically cheaper than visual methods, as they can bemade with some relatively inexpensive electrodes, some form of data acquisition card andappropriate software.ECE, SBMSIT 6
  7. 7. Eye controlled HMI• Lighting Conditions Variable lighting conditions may make some of the visual systems unsuitable or atleast require re-calibration when the user moves between different environments. Onesuch scenario which could pose problems is where the eye tracking system is attached toa user.• Eye Closure is Permissible The EOG is commonly used to record eye movement patterns when the eye isclosed, for example during sleep. Visual methods require the eye to remain open to knowwhere the eye is positioned relative to the head, whereas an attenuated version of theEOG signal is still present when the eye is closed.• Real-Time The EOG can be used in real-time as the EOG signal responds instantaneously toa change in eye position and the eye position can be quickly inferred from the change.The EOG is linear up to 30◦.Limitations of EOG-Based Eye tracking The measured EOG voltage varies for two reasons. Either the eye moves (whichwe want to record), or baseline drift occurs (which we want to ignore). Baseline driftoccurs due to the following factors:ECE, SBMSIT 7
  8. 8. Eye controlled HMI• Lighting Conditions The DC level of the EOG signal varies with lighting conditions over long periodsof time. When the source of the light entering the eye changes from dark conditions toroom lighting, as per studies it can take anywhere from between 29-52 minutes for themeasured potential to stabilise to within 10% of the baseline, and anywhere between17-51 minutes when the transition is from room lighting to darkness.• Electrode Contact The baseline may vary due to the spontaneous movement of ions between the skinand the electrode used to pick up the EOG voltage. The mostly commonly used electrodetype is silver-silver chloride (Ag-AgCl). Large DC potentials of up to 50mV can developacross a pair of Ag-AgCl electrodes in the absence of any bioelectric event, due todifferences in the properties of the two electrode surfaces with respect to the electrolyticconduction gel. The extent of the ion movement is related to a number of variablesincluding the state of the electrode gel used, variables in the subject’s skin and thestrength of the contact between the skin and the electrode. Proper preparation of the skinis necessary to maximize conduction between the skin and the conduction gel, usually bybrushing the skin with alcohol to remove facial oils.• Artifacts due to EMG or Changes in Skin Potential The baseline signal may change due to interference from other bioelectricalsignals in the body, such as the electromyogram (EMG) or the skin potential. EMGactivity arises from movement of the muscles close to the eyes, for example if the subjectfrowns or speaks. These signals may be effectively rejected by careful positioning of theelectrodes and through low pass filtering the signal. Skin potential changes due tosweating oremotional anxiety pose a more serious problem.ECE, SBMSIT 8
  9. 9. Eye controlled HMI• Age and Sex sex have a significant effect on baseline voltage levels, although this should notpose a problem if a system is calibrated initially for each particular user.• Diurnal Variations The baseline potential possibly varies throughout the day. Manual calibration isoften used to compensate for DC drift - the subject shifts his gaze between points ofknown visual angle and the amplifier is balanced until one achieves the desiredrelationship between voltage output and degree of eye rotation. With frequent re-calibration, accuracies of up to ±30′ can be obtained. While manual calibration may beacceptable practice in clinical tests that use the EOG, this restriction hinders the EOGfrom being used independently as a control and communication tool by people withdisabilities.ECE, SBMSIT 9
  10. 10. Eye controlled HMI BLOCK DIAGRAM AND DESCRIPTION Block Diagram of Eye controller Human Machine interface: Embedded Controller Electrodes Instrumentation For data near Eye to Amplifier A/D acquisition sense signal And active Convertor using I2C ADC from eyes filter EOG signals Acquisition and Processing Part Processing for Eye Movement and Eye Blink detections RF Interface RF TX Module Graphics LCD (MENU) Display Devices to be Screen and Eye RF Interface controlled by Device Movements RF RXeye movements controller output Module Application Part ECE, SBMSIT 10
  11. 11. Eye controlled HMIDESCRIPTIONAcquisition PartFor four-five different electrodes two separate acquisition electronics is requiredElectrodes: First thing to interface to body is Electrodes here we are using reusable electrodesto connect electronics with human body and these electrodes will pickup signals whichcorresponds to eye movements signals mixed with some others signals which are noisefor us.We are going to use Ag-AgCl electrodes as they are low cost and easily available.Instrumentation Amplifier: Signals from electrodes are received and sent to Instrumentation Amplifier.An instrumentation amplifier is a type of differential amplifier that has been outfitted withinput buffers, which eliminate the need for input impedance matching and thus make theamplifier particularly suitable for use in measurement and test equipment. Additionalcharacteristics include very low DC offset, low drift, low noise, very high open-loop gain,very high common-mode rejection ratio, and very high input impedances. Instrumentationamplifiers are used where great accuracy and stability of the circuit both short- and long-term are required.We are using AD620 which is precision Instrumentation amplifier.Active Filters and Gain Blocks: Opamp based Active filters are used we have low pass filter so that only eyesignals are going future in the circuit, cutoff frequency for this filter is 20Hz-40Hz. AndECE, SBMSIT 11
  12. 12. Eye controlled HMIhigh pass filter to block DC and frequencies upto 0.1-0.3Hz. These filters and gain blocksare implemented using LM324 Opamp.Analog to Digital Convertor: Final amplified and filtered analog output is converted into Digital signal usingI2C Based 4 channel A2D convertor-PCF8591 to save space as ADC0808 is little biggerin size.Acquisition and processing microcontroller: This is 8051 class of microcontroller and it has to acquire signals from A/Dconvertor for both chains up-down electrode chain and left-right electrode chain. As ourmicrocontroller is fast and powerful we will process the signal here itself and transmitfinal eye move outputs to application part wirelessly.Cmds sent:01– CL: Right eye movement02– CR: Left eye movement03– CU: Up eye movement04– CD: Down eye movement05– BL: Blinking of eyeRF Transmitter: Here we can use 315/433Mhz Tx modules along with HT640 Encoder to send eyemovement commands to the application part.ECE, SBMSIT 12
  13. 13. Eye controlled HMIApplication PartRF Receiver: Wireless signals transmitted by our acquisition part are received in this section,here we use 315/433Mhz Rx modules along with HT648 decoder. Output of RF receivergoes to application part directly.Display and appliance controller: This is a again a microcontroller which receives eye movements signals (R L U DB) as described above via UART interface. We are using P89V51RD2 from NXP(Philips), this microcontroller is connected to Graphic LCD which is displaying Cursorand 4 buttons1. TV2. FAN3. Lights4. Alarm Using eye movements a cursor is controlled and using blink click operation isdone, each button is toggle button i.e. if appliance is on it will become off and vice versa.But alarm button is different when clicked a On-off alarm is generated to call assistance.And assistant has to come and reset the alarm. Now this controller is also connected to relay board so button action is convertedinto relays getting switch off and on. And hence appliances are getting turned on and off.Many other applications are also possible like Computer Mouse interface, virtualkeyboard interface so disable can talk via this keyboard and send mails.ECE, SBMSIT 13
  14. 14. Eye controlled HMI BIO-POTENTIALS & ELECTRODESBiopotentials An electric potential that is measured between points in living cells, tissues, andorganisms, and which accompanies all biochemical processes. Also describes the transferof information between and within cellsECE, SBMSIT 14
  15. 15. Eye controlled HMIMechanism behind Bio Potentials • Concentration of potassium (K+) ions is 30-50 times higher inside as compared to outside • Sodium ion (Na+) concentration is 10 times higher outside the membrane than inside • In resting state the member is permeable only for potassium ions  Potassium flows outwards leaving an equal number of negative ions inside  Electrostatic attraction pulls potassium and chloride ions close to the membrane  Electric field directed inward forms  Electrostatic force vs. diffusional forceDifferent Types of potentials are discussed hereThe Membrane Potential A potential difference usually exists between the inside and outside of any cellmembrane, including the neuron. The membrane potential of a cell usually refers to thepotential of the inside of the cell relative to the outside of the cell i.e. the extracellularfluid surrounding the cell is taken to be at zero potential. When no external triggers areacting on a cell, the cell is described as being in its resting state. A human nerve orskeletal muscle cell has a resting potential of between -55mV and -100mV . Thispotential difference arises from a difference in concentration of the ions K+ and Na+inside and outside the cell. The selectively permeable cell membrane allows K+ ions topass through but blocks Na+ ions. A mechanism known as the ATPase pump pumps onlytwo K+ ions into the cell for every three Na+ cells pumped out of the cell resulting in theoutside of the cell being more positive than the inside. The origin of the resting potentialis explained in further detail in.The Action PotentialECE, SBMSIT 15
  16. 16. Eye controlled HMI As mentioned already, the function of the nerve cell is to transmit informationthroughout the body. A neuron is an excitable cell which may be activated by a stimulus.The neuron’s dendrites are its stimulus receptors. If the stimulus is sufficient to cause thecell membrane to be depolarised beyond the gate threshold potential, then an electricaldischarge of the cell will be triggered. This produces an electrical pulse called the actionpotential or nerve impulse. The action potential is a sequence of depolarisation andrepolarisation of the cell membrane generated by a Na+ current into the cell followed by aK+ current out of the cell. The stages of an action potential are shown in FigureFigure 3.4: An Action Potential. This graph shows the change in membrane potentialas a function of time when an action potential is elicited by a stimulus.• Stage 1 – Activation When the dendrites receive an “activation stimulus” the Na+ channels begin toopen and the Na+ concentration inside the cell increases, making the inside of the cellmore positive. Once the membrane potential is raised past a threshold (typically around-50mV), an action potential occurs.• Stage 2 – DepolarisationECE, SBMSIT 16
  17. 17. Eye controlled HMI As more Na+ channels open, more Na+ ions enter the cell and the inside of thecell membrane rapidly loses its negative charge. This stage is also known as the risingphase of the action potential. It typically lasts 0.2 - 0.5ms.• Stage 3 – Overshoot The inside of the cell eventually becomes positve relative to the outside of thecell. The positive portion of the action potential is known as the overshoot.• Stage 4 – Repolarisation The Na+ channels close and the K+ channels open. The cell membrane begins torepolarise towards the resting potential.• Stage 5 – Hyperpolarisation The membrane potential may temporarily become even more negative than theresting potential. This is to prevent the neuron from responding to another stimulusduring this time, or at least to raise the threshold for any new stimulus.• Stage 6 The membrane returns to its resting potential.Propagation of the Action Potential An action potential in a cell membrane is triggered by an initial stimulus to theneuron. That action potential provides the stimulus for a neighbouring segment of cellmembrane and so on until the neuron’s axon is reached. The action potential thenpropagates down the axon, or nerve fibre, by successive stimulation of sections of theaxon membrane. Because an action potential is an all-or-nothing reaction, once the gatethreshold is reached, the amplitude of the action potential will be constant along the pathECE, SBMSIT 17
  18. 18. Eye controlled HMIof propagation. The speed, or conduction velocity, at which the action potential travelsdown the nerve fibre depends on a number of factors, including the initial restingpotential of the cell, the nerve fibre diameter and also whether or not the nerve fibre ismyelinated. Myelinated nerve fibres have a faster conduction velocity as the actionpotential jumps between the nodes of Ranvier.Synaptic Transmission The action potential propagates along the axon until it reaches the axonal ending.From there, the action potential is transmitted to another cell, which may be another nervecell, a glandular cell or a muscle cell. The junction of the axonal ending with another cellis called a synapse. The action potential is usually transmitted to the next cell through achemical process at the synapse.Resting potential: Nerve and muscle cells are encased in a semi-permeable membrane that permitsselected substances to pass through while others are kept out. Body fluids surroundingcells are conductive solutions containing charged atoms known as ions. In their restingstate, membranes of excitable cells readily permit the entry of K+ and Cl- ions, buteffectively block the entry of Nu+ ions (the permeability for K+ is 50-100 times that forNa+). Various ions seek to establish a balance between the inside and the outside of a cellaccording to charge and concentration. The inability of Nu+ to penetrate a cell membraneresults in the polarization that is called as Resting Potential.ECE, SBMSIT 18
  19. 19. Eye controlled HMI EOG ELECTRODES Because of the very low amplitude of the EOG, the electrodes represent theweakest link in the entire recording system. The following properties are desirable in anEOG electrode:(a) Stable electrode potential: Spontaneous fluctuations of only 2 or 3mV in the potentialdifference between an electrode and the surrounding electrolyte will produce artifactsvery much larger than the EOG.(b) Equal electrode potentials: A small standing potential difference between a pair ofelectrodes will not present major difficulties, apart from producing a temporary deflectionof the trace and possibly blocking of the amplifiers when the electrodes are firstconnected to the recorder. However, if the current flow between the electrode variesowing to changing contact resistances, artifact may result, As it is in practice neverpossible to ensure that conventional electrodes are of equal potential, it follows that athird desirable characteristic is constant electrode contact resistances(c) Equal electrode resistances: EOG recording is bedeviled by electrical interference -particularly from ac mains; there are generally unwanted changes in potential differencebetween the subject and the ECG machine that are seen as common mode signals and canhe rejected by the use of differential amplifiers. Unequal electrode resistances, however,unbalance the system and produce an out-of-phase component that will appear in thetracing.(d) Low electrode resistance: With modern amplifier design, it is now easy to ensure thatthe electrode resistances are very much less than the input impedance so that as much aspossible of the ECG signal is applied at the input of the amplifier. The effects of unequalelectrode resistances are less marked when the actual values are low. In general when theother criteria above are satisfied, the electrode resistance is to be less than 5kΩ andmeasurement of resistance provides a good check on the quality of electrode preparationand application.ECE, SBMSIT 19
  20. 20. Eye controlled HMI The desirable characteristics above can generally be satisfied by the use ofnonpolarisable electrodes, so far as identical physical and chemical structure, securelyattached to skin that has first been cleaned and abraded to remove the outer layer which isof high resistance.TYPES OF ELECTRODES Ag-AgCl electrodes and disposable electrodes were used when the data wasrecorded from the frontal region. These electrode types that are used were shown in fig. Figure: Different types of electrodesECE, SBMSIT 20
  21. 21. Eye controlled HMI BIO-POTENTIAL AMPLIFIERS Bio signals are recorded as potentials, voltages, and electrical field strengthsgenerated by nerves and muscles. The measurements involve voltages at very low levels,typically ranging between 1 μV and 100 mV, with high source impedances andsuperimposed high level interference signals and noise. The signals need to be amplifiedto make them compatible with devices such as displays, recorders, or A/D converters forcomputerized equipment. Amplifiers adequate to measure these signals have to satisfyvery specific requirements. They have to provide amplification selective to thephysiological signal, reject superimposed noise and interference signals, and guaranteeprotection from damages through voltage and current surges for both patient andelectronic equipment. Amplifiers featuring these specifications are known as biopotentialamplifiers. Basic requirements and features, as well as some specialized systems,The basic requirements that a biopotential amplifier has to satisfy are:  the physiological process to be monitored should not be influenced in any way by the amplifier  the measured signal should not be distorted  the amplifier should provide the best possible separation of signal and interferences  the amplifier has to offer protection of the patient from any hazard of electrical shock  the amplifier itself has to be protected against damages that might result from high input voltages as they occur during the application of defibrillators or electrosurgical instrumentation A typical configuration for the measurement of bio potentials is shown in figure.Three electrodes, two of them are picking up the biological signal and the third providingthe reference potential, connect the subject to the amplifier. The input signal to theamplifier consists of five components: (1) the desired biopotential, (2) undesiredECE, SBMSIT 21
  22. 22. Eye controlled HMIbiopotentials, (3) a power line interference signal of 50 Hz and its harmonics, (4)interference signals generated by the tissue/electrode interface, and (5) noise. Properdesign of the amplifier provides rejection of a large portion of the signal interferences.The main task of the differential amplifier as shown in Figure is to reject the linefrequency interference that is electrostatically or magnetically coupled into the subject.The desired biopotential appears as a voltage between the two input terminals of thedifferential amplifier and is referred to as the differential signal. . The line frequencyinterference signal shows only very small differences in amplitude and phase between thetwo measuring electrodes, causing approximately the same potential at both inputs, andthus appears only between the inputs and ground and is called the common mode signal.Strong rejection of the common mode signal is one of the most important characteristicsof a good biopotential amplifier.Fig : Typical configuration for the measurement of biopotentials. The biological signal Vappears between the two measuring electrodes at the right and left arm of the patient, andis fed to the inverting and the non-inverting inputs of the differential amplifier. The rightleg electrode provides the reference potential for the amplifier with a common modevoltage Vc as indiacted.common mode rejection ratio CMRR of an amplifier is defined as the ratio of the differential mode gain over thecommon mode gain. The output of a real biopotential amplifier will always consist of thedesired output component due to a differential biosignal, an undesired component due toECE, SBMSIT 22
  23. 23. Eye controlled HMIincomplete rejection of common mode interference signals as a function of CMRR, andan undesired component due to source impedance unbalance allowing a small proportionof a common mode signal to appear as a differential signal to the amplifier. Since sourceimpedance unbalances of 5,000 to 10,000 Ω, mainly caused by electrodes, are notuncommon, and sufficient rejection of line frequency interferences requires a minimumCMRR of 100 dB, the input impedance of the amplifier should be at least 109 Ω at 60 Hzto prevent source impedance unbalances from deteriorating the overall CMRR of theamplifier. State-of-the-art biopotential amplifiers provide a CMRR of 120 to 140 dB.In order to provide optimum signal quality and adequate voltage level for further signalprocessing, the amplifier has to provide a gain of 100 to 50,000 and needs to maintain thebest possible signal-to noise ratio. The presence of high level interference signals not onlydeteriorates the quality of the physiological signals, but also restricts the design of thebiopotential amplifier. In order to prevent the amplifier from going into saturation, thiscomponent has to be eliminated before the required gain can be provided for thephysiological signal.Fig :Schematic design of the main stages of a biopotential amplifier. Three electrodesconnect the patientA typical design of the various stages of a biopotential amplifier is shown in above figure.The electrodes which provide the transition between the ionic flow of currents inbiological tissue and the electronic flow of current in the amplifier, represent a complexelectrochemical system. The electrodes determine to a large extent the composition of theECE, SBMSIT 23
  24. 24. Eye controlled HMImeasured signal. The preamplifier represents the most critical part of the amplifier itselfsince it sets the stage for the quality of the biosignal. With proper design, the preamplifiercan eliminate, or at least minimize, most of the signals interfering with the measurementof biopotentials.Instrumentation Amplifier An important stage of all biopotential amplifiers is the input preamplifier whichsubstantially contributes to the overall quality of the system. The main tasks of thepreamplifier are to sense the voltage between two measuring electrodes while rejectingthe common mode signal, and minimizing the effect of electrode polarization overpotentials. Crucial to the performance of the preamplifier is the input impedance whichshould be as high as possible. Such a differential amplifier cannot be realized using astandard single operational amplifier (op-amp) design since this does not provide thenecessary high input impedance. The general solution to the problem involves voltagefollowers, or noninverting amplifiers, to attain highinput impedances. A possible realization is shown in figure(a). The main disadvantage ofthis circuit is that it requires high CMRR both in the followers and in the final op-amp.With the input buffers working at unity gain, all the common-mode rejection must beaccomplished in the output amplifier, requiring very precise resistor matching.Additionally, the noise of the final op-amp is added at a low signal level, decreasing thesignal-to-noise ratio unnecessarily. The circuit in Fig(b) eliminates this disadvantage. Itrepresents the standard instrumentation amplifier configuration. The two input op-ampsprovide high differential gain and unity common-mode gain without the requirement ofclose resistor matching.ECE, SBMSIT 24
  25. 25. Eye controlled HMIFig :Circuit drawings for three different realizations of instrumentation amplifiers forbiomedical applications. Voltage follower input stage (a), improved, amplifying inputstage (b) 2 op-amp version (c). R2 R4 G1 = 1 + 2 G2 = − R1 R3 The preamplifier, often implemented as a separate device which is placed close tothe electrodes or even directly attached to the electrodes, also acts as an impedanceconverter which allows the transmission of even weak signals to the remote monitoringunit. Due to the low output impedance of the preamplifier, the input impedance of theECE, SBMSIT 25
  26. 26. Eye controlled HMIfollowing amplifier stage can be low, and still the influence of interference signalscoupled into the transmission lines is reduced.We are using AD620 its specification and details can be found in datasheet.ECE, SBMSIT 26
  27. 27. Eye controlled HMI ACQUISITION FRONT END Electrodes capture the biopotentials from the body but these signals are veryweak and very noisy so there is invariable need of advance acquisition system whichcomprises of precision instrumentation amplifier, active filters, multiple gain block andfor interfacing to ADC we have to do dc shifting(or clamping) of signal followed byclipping to avoid any residual negative voltages.Circuit is given below: Figure: Acquisition circuit diagram (same is for up-down and left-right).ECE, SBMSIT 27
  28. 28. Eye controlled HMI In above circuit diagram IA AD620 is set for gain of 50, which is followed by lowpass filter and high pass filter together the made band pass filter and we are gettingfrequencies 0.5 to 4Hz at output, this is amplified buy gain block 1 which has variablegain now our signal range is few mv, we need one more gain block followed by activelow pass filter to reject all high frequency noises above 3Hz and some gain also can beprovided if required at this stage. After this we have dc level shifter and clipper ckt toensure only positive voltages are going to ADC. Similar circuit is there for up-down but only we have to adjust the again usingvariable resistor and dc level shift voltage. Both the final outputs are fed to I2C based 4channel ADC PCF8591, left right signal is fed to ch0 and up-down signal is fed to ch1.ECE, SBMSIT 28
  29. 29. Eye controlled HMI I2C Based serial ADC PCF8591Introduction to I2C Protocol:Name I2C is shorthand for a standard Inter-IC (integrated circuit) bus. Philips originally developed I2C for communication between devices inside of aTV set. Examples of simple I2C-compatible devices found in embedded systems includeEEPROMs, thermal sensors, and real-time clocks. The main objective behind the invention of I2C bus is to establish a simple lowpin count bus that can connect different ICs on a circuit board of Television or Radio.Later I2C grew beyond the limits of TV and Radio and now it can be found in almostevery computer motherboards and other embedded devices. I2C can also be used forcommunication between multiple circuit boards in equipments with or without using ashielded cable depending on the distance and speed of data transfer. Standard I2C devices operate up to 100Kbps, while fast-mode devices operate atup to 400Kbps. A 1998 revision of the I2C specification (v. 2.0) added a high-speed moderunning at up to 3.4Mbps. Most of the I2C devices available today support 400Kbpsoperation. Higher-speed operation may allow I2C to keep up with the rising demand forbandwidth in multimedia and other applications. I2C is appropriate for interfacing to devices on a single board, and can be stretchedacross multiple boards inside a closed system, but not much further. An example is a hostCPU on a main embedded board using I2C to communicate with user interface deviceslocated on a separate front panel board. A second example is SDRAM DIMMs, whichcan feature an I2C EEPROM containing parameters needed to correctly configure amemory controller for that module.ECE, SBMSIT 29
  30. 30. Eye controlled HMII2C is a two-wire serial bus, as shown in Figure 1. Theres no need for chip select orarbitration logic, making it cheap and simple to implement in hardware. The two I2C signals are serial data (SDA) and serial clock (SCL). Together, thesesignals make it possible to support serial transmission of 8-bit bytes of data-7-bit deviceaddresses plus control bits-over the two-wire serial bus. The device that initiates atransaction on the I2C bus is termed the master. The master normally controls the clocksignal. A device being addressed by the master is called a slave. In a bind, an I2C slave can hold off the master in the middle of a transaction usingwhats called clock stretching (the slave keeps SCL pulled low until its ready tocontinue). Most I2C slave devices dont use this feature, but every master should supportit. The I2C protocol supports multiple masters, but most system designs include onlyone. There may be one or more slaves on the bus. Both masters and slaves can receiveand transmit data bytes. Each I2C-compatible hardware slave device comes with a predefined deviceaddress, the lower bits of which may be configurable at the board level. The mastertransmits the device address of the intended slave at the beginning of every transaction.Each slave is responsible for monitoring the bus and responding only to its own address.This addressing scheme limits the number of identical slave devices that can exist on anI2C bus without contention, with the limit set by the number of user-configurable addressbits (typically two bits, allowing up to four identical devices).ECE, SBMSIT 30
  31. 31. Eye controlled HMICommunicationAs you can see in Figure 2, the master begins the communication by issuing the startcondition (S). The master continues by sending a unique 7-bit slave device address, withthe most significant bit (MSB) first. The eighth bit after the start, read/not-write (),specifies whether the slave is now to receive (0) or to transmit (1). This is followed by anACK bit issued by the receiver, acknowledging receipt of the previous byte. Then thetransmitter (slave or master, as indicated by the bit) transmits a byte of data starting withthe MSB. At the end of the byte, the receiver (whether master or slave) issues a new ACKbit. This 9-bit pattern is repeated if more bytes need to be transmitted. In a write transaction (slave receiving), when the master is done transmitting all ofthe data bytes it wants to send, it monitors the last ACK and then issues the stop condition(P). In a read transaction (slave transmitting), the master does not acknowledge the finalbyte it receives. This tells the slave that its transmission is done. The master then issuesthe stop condition. I2C offers good support for communication with on-board devices that areaccessed on an occasional basis. I2Cs competitive advantage over other low-speed short-distance communication schemes is that its cost and complexity dont scale up with thenumber of devices on the bus. On the other hand, the complexity of the supporting I2Csoftware components can be significantly higher than that of several competing schemes(SPI and MicroWire, to name two) in a very simple configuration. With its built-inaddressing scheme and straightforward means to transfer strings of bytes, I2C is anelegant, minimalist solution for modest, "inside the box" communication needs.ECE, SBMSIT 31
  32. 32. Eye controlled HMIAdvantages of I2C • Only two bus lines are required to establish full-fledged bus. • Each slave device connected is uniquely addressable using slave addresses • Can choose a short 7 bit addressing or 10 bit addressing (which can accommodate large number of devices on the same bus, but less popular). • No strict baud rate specified since the clock is driven directly by the master. Supports up to 3.4 Mbits/sec transfer speeds. • True multimaster support with up to 8 masters in a single bus system. • Very simple protocol which can be emulated by microcontrollers without integrated I2C peripheral device. And its InexpensiveLimitations of I2C • 7 bit addressing supports only a very small number of devices. • Different devices from different manufacturers come with hard coded slave address or address will be configurable in a small range only. This can lead to address clashes sometimes. • No automatic bus configuration or plug and playApplications of I2C • I²C is appropriate for peripherals where simplicity and low manufacturing cost are more important than speed. Common applications of the I²C bus are: • Reading configuration data from SPD EEPROMs on SDRAM, DDR SDRAM, DDR2 SDRAM memory sticks (DIMM) and other stacked PC boards • Supporting systems management for PCI cards, through an SMBus 2.0 connection. • Accessing NVRAM chips that keep user settings.ECE, SBMSIT 32
  33. 33. Eye controlled HMI • Accessing low speed DACs and ADCs. • Changing contrast, hue, and color balance settings in monitors (Display Data Channel). • Changing sound volume in intelligent speakers. • Controlling OLED/LCD displays, like in a cellphone. • Reading hardware monitors and diagnostic sensors, like a CPU thermostat and fan speed. • Reading real time clocks. • Turning on and turning off the power supply of system components. • A particular strength of I²C is that a microcontroller can control a network of device chips with just two general-purpose I/O pins and software. • Peripherals can also be added to or removed from the I²C bus while the system is running, which makes it ideal for applications that require hot swapping of components.Characteristics Of The I2C-Bus The I2C-bus is for bidirectional, two-line communication between different ICs ormodules. The two lines are a serial data line (SDA) and a serial clock line (SCL). Bothlines must be connected to a positive supply via a pull-up resistor. Data transfer may beinitiated only when the bus is not busy.Bit transfer One data bit is transferred during each clock pulse. The data on the SDA line mustremain stable during the HIGH period of the clock pulse as changes in the data line at thistime will be interpreted as a control signal.ECE, SBMSIT 33
  34. 34. Eye controlled HMIStart and stop conditionsBoth data and clock lines remain HIGH when the bus is not busy. A HIGH-to-LOWtransition of the data line, while the clock is HIGH, is defined as the start condition (S). ALOW-to-HIGH transition of the data line while the clock is HIGH, is defined as the stopcondition (P).Acknowledge The number of data bytes transferred between the start and stop conditions fromtransmitter to receiver is not limited. Each data byte of eight bits is followed by oneacknowledge bit. The acknowledge bit is a HIGH level put on the bus by the transmitterwhereas the master also generates an extra acknowledge related clock pulse. A slave receiver which is addressed must generate an acknowledge after thereception of each byte. Also a master must generate an acknowledge after the reception ofeach byte that has been clocked out of the slave transmitter. The device thatacknowledges has to pull down the SDA line during the acknowledge clock pulse, so thatthe SDA line is stable LOW during the HIGH period of the acknowledge related clockpulse. A master receiver must signal an end of data to the transmitter by not generating anECE, SBMSIT 34
  35. 35. Eye controlled HMIacknowledge on the last byte that has been clocked out of the slave. In this event thetransmitter must leave the data line HIGH to enable the master to generate a stopcondition. Out of various devices we are using I2C Serial ADC (Analog to DigitalConvertor) for acquiring EOG signals.ECE, SBMSIT 35
  36. 36. Eye controlled HMI I2C Based Serial ADC PCF8591: An analog-to-digital converter (ADC) is a device which converts a continuousquantity to a discrete time digital representation. An ADC may also provide an isolatedmeasurement. The reverse operation is performed by a digital-to-analog converter (DAC). Typically, an ADC is an electronic device that converts an input analog voltage orcurrent to a digital number proportional to the magnitude of the voltage or current. Various ADC are available with serial and parallel interfacing. To save the pincount and board space we have decided to use serial ADC, again in serial ADC we haveSPI interface and I2C interface, we have choose I2C serial ADC as it requires only 2lines for interfacing and data rates are sufficient for most of the system. In our project weare using PCF8591 from Philips which is I2C serial ADC/DAC chip.PCF8591 Description: The PCF8591 is a single-chip, single-supply low power 8-bit CMOS dataacquisition device with four analog inputs, one analog output and a serial I2C-businterface. Three address pins A0, A1 and A2 are used for programming the hardwareaddress, allowing the use of up to eight devices connected to the I2C-bus withoutadditional hardware. Address, control and data to and from the device are transferredserially via the two-line bidirectional I2C-bus. The functions of the device include analog input multiplexing, on-chip track andhold function, 8-bit analog-to-digital conversion and an 8-bit digital-to-analogconversion. The maximum conversion rate is given by the maximum speed of the I2C-bus.Features: • Single power supply • Operating supply voltage 2.5 V to 6 VECE, SBMSIT 36
  37. 37. Eye controlled HMI • Low standby current • Serial input/output via I2C-bus • Address by 3 hardware address pins • Sampling rate given by I2C-bus speed • 4 analog inputs programmable as single-ended or differential inputs • Auto-incremented channel selection • Analog voltage range from VSS to VDD • On-chip track and hold circuit • 8-bit successive approximation A/D conversion • Multiplying DAC with one analog output.Applications: • Closed loop control systems • Low power converter for remote data acquisition • Battery operated equipment • Acquisition of analog values in automotive, audio and TV applications. •ECE, SBMSIT 37
  38. 38. Eye controlled HMIBlock Diagram:Pin Description and Pin Diagram:ECE, SBMSIT 38
  39. 39. Eye controlled HMIFUNCTIONAL DESCRIPTIONAddressing Each PCF8591 device in an I2C-bus system is activated by sending a validaddress to the device. The address consists of a fixed part and a programmable part. Theprogrammable part must be set according to the address pins A0, A1 and A2. The addressalways has to be sent as the first byte after the start condition in the I2C-bus protocol. Thelast bit of the address byte is the read/write-bit which sets the direction of the followingdata transfer .Control byteECE, SBMSIT 39
  40. 40. Eye controlled HMI The second byte sent to a PCF8591 device will be stored in its control register andis required to control the device function. The upper nibble of the control register is usedfor enabling the analog output, and for programming the analog inputs as single-ended ordifferential inputs. The lower nibble selects one of the analog input channels defined bythe upper nibble. If the auto-increment flag is set, the channel number is incrementedautomatically after each A/D conversion.A/D conversion The A/D converter makes use of the successive approximation conversiontechnique, the on-chip D/A converter and a high-gain comparator are used temporarilyduring an A/D conversion cycle. An A/D conversion cycle is always started after sendinga valid read mode address to a PCF8591 device. The A/D conversion cycle is triggered atthe trailing edge of the acknowledge clock pulse and is executed while transmitting theresult of the previous conversion. Once a conversion cycle is triggered an input voltage sample of the selectedchannel is stored on the chip and is converted to the corresponding 8-bit binary code.ECE, SBMSIT 40
  41. 41. Eye controlled HMISamples picked up from differential inputs are converted to an 8-bit twos complementcode. The conversion result is stored in the ADC data register and awaits transmission. If the auto-increment flag is set the next channel is selected. The first bytetransmitted in a read cycle contains the conversion result code of the previous read cycle.After a Power-on reset condition the first byte read is a hexadecimal 80. The maximumA/D conversion rate is given by the actual speed of the I2C-bus. Before above steps always we have to send channel selection control byte as writecommand, steps are as follows: o Start o Address with Write cmd(0) o Control byte with channel no(00 , 01 , 10 , 11) o Stop We are using P89V51RD2 to acquire and process the data and final eyemovement cmd is send via RF Tx module using Ht640 encoder, at receiving end after RFRx module data goes to decoder HT648 and then to Application Microcontroller allinterfacing is described in following chapters.ECE, SBMSIT 41
  42. 42. Eye controlled HMI MICROCONTROLLER P89V51RX2 Computer in its simplest form needs at least 3 basic blocks: CPU, I/O and theRAM/ROM. The integrated form of CPU is the microprocessor. As the use ofmicroprocessors in control applications increased, development of microcontroller unit orMCU took shape, wherein CPU, I/O and some limited memory on a single chip wasfabricated. Intention was to reduce the chip count as much as possible. We decided to useP89V51RXX series of Microcontroller. The P89V51RB2/RC2/RD2 are 80C51 microcontrollers with 16/32/64 kB flashand 1024 B of data RAM. A key feature of the P89V51RB2/RC2/RD2 is its X2 modeoption. The design engineer can choose to run the application with the conventional80C51 clock rate (12 clocks per machine cycle) or select the X2 mode (six clocks permachine cycle) to achieve twice the throughput at the same clock frequency. Another wayto benefit from this feature is to keep the same performance by reducing the clockfrequency by half, thus dramatically reducing the EMI. The flash program memory supports both parallel programming and in serial ISP.Parallel programming mode offers gang-programming at high speed, reducingprogramming costs and time to market. ISP allows a device to be reprogrammed in theend product under software control. The capability to field/update the applicationfirmware makes a wide range of applications possible. The P89V51RB2/RC2/RD2 is alsocapable of IAP, allowing the flash program memory to be reconfigured even while theapplication is running.Features of P89V51RXX:  80C51 CPU Core  5 V operating voltage from 0 MHz to 40 MHz  16/32/64 kB of on-chip flash user code memory with ISP and IAP  Supports 12-clock (default) or 6-clock mode selection via software or ISP  SPI and enhanced UART  PCA with PWM and capture/compare functions  Four 8-bit I/O ports with three high-current port 1 pins (16 mA each)ECE, SBMSIT 42
  43. 43. Eye controlled HMI  Three 16-bit timers/counters  Programmable watchdog timer  Eight interrupt sources with four priority levels  Second DPTR register  Low EMI mode (ALE inhibit)  TTL- and CMOS-compatible logic levels  Brownout detection  Low power modes o Power-down mode with external interrupt wake-up o Idle mode  DIP40, PLCC44 and TQFP44 packagesBlock Diagram:ECE, SBMSIT 43
  44. 44. Eye controlled HMIPin Diagram:Pin description of P89CV51RXXPort 0: Port 0 is an 8-bit open drain bidirectional I/O port. Port 0 pins that have ‘1’swritten to them float, and in this state can be used as high-impedance inputs. Port 0 is alsothe multiplexed low-order address and data bus during accesses to external code and datamemory. In this application, it uses strong internal pull-ups when transitioning to ‘1’s.Port 0 also receives the code bytes during the external host mode programming, andoutputs the code bytes during the external host mode verification. External pull-ups arerequired during program verification or as a general purpose I/O port.Port 1: Port 1 is an 8-bit bidirectional I/O port with internal pull-ups. The Port 1 pins arepulled high by the internal pull-ups when ‘1’s are written to them and can be used asECE, SBMSIT 44
  45. 45. Eye controlled HMIinputs in this state. As inputs, Port 1 pins that are externally pulled LOW will sourcecurrent (IIL) because of the internal pull-ups. P1.5, P1.6, P1.7 have high current drive of16 mA. Port 1 also receives the low-order address bytes during the external host modeprogramming and verification.Port 2: Port 2 is an 8-bit bidirectional I/O port with internal pull-ups. Port 2 pins arepulled HIGH by the internal pull-ups when ‘1’s are written to them and can be used asinputs in this state. As inputs, Port 2 pins that are externally pulled LOW will sourcecurrent (IIL) because of the internal pull-ups. Port 2 sends the high-order address byteduring fetches from external program memory and during accesses to external DataMemory that use 16-bit address (MOVX@DPTR). In this application, it uses stronginternal pull-ups when transitioning to ‘1’s. Port 2 also receives some control signals anda partial of high-order address bits during the external host mode programming andverification.Port 3: Port 3 is an 8-bit bidirectional I/O port with internal pull-ups. Port 3 pins arepulled HIGH by the internal pull-ups when ‘1’s are written to them and can be used asinputs in this state. As inputs, Port 3 pins that are externally pulled LOW will sourcecurrent (IIL) because of the internal pull-ups. Port 3 also receives some control signalsand a partial of high-order address bits during the external host mode programming andverification.PSEN: Program Store Enable is the read strobe for external program memory.Reset: While the oscillator is running, a HIGH logic state on this pin for two machinecycles will reset the device.External Access Enable: EA must be connected to VSS in order to enable the device tofetch code from the external program memory. EA must be strapped to VDD for internalprogram execution.ECE, SBMSIT 45
  46. 46. Eye controlled HMIAddress Latch Enable: ALE is the output signal for latching the low byte of theaddress during an access to external memory. This pin is also the programming pulseinput (PROG) for flash programming.Crystal 1: Input to the inverting oscillator amplifier and input to the internal clockgenerator circuits.Crystal 2: Output from the inverting oscillator amplifier.VCC: Supply voltage.GND: Ground. The additional feature of Port3ECE, SBMSIT 46
  47. 47. Eye controlled HMICHAPTER RF INTERFACING For transmission of data/Cmds from one point to another point wirelessly we needto have RF interface, RF interfacing requires 4 parts: 1> RF Transmitter Module 2> RF Receiver Module 3> Transmitter Encoder 4> Receiver DecoderRF Transmitter Module ST433/315 The STT-433/315 is ideal for remote control applications where low cost andlonger range is required. The transmitter operates from a 1.5-12V supply, making it idealfor battery-powered applications. The transmitter employs a SAW-stabilized oscillator,ensuring accurate frequency control for best range performance. Output power andharmonic emissions are easy to control, making FCC and ETSI compliance easy. Themanufacturing-friendly SIP style package and low-cost make the STT-433/315 suitablefor high volume applications.Features• 433.92/315 MHz Frequency• Low Cost• 1.5-12V operation• 11mA current consumption at 3V• Small size• 4 dBm output power at 3VECE, SBMSIT 47
  48. 48. Eye controlled HMIApplications• Car security system • On-Site Paging• Sensor reporting • Asset Tracking• Automation system • Wireless Alarm and Security Systems• Remote Keyless Entry (RKE) • Long Range RFID• Remote Lighting Controls • Automated Resource Management Note: 3pin RF Tx module is having helical wire antenna RF Receiver Module STR-433/315 The STR-433/315 is ideal for short-range remote control applications where costis a primary concern. The receiver module requires no external RF components except forthe antenna. It generates virtually no emissions, making FCC and ETSI approvals easy.The super-regenerative design exhibits exceptional sensitivity at a very low cost. Themanufacturing-friendly SIP style package and low-cost make the STR-433/315 suitablefor high volume applications.ECE, SBMSIT 48
  49. 49. Eye controlled HMIFeatures• Low Cost• 5V operation• 3.5mA current drain• No External Parts are required• Receiver Frequency: 433.92/315 MHZ• Typical sensitivity: -105dBm• IF Frequency: 1MHzApplications: Same as Transmitter ModuleECE, SBMSIT 49
  50. 50. Eye controlled HMI Note: 3pin RF Rx module is having helical wire antenna Transmitter Encoder HT12E (212 series) The 212 encoders are a series of CMOS LSIs for remote control systemapplications. They are capable of encoding information which consists of N address bitsand 12-N data bits. Each address/data input can be set to one of the two logic states. Theprogrammed addresses/ data are transmitted together with the header bits via an RF or aninfrared transmission medium upon receipt of a trigger signal. The capability to select aTE trigger on the HT12E or a DATA trigger on the HT12A further enhances theapplication flexibility of the 212 series of encoders. The HT12A additionally provides a38kHz carrier for infrared systems.Features• Operating voltage• 2.4V~5V for the HT12A• 2.4V~12V for the HT12E• Low power and high noise immunity CMOS technologyECE, SBMSIT 50
  51. 51. Eye controlled HMI• Low standby current: 0.1_A (typ.) at VDD=5V• HT12A with a 38kHz carrier for infrared transmission medium• Minimum transmission word• Four words for the HT12E• One word for the HT12A• Built-in oscillator needs only 5% resistor• Data code has positive polarity• Minimal external components• Pair with Holtek’s 212 series of decoders• 18-pin DIP, 20-pin SOP packageApplications• Burglar alarm system• Smoke and fire alarm system• Garage door controllers• Car door controllers• Car alarm system• Security system• Cordless telephones• Other remote control systemsECE, SBMSIT 51
  52. 52. Eye controlled HMIBlock diagram and Pin Diagram:ECE, SBMSIT 52
  53. 53. Eye controlled HMIECE, SBMSIT 53
  54. 54. Eye controlled HMIECE, SBMSIT 54
  55. 55. Eye controlled HMIFlow chart and application circuit of HT12E working:ECE, SBMSIT 55
  56. 56. Eye controlled HMIReceiver Decoder HT12D (212 series) The 212 decoders are a series of CMOS LSIs for remote control systemapplications. They are paired with Holtek’s 212 series of encoders (refer to theencoder/decoder cross reference table). For proper operation, a pair of encoder/decoderwith the same number of addresses and data format should be chosen. The decodersreceive serial addresses and data from a programmed 212 series of encoders that aretransmitted by a carrier using an RF or an IR transmission medium. They compare theserial input data three times continuously with their local addresses. If no error orunmatched codes are found, the input data codes are decoded and then transferred to theoutput pins. The VT pin also goes high to indicate a valid transmission. The 212 series ofdecoders are capable of decoding information that consists of N bits of address and 12_Nbits of data. Of this series, the HT12D is arranged to provide 8 address bits and 4 databits, and HT12F is used to decode 12 bits of address information.Features• Operating voltage: 2.4V~12V• Low power and high noise immunity CMOS technology• Low standby current• Capable of decoding 12 bits of information• Binary address setting• Received codes are checked 3 times• Address/Data number combination• HT12D: 8 address bits and 4 data bits• HT12F: 12 address bits only• Built-in oscillator needs only 5% resistor• Valid transmission indicator• Easy interface with an RF or an infrared transmission medium• Minimal external components• Pair with Holtek’s 212 series of encoders• 18-pin DIP, 20-pin SOP packageECE, SBMSIT 56
  57. 57. Eye controlled HMIApplications• Burglar alarm system• Smoke and fire alarm system• Garage door controllers• Car door controllers• Car alarm system• Security system• Cordless telephones• Other remote control systemsBlock Diagram and Pin Diagram:ECE, SBMSIT 57
  58. 58. Eye controlled HMIECE, SBMSIT 58
  59. 59. Eye controlled HMIOperation The 212 series of decoders provides various combinations of addresses and datapins in different packages so as to pair with the 212 series of encoders. The decodersreceive data that are transmitted by an encoder and interpret the first N bits of code periodas addresses and the last 12_N bits as data, where N is the address code number. A signalon the DIN pin activates the oscillator which in turn decodes the incoming address anddata. The decoders will then check the received address three times continuously. If thereceived address codes all match the contents of the decoder’s local address, the 12_Nbits of data are decoded to activate the output pins and the VT pin is set high to indicate avalid transmission. This will last unless the address code is incorrect or no signal isreceived. The output of the VT pin is high only when the transmissionis valid. Otherwise it is always low.Output Type The 212 series of decoders, the HT12F has no data output pin but its VT pin canbe used as a momentary data output. The HT12D, on the other hand, provides 4 latch typedata pins whose data remain unchanged until new data are received.ECE, SBMSIT 59
  60. 60. Eye controlled HMIFlow Chart and Application Circuit The oscillator is disabled in the standby state and activated when a logic “high”signal applies to the DIN pin. That is to say, the DIN should be kept low if there is nosignal inputECE, SBMSIT 60
  61. 61. Eye controlled HMIFlow Chart:Transmitter Module Interfacing This is complete data transmission module, Microcontroller first send data to datalines of encoder, address lines of encoder are hardwire, after giving data microcontrollerenables TE pin of encoder in order to start encoding process and at Dout pin of encoderserial encoded data is coming this serial encoded data contains not only data but itECE, SBMSIT 61
  62. 62. Eye controlled HMIcontains module address and sync bits, this signal is fed to RF Tx module which doesOOK modulation and transmits data wirelessly. TX RF Encoder Module Microcontroller HT12E Data line D0-D7Transmit EnableReceiver Module Interfacing This is complete data Reception module; RF Rx module receives the wireless bitstream this is fed to Decoder which first compares the address and sync if all is ok then itcompares data two times if both are same then only it enables VT pin (ValidTransmission) and latch the output at data lines, when new data is received VT goes lowand then again with same logic it is set to high if same new data is received two times.Rising edge of VT indicates new data has come falling edge of VT indicates data hasstopped or new data is going to come. We can read data at both edges. This can work intwo modes polling VT mode and Interrupt Mode in which VT is connected to INT0 pinof MCU. RX RF Module DecoderMicrocontroller HT12DVTValid Transmission line Data D0-D7ECE, SBMSIT 62
  63. 63. Eye controlled HMIACQUISITION AND PROCESSING SYSTEM Acquisition front end system will interface to the body get the EOG signal,amplify it, filter it and pre process it to suit to ADC PCF8591 I2C Based 4 Channel ADC.Left-right signal is given to channel 0 and up-down signal is given channel-1, this ADC isinterfaced to Microcontroller P89V51RD2 which is having I2C communication routines.The microcontroller reads the data from ADC using I2C protocol and starts processing.Once data is processed and if any eye movement was there it will conclude which eyemovement was made and decodes which command is given using eye. After decoding itsends the command via RF transmitter module using HT12E Encoder.Circuit diagram is given below: Figure: Acquisition and processing partECE, SBMSIT 63
  64. 64. Eye controlled HMIProcessing of data to decode the eye movements: Basically we get digital data from ADC for each channel, first we are checking forstraight sight to avoid noise and electrode not in use case. This we are doing by checkingthat signal is not varying much it’s in some band near center. After the if signal goes upfor sufficient time > 200ms then its right eye movement in case of L-R and Up movementin case of U-D, but if signal goes down then its left or down depending on which channelyou are processing. Any of the case if it come back before sufficient time then movementis ignored but in case of up down, if signal is up for >50ms to <100ms then its consider asblink movement.Flow chart is for above processing is given below:ECE, SBMSIT 64
  65. 65. Eye controlled HMIWe have looked in straight direction and signals are stabilized, now its turn to givecommands using eye movements.Following are the follow charts to detect left, right, up, down and blink eye movements. Figure: Flow chart for Left-Right DetectionECE, SBMSIT 65
  66. 66. Eye controlled HMI Figure: Flow chart for up-down and blink detections. These commands are sent via RF Tx module at application part end there is RFRx module which receives the commands and send it to application controller which thendrives the cursor and operates buttons on Graphic LCDECE, SBMSIT 66
  67. 67. Eye controlled HMI Graphics LCD JHD12864 JHD12864J is a light weight, low power consumption liquid crystal graphicdisplay. The module measures 54.0x50.0mm only. Supply voltage is 5V matching thevoltage for most microcontrollers. The LCD controller is Samsung KS0108B.This LCD has 20 line interfacing which are described below: Pin Description Symbol Level Function Vss 0V Ground Vdd +5V Power supply for logic Vo - Operating voltage for LCD (contrast adjusting) RS H/L Register selection H:Display data L:Instruction code R/W H/L Read/Write selection H:Read operation L:Write Operation H,H- Enable Signal.Read data when E is high,Write data at the falling E >L Edge of E DB0 H/L Data bit 0 DB1 H/L Data bit 1 DB2 H/L Data bit 2ECE, SBMSIT 67
  68. 68. Eye controlled HMI DB3 H/L Data bit 3 DB4 H/L Data bit 4 DB5 H/L Data bit 5 DB6 H/L Data bit 6 DB7 H/L Data bit 7 CS1 H select the right half of display the CS1 bit is set CS2 H select the left half of display the CS2 bit is set /RST L Reset signal, active low Vout -10V Output voltage for LCD driving LEDA +5V Power supply for LED back light LEDB 0V GND for LED back lightThe display is split logically in half. It contains two controllers with controller #1 (Chipselect 1) controlling the left half of the display and controller #2 (Chip select 2)controlling the right half. Each controller must be addressed independently. The pageaddresses, 0-7, specify one of the 8 horizontal pages which are 8 bits (1 byte) high. Adrawing of the display and how it is mapped to the refresh memory is shown below.ECE, SBMSIT 68
  69. 69. Eye controlled HMIBlock Diagram of GLCD:ECE, SBMSIT 69
  70. 70. Eye controlled HMIFollowing are display control Instruction which MCU has to give whileinterfacing to GLCD module:ECE, SBMSIT 70
  71. 71. Eye controlled HMIBasic interfacing diagram with MCU:ECE, SBMSIT 71
  72. 72. Eye controlled HMIAlgorithm and flowchart for GLCD interfacing1.Send the display off command 3eh2. Send the display on command 3fh3.If required you can use 11xx xxxx instruction to set the display line start4. Set the Y-adddress to first coloumn 40h5.Set the X-address to first page 0B8h6.Blank the Display( clear all 128x64 pixels)ECE, SBMSIT 72
  73. 73. Eye controlled HMI Application Part Application part consists of Graphics LCD, P89V51RD2 microcontroller, RFreceiver module with decoder HT12D, ULN2008 high current Darlington driver forcontrolling high current devices or relays. Eye Movements commands are sent to application part via RF transmitter, the RFreceiver receives the data and HT12D does channel decoding and give digital data toMCU and with VT pin signal it gives indication that data is received. Which trigger theinterrupt and in ISR we are read output of encoder into the MCU, and data is decoded forcursor commands and blinks (clicks). According to commands cursor is move on thescreen but menu is only visible after 4 blinks. After moving to cursor to required button 2blinks are required to click the button and designated operation is performed after 2blinksare received. This microcontroller control the graphic lcd, it prints messages on screen, itcreates menu on screen with 4 buttons and one cursor, for each cursor command cursor onscreen is moved using creating the new cursor at new position and removing old cursor. 4 devices are connected at four MCU pins, and this pins goes to input ofULN2803 so devices using voltages 5-12V and current upto 400-500mA can becontrolled directly and high current and high voltages devices can be controlled viarelays. Buzzer and FAN in our demo project are duty cycle, we duty cycle FAN to savepower and Buzzer is duty cycled so that patient does not have to activate buzzer again andagain till someone comes and attain him. Buzzer is made off by a buzzer reset switch.ECE, SBMSIT 73
  74. 74. Eye controlled HMICircuit diagram and flowcharts are given in following pagesFlow chart for ApplicationECE, SBMSIT 74
  75. 75. Eye controlled HMIECE, SBMSIT 75
  76. 76. Eye controlled HMICircuit for Application part:ECE, SBMSIT 76
  77. 77. Eye controlled HMI Software and hardware requirementsSoftware and Hardware requirements along with component listPC Requirements:Pentium 4 PC or higherOS: Win XP or higher.Minimum 1GB RAM.Software: KEIL, Flash Magic and HyperTerminalHardware components: RF Transmitter and receiver – 315/433 MHzENCODER & DECODER BRDS - WITH HT12E 648 RELAY BOARD - Transistor Based GRAPHIC DISPLAY - 128X64 HT12E - 4 BIT ENCODER HT12D - 4 BIT DECODER LM324– OPAMP X2 INA (Instrumentation amplifier) AD620 X2 I2C based ADC PCF8591 Power Supply 9V DC adapter, and 9V Batteries X3 78L05, 79L05 FAN, Buzzer and Cables P89V51RD2 and General Purpose MCU Board X2 Finally Electrodes and Medical AccessoriesECE, SBMSIT 77
  78. 78. Eye controlled HMI Applications and Future work ApplicationsApplications are not only limited to disable persons although this technology is mostuseful for them: • Control of House hold appliances and Emergency call Bell.A screen is provided with some buttons on that disable person move the cursor with eyemovement/ or cursor is slowly moving and to click a button he/she blinks the eye twotimes and whatever button is meant for its executed, it can be turning on & off lightlights, calling assistance using bell etc. • Speech system for disablethis system enables disable person to talk via computer which has a special applicationrunning having on screen keyboard, a text box not only this it as prediction of text logicas he/she is trying to type by using eye movements on on screen keyboard system providepredicted text in the drop down of text box which one can select to speedtext entry process • Email facility for disablelike previous application it has all interfaces and along with that email client is therewhich sends email to fixed address or address can be typed via eye movement or selectedfrom address book again using eye signals only. • Eye movement controlled wheel chairWheel chair can go forward, backward, turn left and right using eye movements of thedisable person sitting on it.ECE, SBMSIT 78
  79. 79. Eye controlled HMI • US army is doing research on eye movement based Air craft and tank controls • Hands free mouse using eye movement controller very useful specially in gaming • TV operation channel up & down volume up & down, turn on and off TV using eye movements very useful for disables • Eye movements controlled mp3 player when ur hands are busy you can use this system to play/pause, track change, and volume up&down using eye movementsECE, SBMSIT 79
  80. 80. Eye controlled HMI conclusion EXPECTED OUTPUT AND CONCLUSIONIntermediate Output of the system is Eye movements and Eye Blinking CommandsCU - Eye up movement detectedCD - Eye down movement detectedCL - Eye Left movement detectedCR - Eye Right movement detectedBL - Eye Blink detectedAbove o/p is sent to application part which interprets and move cursors accordingly andbutton are clicked using same o/p and corresponding relay/device is operated and henceappliance is controlled using eye movements, in this project we have successfullycontrolled appliances using eye movement.Achievement:Detection of Eye movements and eye blink at minimum 10sec rate with 1Sigma accuracy.ECE, SBMSIT 80
  81. 81. Eye controlled HMI REFERENCES • http://en.wikipedia.org/wiki/Eye_tracking • The 8051 Microcontroller, Kenneth J Ayala • C and the 8051, Thomas W. Schultz • IEEE Paper: EOG signal detection for home appliances activation • Human-Computer Systems Interaction: Backgrounds and Applications By Zdzislaw S. Hippe, Juliusz L. Kulikowski • Hand book Of Biomedical Instrumentation By Khandpur. • Fun n Games By Panos Markopoulos, Boris de Ruyter, Wijnand Ijsselsteijn. • Intelligent wearable interfaces By Yangsheng Xu, Wen J. Li, Ka Keung Caramon Lee • Manuals in keil software • I2C-bus specification (version 2.1), from NXP semiconductors (Philips). • Datasheets of all the IC’s used in the systemECE, SBMSIT 81

×