Master Thesis Improvement of Response Times in        SSVEP-based BCI Ignatius Sapto Condro Atmawan Bisawarna           Ma...
Content•    Introduction•    Simulation•    Implementation•    Experiment•    ConclusionImprovement of Response Times in S...
SSVEP based BCI (I)Brain-Computer Interface (BCI) is a  communication system in which messages  or commands that an indivi...
SSVEP based BCI (II)Steady-state visual evoked potential• Electrophysiological response of the  visual cortex• Resonance p...
Bremen SSVEP-based BCI• Brain-Computer Interface (BCI) research at the  IAT of the University of Bremen started in 2005• I...
Time Series Prediction forBremen BCIImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          6
Time Series Prediction                   •    Three-point Quadratic Model                   •    Regression               ...
Quadratic ModelImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          8
RegressionImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          9
Logical Trend-based DecisionThe present value should be larger than the previous value.• Three points• More pointsImprovem...
Kalman FilterKalman Filter is used with state space model of a systemIt contains 2 steps:• Prediction• Measurement or upda...
Simulation:Three-point Quadratic ModelImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010       ...
Simulation:Regression, 5 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          13
Simulation:Regression, 8 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          14
Simulation:Logical Trend-based Decision• Decision is based on trends or gradients  (of the SNRs).• The result contains 6 c...
Simulation:Logical Trend-based Decision,2 delay taps (three points)Improvement of Response Times in SSVEP-BCIIgnatius Sapt...
Simulation:Logical Trend-based Decision,5 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawa...
Simulation:Logical Trend-based Decision,8 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawa...
Simulation:Kalman Filter, 5 delay taps,Simplified formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atma...
Simulation:Kalman Filter, 5 delay taps,Särkää´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawa...
Simulation:Kalman Filter, 5 delay taps,Joseph´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawa...
Software Implementation:Time Series Prediction in BCI2000Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro A...
Experiment• 11 subjects (2 females, 9 males), with age  range 21-30 years old.• 4 LEDs• 8 EEG electrodes• Sampling frequen...
Experiment:ProtocolImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          24
Experiment:Measured Parameters• Speed• Accuracy• Information transfer rate (ITR)Improvement of Response Times in SSVEP-BCI...
Experiment I : SpeedIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 20...
Experiment I : AccuracyIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan,...
Experiment I : ITRIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010...
Experiment II : SpeedIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2...
Experiment II : AccuracyIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan...
Experiment II : ITRIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 201...
Experiment III : SpeedIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, ...
Experiment III : AccuracyIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawa...
Experiment III : ITRIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 20...
Conclusion• Time Series Prediction can improve response  time• Regression model, with 8 delay taps, has the best ITR• Kalm...
Future works• The Time Series Prediction algorithms can  be implemented in other BCI applications:  spelling, moving wheel...
Thank YouImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010             37
Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          38
Back UpImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010            39
Kalman Filter: state space• System Model• Measurement Model• Output ModelImprovement of Response Times in SSVEP-BCIIgnatiu...
Kalman Filter:System model for TSP                                                  orImprovement of Response Times in SSV...
Kalman Filter:Measurement Model for TSPImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010      ...
Kalman Filter:Output Model for TSPm is number of steps ahead for predictionImprovement of Response Times in SSVEP-BCIIgnat...
Kalman Filter:Prediction stepImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          44
Kalman Filter:Measurement & Updating Step (I)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010...
Kalman Filter:Measurement & Updating Step (II)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 201...
Kalman Filter:Measurement & Updating Step (III)Updated (a posteriori) covariance estimate• Simplified form•• Särkää´s form...
Kalman Filter:Measurement & Updating Step (III)Updated (a posteriori) covariance estimate• Simplified form•               ...
Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          49
Simulation• MATLAB R2006b (version 7.3) from  Mathworks is used• The data used is from the experiment with the  visor cap ...
Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          51
Software Implementation• Two standard C++ classes:  cRegression3 and cKalmanIATBCI.• BCI2000 - version 2 can be compiled  ...
Software Implementation:C++ classes• Method double getRegression(double dYInput,int Nt)• Method double getKalmanFilter(dou...
Software Implementation:Block Diagram• Pvi = Probability values at SNR channel i• Pvi can be called Normalised SNRsImprove...
Software Implementation:BCI2000BCI2000 is a general-purpose system for BCIBCI2000 supports different kinds of• Signal acqu...
Software Implementation:BCI2000, filtering module IImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan...
Software Implementation:ClassifierConnectImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010    ...
Hardware Implementation• Porti7, with 32 channels, as USB Amplifier.• LED Array.• LED Controller, with PIC 16F877.Improvem...
Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          59
Experiment:EEG Cap ConfigurationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          60
Experiment:Subject InformationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          61
Experiment:Subject ParticipationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          62
Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010          63
Upcoming SlideShare
Loading in …5
×

Condro2010 thesis slide_v3

923 views

Published on

This is my master thesis presentation in 2010: "Improvement of Response Times in SSVEP-based Brain-Computer Interface", in the University of Bremen, Germany.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
923
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Condro2010 thesis slide_v3

  1. 1. Master Thesis Improvement of Response Times in SSVEP-based BCI Ignatius Sapto Condro Atmawan Bisawarna Matrikel Number 2113914 Supervised by: Prof. Dr.-Ing. Axel Gräser Dr.-Ing. Ivan Volosyak Thorsten Lüth, Dipl.-Ing.
  2. 2. Content• Introduction• Simulation• Implementation• Experiment• ConclusionImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 2
  3. 3. SSVEP based BCI (I)Brain-Computer Interface (BCI) is a communication system in which messages or commands that an individual sends to the external world do not pass through the brain’s normal output pathways of peripheral nerves and muscles.(Wolpaw, et al. 2002. Clinical Neurophysiology)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 3
  4. 4. SSVEP based BCI (II)Steady-state visual evoked potential• Electrophysiological response of the visual cortex• Resonance phenomena• Rapidly repeating visual stimulus: flickering LED or lamp, blinking picture on screen and other light sources.• Frequency above 4 HzImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 4
  5. 5. Bremen SSVEP-based BCI• Brain-Computer Interface (BCI) research at the IAT of the University of Bremen started in 2005• It is intended to make a faster BCI system, but the system should not lose its accuracy (too much)• IAT Bremen BCI uses minimum energy combination (MEC) algorithm to detect SSVEP• With MEC, the signal power of a certain frequency, as well as the SNR, are estimated.• The SNR is used for classification with thresholdingImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 5
  6. 6. Time Series Prediction forBremen BCIImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 6
  7. 7. Time Series Prediction • Three-point Quadratic Model • Regression • Logical Trend-based • Kalman FilterImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 7
  8. 8. Quadratic ModelImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 8
  9. 9. RegressionImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 9
  10. 10. Logical Trend-based DecisionThe present value should be larger than the previous value.• Three points• More pointsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 10
  11. 11. Kalman FilterKalman Filter is used with state space model of a systemIt contains 2 steps:• Prediction• Measurement or updatingWhat is updated?• State• CovarianceHow they are updated?• Simplified form• Särkää´s form• Joseph´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 11
  12. 12. Simulation:Three-point Quadratic ModelImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 12
  13. 13. Simulation:Regression, 5 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 13
  14. 14. Simulation:Regression, 8 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 14
  15. 15. Simulation:Logical Trend-based Decision• Decision is based on trends or gradients (of the SNRs).• The result contains 6 commands, correlated to 5 LEDs and no selection.• There is redundancy, so the values (of the SNRs) have to be used.• If redundancy happens, the maximum value is selected.Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 15
  16. 16. Simulation:Logical Trend-based Decision,2 delay taps (three points)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 16
  17. 17. Simulation:Logical Trend-based Decision,5 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 17
  18. 18. Simulation:Logical Trend-based Decision,8 delay tapsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 18
  19. 19. Simulation:Kalman Filter, 5 delay taps,Simplified formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 19
  20. 20. Simulation:Kalman Filter, 5 delay taps,Särkää´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 20
  21. 21. Simulation:Kalman Filter, 5 delay taps,Joseph´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 21
  22. 22. Software Implementation:Time Series Prediction in BCI2000Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 22
  23. 23. Experiment• 11 subjects (2 females, 9 males), with age range 21-30 years old.• 4 LEDs• 8 EEG electrodes• Sampling frequency = 2048 Hz• Segment Length = 2 sThere are 3 experiments• Experiment I : 8 subjects• Experiment II : 7 subjects• Experiment III: 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 23
  24. 24. Experiment:ProtocolImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 24
  25. 25. Experiment:Measured Parameters• Speed• Accuracy• Information transfer rate (ITR)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 25
  26. 26. Experiment I : SpeedIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 26
  27. 27. Experiment I : AccuracyIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 27
  28. 28. Experiment I : ITRIdle Period 1 s, 8 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 28
  29. 29. Experiment II : SpeedIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 29
  30. 30. Experiment II : AccuracyIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 30
  31. 31. Experiment II : ITRIdle Period 2 s, 7 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 31
  32. 32. Experiment III : SpeedIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 32
  33. 33. Experiment III : AccuracyIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 33
  34. 34. Experiment III : ITRIdle Period 2 s, 3 subjectsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 34
  35. 35. Conclusion• Time Series Prediction can improve response time• Regression model, with 8 delay taps, has the best ITR• Kalman Filter can improve ITR, if 5 or 10 steps are chosen.• The optimal forms of Kalman Filter are the simplified and the Särkää´s• Joseph´s form of Kalman Filter has failed in simulation, so it is not implemented.• The Quadratic Three-point model increases the speed but lose the accuracy too much so it shows poor ITR• Logical trend-based decision has failed in simulation, so it is not implemented• A decision based on only trend or gradient does not work• Idle period should not be lower than segment lengthImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 35
  36. 36. Future works• The Time Series Prediction algorithms can be implemented in other BCI applications: spelling, moving wheelchair or robots and so on.• The transient response of Kalman Filter can be observed and recorded by adding more C++ code for data acquisition.Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 36
  37. 37. Thank YouImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 37
  38. 38. Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 38
  39. 39. Back UpImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 39
  40. 40. Kalman Filter: state space• System Model• Measurement Model• Output ModelImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 40
  41. 41. Kalman Filter:System model for TSP orImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 41
  42. 42. Kalman Filter:Measurement Model for TSPImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 42
  43. 43. Kalman Filter:Output Model for TSPm is number of steps ahead for predictionImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 43
  44. 44. Kalman Filter:Prediction stepImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 44
  45. 45. Kalman Filter:Measurement & Updating Step (I)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 45
  46. 46. Kalman Filter:Measurement & Updating Step (II)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 46
  47. 47. Kalman Filter:Measurement & Updating Step (III)Updated (a posteriori) covariance estimate• Simplified form•• Särkää´s form•• Joseph´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 47
  48. 48. Kalman Filter:Measurement & Updating Step (III)Updated (a posteriori) covariance estimate• Simplified form• with• Särkää´s form• with• Joseph´s formImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 48
  49. 49. Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 49
  50. 50. Simulation• MATLAB R2006b (version 7.3) from Mathworks is used• The data used is from the experiment with the visor cap (wearable SSVEP stimulator).• 6 EEG electrodes• 5 LEDs with different frequencies• Sampling frequency = 128 Hz• Segment Length = 2 sImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 50
  51. 51. Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 51
  52. 52. Software Implementation• Two standard C++ classes: cRegression3 and cKalmanIATBCI.• BCI2000 - version 2 can be compiled only with Borland C++ Builder 6.0• ClassifierConnect.Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 52
  53. 53. Software Implementation:C++ classes• Method double getRegression(double dYInput,int Nt)• Method double getKalmanFilter(double dInput, double dVariance, int iStep, bool bChoice)Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 53
  54. 54. Software Implementation:Block Diagram• Pvi = Probability values at SNR channel i• Pvi can be called Normalised SNRsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 54
  55. 55. Software Implementation:BCI2000BCI2000 is a general-purpose system for BCIBCI2000 supports different kinds of• Signal acquisition devices• Signal processing• BCI applicationsImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 55
  56. 56. Software Implementation:BCI2000, filtering module IImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 56
  57. 57. Software Implementation:ClassifierConnectImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 57
  58. 58. Hardware Implementation• Porti7, with 32 channels, as USB Amplifier.• LED Array.• LED Controller, with PIC 16F877.Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 58
  59. 59. Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 59
  60. 60. Experiment:EEG Cap ConfigurationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 60
  61. 61. Experiment:Subject InformationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 61
  62. 62. Experiment:Subject ParticipationImprovement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 62
  63. 63. Improvement of Response Times in SSVEP-BCIIgnatius Sapto Condro Atmawan, 2010 63

×