Feature-Based Information Processing
of Selective Attention through Entropy
Analysis system
Giacomo Veneri
November 2012
G...
Objectives
• Study the influence of (eye) motor control on
selective attention
• Develop a method to extract motor control...
Selective Attention
• Selective attention ( Posner,
1980) is the process to select
some region of the scene to
be processe...
Motor Control and Cerebellum
• The neuronal circuitry of the
cerebellum is thought to
encode internal models that
reproduc...
Attention and Motor control
(Corbetta2001, Osborne2011)
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science...
Methods
1. Veneri, G., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2010). Influences of data filtering on human-c...
PSYCHOLOGICAL TEST
Eye Tracking, TMT, ET
Methods Results
Attention
FE
Motor
Control
FE
TMT
ET
Healthy
Subjects
Patients
SC...
Eye Tracking
• Eye tracking is the
process of measuring
either the point of gaze
(where one is looking)
or the motion of a...
Visual (conjunction) Search Test
E Search (Wolfe, 1994) Sequencing (Reitan, 1958)
... and others (Veneri 2010, Veneri 2012...
SELECTIVE ATTENTION FEATURES
EXTRACT
Psycological Test, Mathematical Method
Methods Results
Attention
FE
Motor
Control
FE
...
Attention Features Extraction 1/2
Common Method
• Visited ROI
• Reaction Time
Our geometric Method (Veneri,
Rosini 2012)
•...
Sequencing (2/2)
• Look for the best path (Veneri, Rosini 2012)
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral...
MOTOR CONTROL FEATURES
EXTRACTION
Wavelet Entropy
Methods Results
Attention
FE
Motor
Control
FE
TMT
ET
Healthy
Subjects
Pa...
Motor Control Noise Evaluation
• (Beers2007, Veneri2011)
gaze noise may be additive
with or multiplicative of the
eye move...
Frequency Analysis
Fourier analysis
• A signal is a «sum» of a sine
curve
ECG Example
Giacomo Veneri – EVALab - Dep. Neuro...
Wavelet and Entropy
Wavelet Multiscal
decomposition Wavelet (Mallat, 1989)
Giacomo Veneri – EVALab - Dep. Neurological and...
Decomposed Eye Signal
Original signal
Noise?
Main componet
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Scie...
Wavelet Entropy
The idea (Veneri 2011)
• After decomposition
• We removed spikes
• We evaluated Entropy
• Entropy is the m...
RESULTS
Healthy Subjects and Patients
Methods Results
Attention
FE
Motor
Control
FE
TMT
ET
Healthy
Subjects
Patients
SCA2,...
Despiking
Healthy Subject Patient
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
20
Despiking
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
21
Healthy Subjects
Clusters ROC (20% error rate)
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
...
Patients
P-value Clusters
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
23
Entropy levels
All levels Last level
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
24
Variance
Signal Signal on fixations
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
25
Before conclusions
• Proposed Wavelet
Entropy Implementation
is NOT noise on fixations
or noise of global signal
• Propose...
Selective attention
• DT provided a indicator to under-
stand the ability of humans to
converge to the target.
• ANOVA rep...
Correlation DT-E
• Pearson and Spearman test reported correlation between E and DT
for NDC patients (p < 0.05, ρ = 0.892, ...
CONCLUSIONS
Tools and Hypothesis
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
29
Summary
• In the current work two methods have been developed:
• Selective attention evaluation
• Entropy analysis through...
Tool
1. Import Eye gaze data
2. Export Eye gaze data
3. Fixations recognition
(Veneri, Piu, et al., 2010,
2011; Salvucci &...
Study the influence
• Does the motor control (cerebellum) influence
selective attention?
Giacomo Veneri – EVALab - Dep. Ne...
Cerebellum could influence selective
attention (Top-Down) sending
afferent information of noise in order
to minimize the f...
THANKS
Feature-Based Information Processing of Selective Attention through
Entropy Analysis system
Giacomo Veneri – EVALab...
Model
Energy Saccade length
Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI
35
Upcoming SlideShare
Loading in …5
×

Giacomo Veneri 2012 phd dissertation

1,071 views

Published on

Study the influence of (eye) motor control on selective attention
Develop a method to extract motor control parameters during visual search
Develop a method to extract selective attention features during visual search

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,071
On SlideShare
0
From Embeds
0
Number of Embeds
108
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Giacomo Veneri 2012 phd dissertation

  1. 1. Feature-Based Information Processing of Selective Attention through Entropy Analysis system Giacomo Veneri November 2012 Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 1
  2. 2. Objectives • Study the influence of (eye) motor control on selective attention • Develop a method to extract motor control parameters during visual search • Develop a method to extract selective attention features during visual search Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 2 Methods Results Attention FE Motor Control FE TMT ET Healthy Subjects Patients SCA2,NDC Psychological Test
  3. 3. Selective Attention • Selective attention ( Posner, 1980) is the process to select some region of the scene to be processed in detail; then, selective attention works as filter. • Top-Down: attentional process that influences sensory processing in an automatic and persistent manner • Bottom-Up: influence on the nervous system due to extrinsic properties of the stimuli Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 3
  4. 4. Motor Control and Cerebellum • The neuronal circuitry of the cerebellum is thought to encode internal models that reproduce the dynamic properties of body parts (Kelly2003,Ito2005,Ito2006a). • These models control the movement allowing the brain to precisely control the movement without the need for sensory feedback (Barlow2002,Ito2008,King2011 ) • SCA2 and NDC Patients Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 4
  5. 5. Attention and Motor control (Corbetta2001, Osborne2011) Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 5
  6. 6. Methods 1. Veneri, G., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2010). Influences of data filtering on human-computer interaction by gaze-contingent display and eye-tracking applications. Computers in Human Behavior , 26 (6), 1555 - 1563. doi: 10.1016/j.chb.2010.05.030 [SCOPUS, ACM] 2. Veneri, G., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2011, Mar). Spike removal through multiscale wavelet and entropy analysis of ocular motor noise: A case study in patients with cerebellar disease. Journal of Neuroscience Methods , 196 (2), 318–326. doi: 10.1016/j.jneumeth.2011.01.006 [MEDLINE, SCOPUS] 3. Veneri, G., Piu, P., Rosini, F., Federighi, P., Federico, A., & Rufa, A. (2011). Automatic eye fixations identification based on analysis of variance and covariance. Pattern Recognition Letters , 32 (13), 1588 - 1593. doi: 10.1016/j.patrec.2011.06.012 [SCOPUS] 4. Veneri, G., Pretegiani, E., Rosini, F., Federighi, P., Federico, A., & Rufa, A. (2011, Mar). Evaluating the human ongoing visual search performance by eye tracking application and se-quencing tests. Comput Methods Programs Biomed . Retrieved from http://dx.doi.org/10.1016/j.cmpb.2011.02.006 doi:10.1016/j.cmpb.2011.02.006 [SCOPUS. MEDLINE, ACM] 5. Veneri, G., Rosini, F., Federighi, P., Federico, A., & Rufa, A.(2012, Feb). Evaluating gaze control on a multi-target sequenc-ing task: The distribution of fixations is evidence of exploration optimisation. Comput Biol Med , 42 (2), 235–244. Retrieved from http://dx.doi.org/10.1016/j.compbiomed.2011.11.013 doi: 10.1016/j.compbiomed.2011.11.013 [SCOPUS. MEDLINE, ACM] InProceedings 1. Veneri, G., Federighi, P., Pretegiani, E., Rosini, F., Federico, A., & Rufa, A. (2009). Eye tracking - stimulus integrated semi automatic case base system. In Proceeding of the 13th world multi-conference on systemics, cybernetics and informatics. 2. Veneri, G., Pretegiani, E., Federighi, P., Rosini, F., & Rufa, A. (2010). Evaluating human visual search performance by monte carlo methods and heuristic model. In IEEE (Ed.), 10th ieee international conference on information technology and applications in biomedicine (itab 2010). [SCOPUS, IEEE] 3. Veneri, G., Piu, P., Federighi, P., Rosini, F., Federico, A., & Rufa, A. (2010, jun.). Eye fixations identification based on statistical analysis - case study. In Cognitive information processing (cip), 2010 2nd international workshop on (p. 446 -451). IEEE. doi: 10.1109/CIP.2010.5604221 [SCOPUS, IEEE] Others (posters) 1. Veneri, G., Federighi, P., Rosini, F., Pretegiani, E., Federico, A., & Rufa, A. (2009). The role of latest fixations on ongoing visual search: a model to evaluate the selection mechanism. In Rovereto workshop of attention. 2. Veneri, G., Olivetti, E., Avesani, P., Federico, A., & Rufa, A. (2011). Bayesian hypothesis on selective attention. In Rovereto visual attention congress. Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 6
  7. 7. PSYCHOLOGICAL TEST Eye Tracking, TMT, ET Methods Results Attention FE Motor Control FE TMT ET Healthy Subjects Patients SCA2,NDC Psychological Test Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 7
  8. 8. Eye Tracking • Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. • ASL 3000 (240Hz) Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 8
  9. 9. Visual (conjunction) Search Test E Search (Wolfe, 1994) Sequencing (Reitan, 1958) ... and others (Veneri 2010, Veneri 2012)Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 9
  10. 10. SELECTIVE ATTENTION FEATURES EXTRACT Psycological Test, Mathematical Method Methods Results Attention FE Motor Control FE TMT ET Healthy Subjects Patients SCA2,NDC Psychological Test Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 10
  11. 11. Attention Features Extraction 1/2 Common Method • Visited ROI • Reaction Time Our geometric Method (Veneri, Rosini 2012) • Distance to nearest Target • Distance to Nearest ROI • Sequencing Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 11 DN DT
  12. 12. Sequencing (2/2) • Look for the best path (Veneri, Rosini 2012) Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 12
  13. 13. MOTOR CONTROL FEATURES EXTRACTION Wavelet Entropy Methods Results Attention FE Motor Control FE TMT ET Healthy Subjects Patients SCA2,NDC Psychological Test Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 13
  14. 14. Motor Control Noise Evaluation • (Beers2007, Veneri2011) gaze noise may be additive with or multiplicative of the eye movement, and is lost in recording noise (RN) due to blinks or signal loss; • noise = PN + RN = SDN (signal) + ADN + RN where SDN is physiological signal dependent noise and ADN physiological additive noise. Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 14
  15. 15. Frequency Analysis Fourier analysis • A signal is a «sum» of a sine curve ECG Example Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 15
  16. 16. Wavelet and Entropy Wavelet Multiscal decomposition Wavelet (Mallat, 1989) Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 16
  17. 17. Decomposed Eye Signal Original signal Noise? Main componet Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 17
  18. 18. Wavelet Entropy The idea (Veneri 2011) • After decomposition • We removed spikes • We evaluated Entropy • Entropy is the measure of the chaos on a system Algorithm Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 18
  19. 19. RESULTS Healthy Subjects and Patients Methods Results Attention FE Motor Control FE TMT ET Healthy Subjects Patients SCA2,NDC Psychological Test Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 19
  20. 20. Despiking Healthy Subject Patient Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 20
  21. 21. Despiking Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 21
  22. 22. Healthy Subjects Clusters ROC (20% error rate) Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 22
  23. 23. Patients P-value Clusters Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 23
  24. 24. Entropy levels All levels Last level Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 24
  25. 25. Variance Signal Signal on fixations Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 25
  26. 26. Before conclusions • Proposed Wavelet Entropy Implementation is NOT noise on fixations or noise of global signal • Proposed Wavelet Entropy Implementation «catches» motor noise topical featurese of each subject (colored noise) • Wavelet Type or levels are critical Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 26
  27. 27. Selective attention • DT provided a indicator to under- stand the ability of humans to converge to the target. • ANOVA reported significant difference among groups (F (2, 35) = 9.476, p < 0.01) • post-hoc Sidak procedure confirmed significant difference between – CTRL-SCA2 (p CTRL−SCA2 < 0.01), – CTRL-NDC (p NDC−SCA2 ≤ 0.01); – no significant dif-ference was found between SCA2-NDC (p SCA2−NDC = 0.622). Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 27
  28. 28. Correlation DT-E • Pearson and Spearman test reported correlation between E and DT for NDC patients (p < 0.05, ρ = 0.892, A), and correlation for SCA2 patients (p < 0.05, ρ = 0.736, B) not confirmed by Spearman (p = 0.18). No correlation was found for CTRL subjects (p = 0.43). Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 28
  29. 29. CONCLUSIONS Tools and Hypothesis Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 29
  30. 30. Summary • In the current work two methods have been developed: • Selective attention evaluation • Entropy analysis through wavelet decomposition. • Both methods are based on eye tracking • Subjects and patients cannot control eye movements or fixations perfectly, then, analysing eye motor entropy it is possible to extract some important features and conclusions. Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 30
  31. 31. Tool 1. Import Eye gaze data 2. Export Eye gaze data 3. Fixations recognition (Veneri, Piu, et al., 2010, 2011; Salvucci & Gold- berg, 2000) 4. Saccades recognition (Fischer et al., 1993) 5. TMT sequencing analysis 6. Transition Matrix analysis 7. ROI Analysis 8. Experiment segmentation Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 31
  32. 32. Study the influence • Does the motor control (cerebellum) influence selective attention? Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 32
  33. 33. Cerebellum could influence selective attention (Top-Down) sending afferent information of noise in order to minimize the functional cost of energy. Our hypothesis is systematically supported by recent application of opti-mal control theory; (Najemnik & Geisler, 2005), (Beers, 2007) and (Osborne, 2011) argued that humans’ vision is an optimal mechanism minimizing the effect of motor or cognitive noise. Our findings are compatible with this hypothesis: patients preferred sparser fixations avoiding saccade directed to the target. The non correlation of DN with WS suggested that this mechanism was a strategy to minimize the effort to control saccade rather than a direct influence on visual search. Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 33
  34. 34. THANKS Feature-Based Information Processing of Selective Attention through Entropy Analysis system Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 34
  35. 35. Model Energy Saccade length Giacomo Veneri – EVALab - Dep. Neurological and Behavioral Science - UNISI 35

×