Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The symbiotic relationship between neuroscience and machine learning

228 views

Published on

The goal of this tutorial is to provide a practical guide to application of machine learning to brain imaging data, including overview of the limitations of the data sources and required preprocessing, framed in a wider discussion relating to the close relationship between neuroscience and artificial intelligence research

Published in: Science
  • Be the first to comment

  • Be the first to like this

The symbiotic relationship between neuroscience and machine learning

  1. 1. The symbiotic relationship between neuroscience and machine learning Dr Emma C. Robinson @emrobSci https://emmarobinson01.com/ https://kclpure.kcl.ac.uk/portal/emma.robinson.html 1
  2. 2. Overview !2 1. How understanding of biological neural networks inspires design of artificial intelligence systems 2. The application of machine learning to improve understanding the human brain 3. Applying artificial neural networks to problems in neuroscience
  3. 3. Artificial Neurons !3 Biological neuron
  4. 4. Artificial Neurons !4 Biological neuron Artificial neuron: http://cs231n.github.io/neural-networks-1/
  5. 5. Artificial Neurons !5 http://cs231n.github.io/neural-networks-1/ Linear Classifier: E.g. f uses sigmoid function Where Squashes function to range 0->1 = 1 1 + ez <latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit> z = !0x0 + !1x1 + !2x2 + b<latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit> where
  6. 6. Artificial Neurons !5 http://cs231n.github.io/neural-networks-1/ Linear Classifier: E.g. f uses sigmoid function Where Squashes function to range 0->1 = 1 1 + ez <latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit><latexit sha1_base64="PVzq+gtkigSOdx15V3TmmjGK7Mk=">AAAB/nicbVBNS8NAEN3Ur1q/ooIXL4tFEISSiKAehKIXjxWMLTSxbLabdunuJuxuhBpz8K948aDi1d/hzX/jts1BWx8MPN6bYWZemDCqtON8W6W5+YXFpfJyZWV1bX3D3ty6VXEqMfFwzGLZCpEijAriaaoZaSWSIB4y0gwHlyO/eU+korG40cOEBBz1BI0oRtpIHXvHV7TH0bkfSYQzN8/cQ3L3kHfsqlNzxoCzxC1IFRRodOwvvxvjlBOhMUNKtV0n0UGGpKaYkbzip4okCA9Qj7QNFYgTFWTj+3O4b5QujGJpSmg4Vn9PZIgrNeSh6eRI99W0NxL/89qpjk6DjIok1UTgyaIoZVDHcBQG7FJJsGZDQxCW1NwKcR+ZJLSJrGJCcKdfniXeUe2s5l4fV+sXRRplsAv2wAFwwQmogyvQAB7A4BE8g1fwZj1ZL9a79TFpLVnFzDb4A+vzBxwjlbc=</latexit> z = !0x0 + !1x1 + !2x2 + b<latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit><latexit sha1_base64="crPtn9fuq+aJ/QiGd+Y58irm3j4=">AAACFnicbVBNS8MwGE79nPOr6tFLcAjCoLRDUA/C0IvHCdYNtlLSLN3C0qQkqTjL/oUX/4oXDypexZv/xmzrQTdfCDwf78ub94lSRpV23W9rYXFpeWW1tFZe39jc2rZ3dm+VyCQmPhZMyFaEFGGUE19TzUgrlQQlESPNaHA59pt3RCoq+I0epiRIUI/TmGKkjRTazsN5RySkh0IX3odutSCeIR6swoLWDK0ZGoV2xXXcScF54BWgAopqhPZXpytwlhCuMUNKtT031UGOpKaYkVG5kymSIjxAPdI2kKOEqCCf3DWCh0bpwlhI87iGE/X3RI4SpYZJZDoTpPtq1huL/3ntTMenQU55mmnC8XRRnDGoBRyHBLtUEqzZ0ACEJTV/hbiPJMLaRFk2IXizJ88Dv+acOd71caV+UaRRAvvgABwBD5yAOrgCDeADDB7BM3gFb9aT9WK9Wx/T1gWrmNkDf8r6/AHN0py8</latexit> where
  7. 7. Artificial Neurons !6 Linear Classifier: w1x1 + w2x2 + w0 = 0<latexit sha1_base64="M6eAlkt6VPk6r5STizNG9EQwvk8=">AAAB/HicbVDLSgMxFM34rPU1PnZugkUQhCEzWtouhKIblxUcW2iHIZOmbWjmQZKxraX4K25cqLj1Q9z5N6YPRK0HLvdwzr3k5gQJZ1Ih9GksLC4tr6xm1rLrG5tb2+bO7q2MU0GoS2Iei1qAJeUsoq5iitNaIigOA06rQfdy7FfvqJAsjm7UIKFeiNsRazGClZZ8c7/n233fPun5Tt93dEPnKOubOWTZp04hX4DIyjtFVHKgbaEJvkkOzFDxzY9GMyZpSCNFOJaybqNEeUMsFCOcjrKNVNIEky5u07qmEQ6p9IaT60fwSCtN2IqFrkjBifpzY4hDKQdhoCdDrDryrzcW//PqqWoVvSGLklTRiEwfaqUcqhiOo4BNJihRfKAJJoLpWyHpYIGJ0oGNQ5j78jxxHatk2ddnufLFLI0MOACH4BjYoADK4ApUgAsIuAeP4Bm8GA/Gk/FqvE1HF4zZzh74BeP9C19jk24=</latexit><latexit sha1_base64="M6eAlkt6VPk6r5STizNG9EQwvk8=">AAAB/HicbVDLSgMxFM34rPU1PnZugkUQhCEzWtouhKIblxUcW2iHIZOmbWjmQZKxraX4K25cqLj1Q9z5N6YPRK0HLvdwzr3k5gQJZ1Ih9GksLC4tr6xm1rLrG5tb2+bO7q2MU0GoS2Iei1qAJeUsoq5iitNaIigOA06rQfdy7FfvqJAsjm7UIKFeiNsRazGClZZ8c7/n233fPun5Tt93dEPnKOubOWTZp04hX4DIyjtFVHKgbaEJvkkOzFDxzY9GMyZpSCNFOJaybqNEeUMsFCOcjrKNVNIEky5u07qmEQ6p9IaT60fwSCtN2IqFrkjBifpzY4hDKQdhoCdDrDryrzcW//PqqWoVvSGLklTRiEwfaqUcqhiOo4BNJihRfKAJJoLpWyHpYIGJ0oGNQ5j78jxxHatk2ddnufLFLI0MOACH4BjYoADK4ApUgAsIuAeP4Bm8GA/Gk/FqvE1HF4zZzh74BeP9C19jk24=</latexit><latexit sha1_base64="M6eAlkt6VPk6r5STizNG9EQwvk8=">AAAB/HicbVDLSgMxFM34rPU1PnZugkUQhCEzWtouhKIblxUcW2iHIZOmbWjmQZKxraX4K25cqLj1Q9z5N6YPRK0HLvdwzr3k5gQJZ1Ih9GksLC4tr6xm1rLrG5tb2+bO7q2MU0GoS2Iei1qAJeUsoq5iitNaIigOA06rQfdy7FfvqJAsjm7UIKFeiNsRazGClZZ8c7/n233fPun5Tt93dEPnKOubOWTZp04hX4DIyjtFVHKgbaEJvkkOzFDxzY9GMyZpSCNFOJaybqNEeUMsFCOcjrKNVNIEky5u07qmEQ6p9IaT60fwSCtN2IqFrkjBifpzY4hDKQdhoCdDrDryrzcW//PqqWoVvSGLklTRiEwfaqUcqhiOo4BNJihRfKAJJoLpWyHpYIGJ0oGNQ5j78jxxHatk2ddnufLFLI0MOACH4BjYoADK4ApUgAsIuAeP4Bm8GA/Gk/FqvE1HF4zZzh74BeP9C19jk24=</latexit><latexit sha1_base64="M6eAlkt6VPk6r5STizNG9EQwvk8=">AAAB/HicbVDLSgMxFM34rPU1PnZugkUQhCEzWtouhKIblxUcW2iHIZOmbWjmQZKxraX4K25cqLj1Q9z5N6YPRK0HLvdwzr3k5gQJZ1Ih9GksLC4tr6xm1rLrG5tb2+bO7q2MU0GoS2Iei1qAJeUsoq5iitNaIigOA06rQfdy7FfvqJAsjm7UIKFeiNsRazGClZZ8c7/n233fPun5Tt93dEPnKOubOWTZp04hX4DIyjtFVHKgbaEJvkkOzFDxzY9GMyZpSCNFOJaybqNEeUMsFCOcjrKNVNIEky5u07qmEQ6p9IaT60fwSCtN2IqFrkjBifpzY4hDKQdhoCdDrDryrzcW//PqqWoVvSGLklTRiEwfaqUcqhiOo4BNJihRfKAJJoLpWyHpYIGJ0oGNQ5j78jxxHatk2ddnufLFLI0MOACH4BjYoADK4ApUgAsIuAeP4Bm8GA/Gk/FqvE1HF4zZzh74BeP9C19jk24=</latexit> + + + + + - - - - - - - w1x1 + w2x2 + w0 > 0<latexit sha1_base64="YSO/gynGP/X0WLp/5vtFOjEa/F0=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpgrqRohuXFYwttCFMppN26OTBzMS2luKvuHGh4tYPceffOGmz0NYDl3s4517mzvETzqSyrG9jYXFpeWW1sFZc39jc2jZ3du9lnApCHRLzWDR8LClnEXUUU5w2EkFx6HNa93vXmV9/oEKyOLpTw4S6Ie5ELGAEKy155n7fsweefdL3KgOvopt1aRU9s2SVrQnQPLFzUoIcNc/8arVjkoY0UoRjKZu2lSh3hIVihNNxsZVKmmDSwx3a1DTCIZXuaHL9GB1ppY2CWOiKFJqovzdGOJRyGPp6MsSqK2e9TPzPa6YqOHdHLEpSRSMyfShIOVIxyqJAbSYoUXyoCSaC6VsR6WKBidKBZSHYs1+eJ06lfFG2b09L1as8jQIcwCEcgw1nUIUbqIEDBB7hGV7hzXgyXox342M6umDkO3vwB8bnDxURkzs=</latexit><latexit sha1_base64="YSO/gynGP/X0WLp/5vtFOjEa/F0=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpgrqRohuXFYwttCFMppN26OTBzMS2luKvuHGh4tYPceffOGmz0NYDl3s4517mzvETzqSyrG9jYXFpeWW1sFZc39jc2jZ3du9lnApCHRLzWDR8LClnEXUUU5w2EkFx6HNa93vXmV9/oEKyOLpTw4S6Ie5ELGAEKy155n7fsweefdL3KgOvopt1aRU9s2SVrQnQPLFzUoIcNc/8arVjkoY0UoRjKZu2lSh3hIVihNNxsZVKmmDSwx3a1DTCIZXuaHL9GB1ppY2CWOiKFJqovzdGOJRyGPp6MsSqK2e9TPzPa6YqOHdHLEpSRSMyfShIOVIxyqJAbSYoUXyoCSaC6VsR6WKBidKBZSHYs1+eJ06lfFG2b09L1as8jQIcwCEcgw1nUIUbqIEDBB7hGV7hzXgyXox342M6umDkO3vwB8bnDxURkzs=</latexit><latexit sha1_base64="YSO/gynGP/X0WLp/5vtFOjEa/F0=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpgrqRohuXFYwttCFMppN26OTBzMS2luKvuHGh4tYPceffOGmz0NYDl3s4517mzvETzqSyrG9jYXFpeWW1sFZc39jc2jZ3du9lnApCHRLzWDR8LClnEXUUU5w2EkFx6HNa93vXmV9/oEKyOLpTw4S6Ie5ELGAEKy155n7fsweefdL3KgOvopt1aRU9s2SVrQnQPLFzUoIcNc/8arVjkoY0UoRjKZu2lSh3hIVihNNxsZVKmmDSwx3a1DTCIZXuaHL9GB1ppY2CWOiKFJqovzdGOJRyGPp6MsSqK2e9TPzPa6YqOHdHLEpSRSMyfShIOVIxyqJAbSYoUXyoCSaC6VsR6WKBidKBZSHYs1+eJ06lfFG2b09L1as8jQIcwCEcgw1nUIUbqIEDBB7hGV7hzXgyXox342M6umDkO3vwB8bnDxURkzs=</latexit><latexit sha1_base64="YSO/gynGP/X0WLp/5vtFOjEa/F0=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpgrqRohuXFYwttCFMppN26OTBzMS2luKvuHGh4tYPceffOGmz0NYDl3s4517mzvETzqSyrG9jYXFpeWW1sFZc39jc2jZ3du9lnApCHRLzWDR8LClnEXUUU5w2EkFx6HNa93vXmV9/oEKyOLpTw4S6Ie5ELGAEKy155n7fsweefdL3KgOvopt1aRU9s2SVrQnQPLFzUoIcNc/8arVjkoY0UoRjKZu2lSh3hIVihNNxsZVKmmDSwx3a1DTCIZXuaHL9GB1ppY2CWOiKFJqovzdGOJRyGPp6MsSqK2e9TPzPa6YqOHdHLEpSRSMyfShIOVIxyqJAbSYoUXyoCSaC6VsR6WKBidKBZSHYs1+eJ06lfFG2b09L1as8jQIcwCEcgw1nUIUbqIEDBB7hGV7hzXgyXox342M6umDkO3vwB8bnDxURkzs=</latexit> w1x1 + w2x2 + w0 < 0<latexit sha1_base64="8Kx3hYRWdh+HI8oqkpTUnhITmPs=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpggouim5cVjC20IYwmU7aoZMHMxPbWoq/4saFils/xJ1/46TNQlsPXO7hnHuZO8dPOJPKsr6NhcWl5ZXVwlpxfWNza9vc2b2XcSoIdUjMY9HwsaScRdRRTHHaSATFoc9p3e9dZ379gQrJ4uhODRPqhrgTsYARrLTkmft9zx549knfqwy8im7WpVX0zJJVtiZA88TOSQly1Dzzq9WOSRrSSBGOpWzaVqLcERaKEU7HxVYqaYJJD3doU9MIh1S6o8n1Y3SklTYKYqErUmii/t4Y4VDKYejryRCrrpz1MvE/r5mq4NwdsShJFY3I9KEg5UjFKIsCtZmgRPGhJpgIpm9FpIsFJkoHloVgz355njiV8kXZvj0tVa/yNApwAIdwDDacQRVuoAYOEHiEZ3iFN+PJeDHejY/p6IKR7+zBHxifPxIHkzk=</latexit><latexit sha1_base64="8Kx3hYRWdh+HI8oqkpTUnhITmPs=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpggouim5cVjC20IYwmU7aoZMHMxPbWoq/4saFils/xJ1/46TNQlsPXO7hnHuZO8dPOJPKsr6NhcWl5ZXVwlpxfWNza9vc2b2XcSoIdUjMY9HwsaScRdRRTHHaSATFoc9p3e9dZ379gQrJ4uhODRPqhrgTsYARrLTkmft9zx549knfqwy8im7WpVX0zJJVtiZA88TOSQly1Dzzq9WOSRrSSBGOpWzaVqLcERaKEU7HxVYqaYJJD3doU9MIh1S6o8n1Y3SklTYKYqErUmii/t4Y4VDKYejryRCrrpz1MvE/r5mq4NwdsShJFY3I9KEg5UjFKIsCtZmgRPGhJpgIpm9FpIsFJkoHloVgz355njiV8kXZvj0tVa/yNApwAIdwDDacQRVuoAYOEHiEZ3iFN+PJeDHejY/p6IKR7+zBHxifPxIHkzk=</latexit><latexit sha1_base64="8Kx3hYRWdh+HI8oqkpTUnhITmPs=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpggouim5cVjC20IYwmU7aoZMHMxPbWoq/4saFils/xJ1/46TNQlsPXO7hnHuZO8dPOJPKsr6NhcWl5ZXVwlpxfWNza9vc2b2XcSoIdUjMY9HwsaScRdRRTHHaSATFoc9p3e9dZ379gQrJ4uhODRPqhrgTsYARrLTkmft9zx549knfqwy8im7WpVX0zJJVtiZA88TOSQly1Dzzq9WOSRrSSBGOpWzaVqLcERaKEU7HxVYqaYJJD3doU9MIh1S6o8n1Y3SklTYKYqErUmii/t4Y4VDKYejryRCrrpz1MvE/r5mq4NwdsShJFY3I9KEg5UjFKIsCtZmgRPGhJpgIpm9FpIsFJkoHloVgz355njiV8kXZvj0tVa/yNApwAIdwDDacQRVuoAYOEHiEZ3iFN+PJeDHejY/p6IKR7+zBHxifPxIHkzk=</latexit><latexit sha1_base64="8Kx3hYRWdh+HI8oqkpTUnhITmPs=">AAAB/HicbVDLSsNAFL3xWesrPnZuBosgCCUpggouim5cVjC20IYwmU7aoZMHMxPbWoq/4saFils/xJ1/46TNQlsPXO7hnHuZO8dPOJPKsr6NhcWl5ZXVwlpxfWNza9vc2b2XcSoIdUjMY9HwsaScRdRRTHHaSATFoc9p3e9dZ379gQrJ4uhODRPqhrgTsYARrLTkmft9zx549knfqwy8im7WpVX0zJJVtiZA88TOSQly1Dzzq9WOSRrSSBGOpWzaVqLcERaKEU7HxVYqaYJJD3doU9MIh1S6o8n1Y3SklTYKYqErUmii/t4Y4VDKYejryRCrrpz1MvE/r5mq4NwdsShJFY3I9KEg5UjFKIsCtZmgRPGhJpgIpm9FpIsFJkoHloVgz355njiV8kXZvj0tVa/yNApwAIdwDDacQRVuoAYOEHiEZ3iFN+PJeDHejY/p6IKR7+zBHxifPxIHkzk=</latexit> Set: y=1 if f(x)>0 .5 y=0 if f(x)<0 .5 Therefore: y=1 if z> 0 y=0 if z<0 Optimise for w C = 1 n X x y ln( (z))+ (1 y) ln(1 (z))<latexit sha1_base64="cLaIz7Of9aZ2LEqmMCk2O7roAiw=">AAACKnicbVDLSgMxFM34rPVVdekmWIQWaZkRQV0Ipd24rGBtoVNKJs20oUlmSDLiOPR/3PgrLnShxa0fYtoOqK0HLvdwzr0k93gho0rb9thaWl5ZXVvPbGQ3t7Z3dnN7+3cqiCQmDRywQLY8pAijgjQ01Yy0QkkQ9xhpesPaxG/eE6loIG51HJIOR31BfYqRNlI3V61dlVxfIpw4o0SMXBXx7gOMocsELLiK9jkqPBaL8AS6bhYWnFJcNJbpP143l7fL9hRwkTgpyYMU9W7u1e0FOOJEaMyQUm3HDnUnQVJTzMgo60aKhAgPUZ+0DRWIE9VJpreO4LFRetAPpCmh4VT9vZEgrlTMPTPJkR6oeW8i/ue1I+1fdBIqwkgTgWcP+RGDOoCT4GCPSoI1iw1BWFLzV4gHyCSnTbxZE4Izf/IiaZyWL8vOzVm+Uk3TyIBDcAQKwAHnoAKuQR00AAZP4AW8gw/r2XqzxtbnbHTJSncOwB9YX9/0EKP5</latexit><latexit sha1_base64="cLaIz7Of9aZ2LEqmMCk2O7roAiw=">AAACKnicbVDLSgMxFM34rPVVdekmWIQWaZkRQV0Ipd24rGBtoVNKJs20oUlmSDLiOPR/3PgrLnShxa0fYtoOqK0HLvdwzr0k93gho0rb9thaWl5ZXVvPbGQ3t7Z3dnN7+3cqiCQmDRywQLY8pAijgjQ01Yy0QkkQ9xhpesPaxG/eE6loIG51HJIOR31BfYqRNlI3V61dlVxfIpw4o0SMXBXx7gOMocsELLiK9jkqPBaL8AS6bhYWnFJcNJbpP143l7fL9hRwkTgpyYMU9W7u1e0FOOJEaMyQUm3HDnUnQVJTzMgo60aKhAgPUZ+0DRWIE9VJpreO4LFRetAPpCmh4VT9vZEgrlTMPTPJkR6oeW8i/ue1I+1fdBIqwkgTgWcP+RGDOoCT4GCPSoI1iw1BWFLzV4gHyCSnTbxZE4Izf/IiaZyWL8vOzVm+Uk3TyIBDcAQKwAHnoAKuQR00AAZP4AW8gw/r2XqzxtbnbHTJSncOwB9YX9/0EKP5</latexit><latexit sha1_base64="cLaIz7Of9aZ2LEqmMCk2O7roAiw=">AAACKnicbVDLSgMxFM34rPVVdekmWIQWaZkRQV0Ipd24rGBtoVNKJs20oUlmSDLiOPR/3PgrLnShxa0fYtoOqK0HLvdwzr0k93gho0rb9thaWl5ZXVvPbGQ3t7Z3dnN7+3cqiCQmDRywQLY8pAijgjQ01Yy0QkkQ9xhpesPaxG/eE6loIG51HJIOR31BfYqRNlI3V61dlVxfIpw4o0SMXBXx7gOMocsELLiK9jkqPBaL8AS6bhYWnFJcNJbpP143l7fL9hRwkTgpyYMU9W7u1e0FOOJEaMyQUm3HDnUnQVJTzMgo60aKhAgPUZ+0DRWIE9VJpreO4LFRetAPpCmh4VT9vZEgrlTMPTPJkR6oeW8i/ue1I+1fdBIqwkgTgWcP+RGDOoCT4GCPSoI1iw1BWFLzV4gHyCSnTbxZE4Izf/IiaZyWL8vOzVm+Uk3TyIBDcAQKwAHnoAKuQR00AAZP4AW8gw/r2XqzxtbnbHTJSncOwB9YX9/0EKP5</latexit><latexit sha1_base64="cLaIz7Of9aZ2LEqmMCk2O7roAiw=">AAACKnicbVDLSgMxFM34rPVVdekmWIQWaZkRQV0Ipd24rGBtoVNKJs20oUlmSDLiOPR/3PgrLnShxa0fYtoOqK0HLvdwzr0k93gho0rb9thaWl5ZXVvPbGQ3t7Z3dnN7+3cqiCQmDRywQLY8pAijgjQ01Yy0QkkQ9xhpesPaxG/eE6loIG51HJIOR31BfYqRNlI3V61dlVxfIpw4o0SMXBXx7gOMocsELLiK9jkqPBaL8AS6bhYWnFJcNJbpP143l7fL9hRwkTgpyYMU9W7u1e0FOOJEaMyQUm3HDnUnQVJTzMgo60aKhAgPUZ+0DRWIE9VJpreO4LFRetAPpCmh4VT9vZEgrlTMPTPJkR6oeW8i/ue1I+1fdBIqwkgTgWcP+RGDOoCT4GCPSoI1iw1BWFLzV4gHyCSnTbxZE4Izf/IiaZyWL8vOzVm+Uk3TyIBDcAQKwAHnoAKuQR00AAZP4AW8gw/r2XqzxtbnbHTJSncOwB9YX9/0EKP5</latexit> 1 1 + ez <latexit sha1_base64="EHt0D5r5E17mwuH7HaOlapKR6bA=">AAAB+HicbVBNS8NAEN34WeNHox69LBZBEEoignorevFYwdhCG8tmO2mXbjZhdyPUkF/iSVDEq3/Fk//GbZuDtj4YeLw3w8y8MOVMadf9tpaWV1bX1isb9ubW9k7V2d27V0kmKfg04Ylsh0QBZwJ8zTSHdiqBxCGHVji6nvitR5CKJeJOj1MIYjIQLGKUaCP1nGo3koTmXpF7J/DwVPScmlt3p8CLxCtJDZVo9pyvbj+hWQxCU06U6nhuqoOcSM0oh8LuZgpSQkdkAB1DBYlBBfn08AIfGaWPo0SaEhpP1d8TOYmVGseh6YyJHqp5byL+53UyHV0EORNppkHQ2aIo41gneJIC7jMJVPOxIYRKZm7FdEhMEtpkZdsmBW/+50Xin9Yv697tWa1xVcZRQQfoEB0jD52jBrpBTeQjijL0jF7Rm5VbL9a79TFrXbLKmX30B9bnD2irktc=</latexit><latexit sha1_base64="EHt0D5r5E17mwuH7HaOlapKR6bA=">AAAB+HicbVBNS8NAEN34WeNHox69LBZBEEoignorevFYwdhCG8tmO2mXbjZhdyPUkF/iSVDEq3/Fk//GbZuDtj4YeLw3w8y8MOVMadf9tpaWV1bX1isb9ubW9k7V2d27V0kmKfg04Ylsh0QBZwJ8zTSHdiqBxCGHVji6nvitR5CKJeJOj1MIYjIQLGKUaCP1nGo3koTmXpF7J/DwVPScmlt3p8CLxCtJDZVo9pyvbj+hWQxCU06U6nhuqoOcSM0oh8LuZgpSQkdkAB1DBYlBBfn08AIfGaWPo0SaEhpP1d8TOYmVGseh6YyJHqp5byL+53UyHV0EORNppkHQ2aIo41gneJIC7jMJVPOxIYRKZm7FdEhMEtpkZdsmBW/+50Xin9Yv697tWa1xVcZRQQfoEB0jD52jBrpBTeQjijL0jF7Rm5VbL9a79TFrXbLKmX30B9bnD2irktc=</latexit><latexit sha1_base64="EHt0D5r5E17mwuH7HaOlapKR6bA=">AAAB+HicbVBNS8NAEN34WeNHox69LBZBEEoignorevFYwdhCG8tmO2mXbjZhdyPUkF/iSVDEq3/Fk//GbZuDtj4YeLw3w8y8MOVMadf9tpaWV1bX1isb9ubW9k7V2d27V0kmKfg04Ylsh0QBZwJ8zTSHdiqBxCGHVji6nvitR5CKJeJOj1MIYjIQLGKUaCP1nGo3koTmXpF7J/DwVPScmlt3p8CLxCtJDZVo9pyvbj+hWQxCU06U6nhuqoOcSM0oh8LuZgpSQkdkAB1DBYlBBfn08AIfGaWPo0SaEhpP1d8TOYmVGseh6YyJHqp5byL+53UyHV0EORNppkHQ2aIo41gneJIC7jMJVPOxIYRKZm7FdEhMEtpkZdsmBW/+50Xin9Yv697tWa1xVcZRQQfoEB0jD52jBrpBTeQjijL0jF7Rm5VbL9a79TFrXbLKmX30B9bnD2irktc=</latexit><latexit sha1_base64="EHt0D5r5E17mwuH7HaOlapKR6bA=">AAAB+HicbVBNS8NAEN34WeNHox69LBZBEEoignorevFYwdhCG8tmO2mXbjZhdyPUkF/iSVDEq3/Fk//GbZuDtj4YeLw3w8y8MOVMadf9tpaWV1bX1isb9ubW9k7V2d27V0kmKfg04Ylsh0QBZwJ8zTSHdiqBxCGHVji6nvitR5CKJeJOj1MIYjIQLGKUaCP1nGo3koTmXpF7J/DwVPScmlt3p8CLxCtJDZVo9pyvbj+hWQxCU06U6nhuqoOcSM0oh8LuZgpSQkdkAB1DBYlBBfn08AIfGaWPo0SaEhpP1d8TOYmVGseh6YyJHqp5byL+53UyHV0EORNppkHQ2aIo41gneJIC7jMJVPOxIYRKZm7FdEhMEtpkZdsmBW/+50Xin9Yv697tWa1xVcZRQQfoEB0jD52jBrpBTeQjijL0jF7Rm5VbL9a79TFrXbLKmX30B9bnD2irktc=</latexit>
  8. 8. Artificial Neural Networks !7 Neural networks with hidden layers can approximate complex non-linear functions x1<latexit sha1_base64="Y25RG+R1TcYqcShT+onWYYQeJcE=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF262YTdjVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zEZXmibw3oxSDmPYljzijxkp3T12vW6m6NXcKski8glShQKNb+er0EpbFKA0TVOu256YmyKkynAkclzuZxpSyIe1j21JJY9RBPj11TI6t0iNRomxJQ6bq74mcxlqP4tB2xtQM9Lw3Ef/z2pmJLoKcyzQzKNlsUZQJYhIy+Zv0uEJmxMgSyhS3txI2oIoyY9Mp2xC8+ZcXiX9au6x5t2fV+lWRRgkO4QhOwINzqMMNNMAHBn14hld4c4Tz4rw7H7PWJaeYOYA/cD5/AHl1jXQ=</latexit><latexit sha1_base64="Y25RG+R1TcYqcShT+onWYYQeJcE=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF262YTdjVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zEZXmibw3oxSDmPYljzijxkp3T12vW6m6NXcKski8glShQKNb+er0EpbFKA0TVOu256YmyKkynAkclzuZxpSyIe1j21JJY9RBPj11TI6t0iNRomxJQ6bq74mcxlqP4tB2xtQM9Lw3Ef/z2pmJLoKcyzQzKNlsUZQJYhIy+Zv0uEJmxMgSyhS3txI2oIoyY9Mp2xC8+ZcXiX9au6x5t2fV+lWRRgkO4QhOwINzqMMNNMAHBn14hld4c4Tz4rw7H7PWJaeYOYA/cD5/AHl1jXQ=</latexit><latexit sha1_base64="Y25RG+R1TcYqcShT+onWYYQeJcE=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF262YTdjVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zEZXmibw3oxSDmPYljzijxkp3T12vW6m6NXcKski8glShQKNb+er0EpbFKA0TVOu256YmyKkynAkclzuZxpSyIe1j21JJY9RBPj11TI6t0iNRomxJQ6bq74mcxlqP4tB2xtQM9Lw3Ef/z2pmJLoKcyzQzKNlsUZQJYhIy+Zv0uEJmxMgSyhS3txI2oIoyY9Mp2xC8+ZcXiX9au6x5t2fV+lWRRgkO4QhOwINzqMMNNMAHBn14hld4c4Tz4rw7H7PWJaeYOYA/cD5/AHl1jXQ=</latexit><latexit sha1_base64="Y25RG+R1TcYqcShT+onWYYQeJcE=">AAAB6XicbVBNS8NAEJ34WetX1aOXxSJ4KokI6q3oxWNFYwttKJvtpF262YTdjVhCf4IXDype/Ufe/Ddu2xy09cHA470ZZuaFqeDauO63s7S8srq2Xtoob25t7+xW9vYfdJIphj5LRKJaIdUouETfcCOwlSqkcSiwGQ6vJ37zEZXmibw3oxSDmPYljzijxkp3T12vW6m6NXcKski8glShQKNb+er0EpbFKA0TVOu256YmyKkynAkclzuZxpSyIe1j21JJY9RBPj11TI6t0iNRomxJQ6bq74mcxlqP4tB2xtQM9Lw3Ef/z2pmJLoKcyzQzKNlsUZQJYhIy+Zv0uEJmxMgSyhS3txI2oIoyY9Mp2xC8+ZcXiX9au6x5t2fV+lWRRgkO4QhOwINzqMMNNMAHBn14hld4c4Tz4rw7H7PWJaeYOYA/cD5/AHl1jXQ=</latexit> x2<latexit sha1_base64="xm50HuSe3nfFb2vfiO2F5KtqiVc=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN6KXjxWNLbQhrLZTtulm03Y3Ygl9Cd48aDi1X/kzX/jts1BWx8MPN6bYWZemAiujet+O4WV1bX1jeJmaWt7Z3evvH/woONUMfRZLGLVCqlGwSX6hhuBrUQhjUKBzXB0PfWbj6g0j+W9GScYRHQgeZ8zaqx099StdcsVt+rOQJaJl5MK5Gh0y1+dXszSCKVhgmrd9tzEBBlVhjOBk1In1ZhQNqIDbFsqaYQ6yGanTsiJVXqkHytb0pCZ+nsio5HW4yi0nRE1Q73oTcX/vHZq+hdBxmWSGpRsvqifCmJiMv2b9LhCZsTYEsoUt7cSNqSKMmPTKdkQvMWXl4lfq15WvduzSv0qT6MIR3AMp+DBOdThBhrgA4MBPMMrvDnCeXHenY95a8HJZw7hD5zPH3r4jXU=</latexit><latexit sha1_base64="xm50HuSe3nfFb2vfiO2F5KtqiVc=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN6KXjxWNLbQhrLZTtulm03Y3Ygl9Cd48aDi1X/kzX/jts1BWx8MPN6bYWZemAiujet+O4WV1bX1jeJmaWt7Z3evvH/woONUMfRZLGLVCqlGwSX6hhuBrUQhjUKBzXB0PfWbj6g0j+W9GScYRHQgeZ8zaqx099StdcsVt+rOQJaJl5MK5Gh0y1+dXszSCKVhgmrd9tzEBBlVhjOBk1In1ZhQNqIDbFsqaYQ6yGanTsiJVXqkHytb0pCZ+nsio5HW4yi0nRE1Q73oTcX/vHZq+hdBxmWSGpRsvqifCmJiMv2b9LhCZsTYEsoUt7cSNqSKMmPTKdkQvMWXl4lfq15WvduzSv0qT6MIR3AMp+DBOdThBhrgA4MBPMMrvDnCeXHenY95a8HJZw7hD5zPH3r4jXU=</latexit><latexit sha1_base64="xm50HuSe3nfFb2vfiO2F5KtqiVc=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN6KXjxWNLbQhrLZTtulm03Y3Ygl9Cd48aDi1X/kzX/jts1BWx8MPN6bYWZemAiujet+O4WV1bX1jeJmaWt7Z3evvH/woONUMfRZLGLVCqlGwSX6hhuBrUQhjUKBzXB0PfWbj6g0j+W9GScYRHQgeZ8zaqx099StdcsVt+rOQJaJl5MK5Gh0y1+dXszSCKVhgmrd9tzEBBlVhjOBk1In1ZhQNqIDbFsqaYQ6yGanTsiJVXqkHytb0pCZ+nsio5HW4yi0nRE1Q73oTcX/vHZq+hdBxmWSGpRsvqifCmJiMv2b9LhCZsTYEsoUt7cSNqSKMmPTKdkQvMWXl4lfq15WvduzSv0qT6MIR3AMp+DBOdThBhrgA4MBPMMrvDnCeXHenY95a8HJZw7hD5zPH3r4jXU=</latexit><latexit sha1_base64="xm50HuSe3nfFb2vfiO2F5KtqiVc=">AAAB6XicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN6KXjxWNLbQhrLZTtulm03Y3Ygl9Cd48aDi1X/kzX/jts1BWx8MPN6bYWZemAiujet+O4WV1bX1jeJmaWt7Z3evvH/woONUMfRZLGLVCqlGwSX6hhuBrUQhjUKBzXB0PfWbj6g0j+W9GScYRHQgeZ8zaqx099StdcsVt+rOQJaJl5MK5Gh0y1+dXszSCKVhgmrd9tzEBBlVhjOBk1In1ZhQNqIDbFsqaYQ6yGanTsiJVXqkHytb0pCZ+nsio5HW4yi0nRE1Q73oTcX/vHZq+hdBxmWSGpRsvqifCmJiMv2b9LhCZsTYEsoUt7cSNqSKMmPTKdkQvMWXl4lfq15WvduzSv0qT6MIR3AMp+DBOdThBhrgA4MBPMMrvDnCeXHenY95a8HJZw7hD5zPH3r4jXU=</latexit> ˆy<latexit sha1_base64="46p1wsZTyOWpxqoS9s+B3j/golU=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGFtoQ9lst+3SzSbsToQQ+iO8eFDx6v/x5r9x0+ag1QcDj/dmmJkXJlIYdN0vp7Kyura+Ud2sbW3v7O7V9w8eTJxqxn0Wy1h3Q2q4FIr7KFDybqI5jULJO+H0pvA7j1wbEat7zBIeRHSsxEgwilbq9CcUSVYb1Btu052D/CVeSRpQoj2of/aHMUsjrpBJakzPcxMMcqpRMMlntX5qeELZlI55z1JFI26CfH7ujJxYZUhGsbalkMzVnxM5jYzJotB2RhQnZtkrxP+8XoqjyyAXKkmRK7ZYNEolwZgUv5Oh0JyhzCyhTAt7K2ETqilDm1ARgrf88l/inzWvmt7deaN1XaZRhSM4hlPw4AJacAtt8IHBFJ7gBV6dxHl23pz3RWvFKWcO4Recj2/lUY7Q</latexit><latexit sha1_base64="46p1wsZTyOWpxqoS9s+B3j/golU=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGFtoQ9lst+3SzSbsToQQ+iO8eFDx6v/x5r9x0+ag1QcDj/dmmJkXJlIYdN0vp7Kyura+Ud2sbW3v7O7V9w8eTJxqxn0Wy1h3Q2q4FIr7KFDybqI5jULJO+H0pvA7j1wbEat7zBIeRHSsxEgwilbq9CcUSVYb1Btu052D/CVeSRpQoj2of/aHMUsjrpBJakzPcxMMcqpRMMlntX5qeELZlI55z1JFI26CfH7ujJxYZUhGsbalkMzVnxM5jYzJotB2RhQnZtkrxP+8XoqjyyAXKkmRK7ZYNEolwZgUv5Oh0JyhzCyhTAt7K2ETqilDm1ARgrf88l/inzWvmt7deaN1XaZRhSM4hlPw4AJacAtt8IHBFJ7gBV6dxHl23pz3RWvFKWcO4Recj2/lUY7Q</latexit><latexit sha1_base64="46p1wsZTyOWpxqoS9s+B3j/golU=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGFtoQ9lst+3SzSbsToQQ+iO8eFDx6v/x5r9x0+ag1QcDj/dmmJkXJlIYdN0vp7Kyura+Ud2sbW3v7O7V9w8eTJxqxn0Wy1h3Q2q4FIr7KFDybqI5jULJO+H0pvA7j1wbEat7zBIeRHSsxEgwilbq9CcUSVYb1Btu052D/CVeSRpQoj2of/aHMUsjrpBJakzPcxMMcqpRMMlntX5qeELZlI55z1JFI26CfH7ujJxYZUhGsbalkMzVnxM5jYzJotB2RhQnZtkrxP+8XoqjyyAXKkmRK7ZYNEolwZgUv5Oh0JyhzCyhTAt7K2ETqilDm1ARgrf88l/inzWvmt7deaN1XaZRhSM4hlPw4AJacAtt8IHBFJ7gBV6dxHl23pz3RWvFKWcO4Recj2/lUY7Q</latexit><latexit sha1_base64="46p1wsZTyOWpxqoS9s+B3j/golU=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGFtoQ9lst+3SzSbsToQQ+iO8eFDx6v/x5r9x0+ag1QcDj/dmmJkXJlIYdN0vp7Kyura+Ud2sbW3v7O7V9w8eTJxqxn0Wy1h3Q2q4FIr7KFDybqI5jULJO+H0pvA7j1wbEat7zBIeRHSsxEgwilbq9CcUSVYb1Btu052D/CVeSRpQoj2of/aHMUsjrpBJakzPcxMMcqpRMMlntX5qeELZlI55z1JFI26CfH7ujJxYZUhGsbalkMzVnxM5jYzJotB2RhQnZtkrxP+8XoqjyyAXKkmRK7ZYNEolwZgUv5Oh0JyhzCyhTAt7K2ETqilDm1ARgrf88l/inzWvmt7deaN1XaZRhSM4hlPw4AJacAtt8IHBFJ7gBV6dxHl23pz3RWvFKWcO4Recj2/lUY7Q</latexit> Layer 1 3 neurons Layer 2 2 neurons Multi Layer Perceptron (MLP)
  9. 9. Artificial Neural Networks !8 https://medium.com/@vivek.yadav/how- neural-networks-learn-nonlinear- functions-and-classify-linearly-non- separable-data-22328e7e5be1 • 3 hidden nodes • learn 3 linear decision boundaries • Combined by the output function Neural nets as universal approximators: http:// neuralnetworksanddeeplearning.com/ chap4.html Neural network playground (tensorflow): https://goo.gl/qAaR1i
  10. 10. Image classification !9 e.g. • Object recognition • Localisation • Semantic segmentation Artificial Neural Networks
  11. 11. Image classification !9 e.g. • Object recognition • Localisation • Semantic segmentation Artificial Neural Networks Training of fully connected networks on images would require prohibitive numbers of parameters
  12. 12. Convolutional neural networks reflect the brain’s visual processing system !10 Van Essen, David C., and Jack L. Gallant. "Neural mechanisms of form and motion processing in the primate visual system." Neuron 13.1 (1994): 1-10.
  13. 13. Convolutional neural networks reflect the brain’s visual processing system !11 Yamins, Daniel LK, and James J. DiCarlo. "Using goal-driven deep learning models to understand sensory cortex." Nature neuroscience 19.3 (2016): 356.
  14. 14. Convolutional network design !12 Convolutional Layer
  15. 15. Convolutional network design !12 Convolutional Layer • Learns filters (kernel weights w, size [h,w,d]) d=input layer depth i.e. 3 for RGB • Via convolutions strided across image • Output y= activation map Each output ‘neuron’ ‘sees’ only one location in space
  16. 16. Convolutional network design !12 Convolutional Layer • Learns filters (kernel weights w, size [h,w,d]) d=input layer depth i.e. 3 for RGB • Via convolutions strided across image • Output y= activation map Each output ‘neuron’ ‘sees’ only one location in space • Parameters are shared between neurons Learns consistent set of filters Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.
  17. 17. Convolutional network design !13 Layers (classification network) • INPUT • CONV • POOL/STRIDED CONVOLUTION • MLP (fully connected)
  18. 18. Convolutional network design !13 Layers (classification network) • INPUT • CONV • POOL/STRIDED CONVOLUTION • MLP (fully connected) Multiple blocks
  19. 19. Convolutional network design !13 Layers (classification network) • INPUT • CONV • POOL/STRIDED CONVOLUTION • MLP (fully connected) Multiple blocks
  20. 20. Convolutional networks mimics spatial smoothness observed for fMRI !14
  21. 21. Convolutional networks mimics spatial smoothness observed for fMRI !14 • Functional MRI is low spatial and temporal resolution Brain activation maps are coarse and smooth
  22. 22. Convolutional networks mimics spatial smoothness observed for fMRI !14 • Functional MRI is low spatial and temporal resolution • But still can decode visual representations Brain activation maps are coarse and smooth Nishimoto, Shinji, et al. "Reconstructing visual experiences from brain activity evoked by natural movies." Current Biology21.19 (2011): 1641-1646.
  23. 23. Convolutional networks mimics spatial smoothness observed for fMRI !14 • Functional MRI is low spatial and temporal resolution • But still can decode visual representations • Thus neural code must be coarse & functionally smooth (at some level) Brain activation maps are coarse and smooth Nishimoto, Shinji, et al. "Reconstructing visual experiences from brain activity evoked by natural movies." Current Biology21.19 (2011): 1641-1646.
  24. 24. Convolutional networks mimics spatial smoothness observed for fMRI !14 • Functional MRI is low spatial and temporal resolution • But still can decode visual representations • Thus neural code must be coarse & functionally smooth (at some level) • This also is true of convolutional networks Guest, Olivia, and Bradley C. Love. "What the success of brain imaging implies about the neural code." Elife 6 (2017). Brain activation maps are coarse and smooth Network representations of stimulus similarity fall away with addition of distortion as expected
  25. 25. Enhancing AI through even more sophisticated models of neurological processes !15 Models how biological vision • Factorised representation of shape and appearance • Has lateral connections • Feedforward & feedback connections Within loopy belief propagation inference model Recursive Cortical Network George, Dileep, et al. "A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs." Science 358.6368 (2017): eaag2612.
  26. 26. Enhancing AI through even more sophisticated models of neurological processes !15 Models how biological vision • Factorised representation of shape and appearance • Has lateral connections • Feedforward & feedback connections Within loopy belief propagation inference model Recursive Cortical Network George, Dileep, et al. "A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs." Science 358.6368 (2017): eaag2612. Generalises better to complex object recognition tasks e.g. captcha recognition
  27. 27. !16 A. Attentional mechanisms (glimpses B. Artificial hippocampi for Deep RL algorithms C. LSTMs build from working memory systems D. Brain plasticity modelled by elastic weight consolidation (EWC) algorithms Enhancing AI through even more sophisticated models of neurological processes Hassabis, Demis, et al. "Neuroscience-inspired artificial intelligence."  Neuron 95.2 (2017): 245-258.
  28. 28. The limits of current AI relative to the capabilities of the human brain !17 The human brain is a vastly complex system • Multiple forward and feedback systems • Parallel processing of multiple different interacting systems Roelfsema, Pieter R., and Anthony Holtmaat. "Control of synaptic plasticity in deep cortical networks." Nature Reviews Neuroscience 19.3 (2018): 166.
  29. 29. The limits of understanding of the human brain !18 • 100 billion nerve cells; 100 trillion connections • Vast scales involved make comprehensive making of even a single human brain impossible
  30. 30. Studying the brain over multiple scales !19 2. Coarse scale models of the whole brain: • Derived from MRI imaging data Full brain coverage In vivo Non invasive Multiple properties: Morphology Function Microstructure
  31. 31. Studying the brain over multiple scales !19 2. Coarse scale models of the whole brain: • Derived from MRI imaging data Full brain coverage In vivo Non invasive Multiple properties: Morphology Function Microstructure
  32. 32. 1. Data Limitations The Quest for a Macroscale Model of Human Brain Networks !20 • Low spatial resolution • Voxel sizes 1mm^3 or 2mm^3 • Low Temporal resolution • fMRI measures ‘haemodynamic response’ function (HRF) • Flow of blood to supply nutrients to neutrons for firing • Indirect and approximate HRF
  33. 33. Macroscale network models !21 Brain network model: • Functionally specialised regions • Connected through axon bundles • Complex brain functions result from coordinated actions of multiple regions
  34. 34. !22 i. Reconstruction ii. Denoising/artifact correction iii. Image registration iv. Modelling of the signal 2. Processing The Quest for a Macroscale Model of Human Brain Networks
  35. 35. !22 i. Reconstruction ii. Denoising/artifact correction iii. Image registration iv. Modelling of the signal 2. Processing The Quest for a Macroscale Model of Human Brain Networks
  36. 36. ML models for fMRI denoising !23 The fMRI signal
  37. 37. ML models for fMRI denoising !24 Sources of noise • Scanner Artifacts • Head Motion • Physiological (breathing/cardiac pulsation)
  38. 38. ML models for fMRI denoising !24 Sources of noise • Scanner Artifacts • Head Motion • Physiological (breathing/cardiac pulsation) Corrected separately
  39. 39. ML models for fMRI denoising !25 • Use Independent Component Analysis (ICA) • Decompose data (X) into a reduced set of spatial maps and temporal components • Train classifier to distinguish true signal from noise • Regress noise components from X
  40. 40. ML models for fMRI denoising !26 • ICA assumes linear model of the brain • Optimises to find matrix A-1 that un-mixes independent sources S (Spatial ICA) • Assumption is valid data is spatially sparse X(t, v) = LX l Al(t)Sl(v) <latexit sha1_base64="xxgf4TsL5DIM1cU20XqUhL3G4J0=">AAACBHicbVC7SgNBFJ2NrxhfUUstBoOQgIRdEdRCiNpYWER0TSBZl9nJbDJk9sHM3UAIaWz8FRsLFVs/ws6/cZJsoYkH7uVwzr3M3OPFgiswzW8jMze/sLiUXc6trK6tb+Q3t+5VlEjKbBqJSNY9opjgIbOBg2D1WDISeILVvO7lyK/1mFQ8Cu+gHzMnIO2Q+5wS0JKb360X4aBXOmuqJHDFwzU+d0URSre690puvmCWzTHwLLFSUkApqm7+q9mKaBKwEKggSjUsMwZnQCRwKtgw10wUiwntkjZraBqSgClnML5iiPe10sJ+JHWFgMfq740BCZTqB56eDAh01LQ3Ev/zGgn4J86Ah3ECLKSTh/xEYIjwKBLc4pJREH1NCJVc/xXTDpGEgg4up0Owpk+eJfZh+bRs3RwVKhdpGlm0g/ZQEVnoGFXQFaoiG1H0iJ7RK3oznowX4934mIxmjHRnG/2B8fkDCbqWjw==</latexit><latexit sha1_base64="xxgf4TsL5DIM1cU20XqUhL3G4J0=">AAACBHicbVC7SgNBFJ2NrxhfUUstBoOQgIRdEdRCiNpYWER0TSBZl9nJbDJk9sHM3UAIaWz8FRsLFVs/ws6/cZJsoYkH7uVwzr3M3OPFgiswzW8jMze/sLiUXc6trK6tb+Q3t+5VlEjKbBqJSNY9opjgIbOBg2D1WDISeILVvO7lyK/1mFQ8Cu+gHzMnIO2Q+5wS0JKb360X4aBXOmuqJHDFwzU+d0URSre690puvmCWzTHwLLFSUkApqm7+q9mKaBKwEKggSjUsMwZnQCRwKtgw10wUiwntkjZraBqSgClnML5iiPe10sJ+JHWFgMfq740BCZTqB56eDAh01LQ3Ev/zGgn4J86Ah3ECLKSTh/xEYIjwKBLc4pJREH1NCJVc/xXTDpGEgg4up0Owpk+eJfZh+bRs3RwVKhdpGlm0g/ZQEVnoGFXQFaoiG1H0iJ7RK3oznowX4934mIxmjHRnG/2B8fkDCbqWjw==</latexit><latexit sha1_base64="xxgf4TsL5DIM1cU20XqUhL3G4J0=">AAACBHicbVC7SgNBFJ2NrxhfUUstBoOQgIRdEdRCiNpYWER0TSBZl9nJbDJk9sHM3UAIaWz8FRsLFVs/ws6/cZJsoYkH7uVwzr3M3OPFgiswzW8jMze/sLiUXc6trK6tb+Q3t+5VlEjKbBqJSNY9opjgIbOBg2D1WDISeILVvO7lyK/1mFQ8Cu+gHzMnIO2Q+5wS0JKb360X4aBXOmuqJHDFwzU+d0URSre690puvmCWzTHwLLFSUkApqm7+q9mKaBKwEKggSjUsMwZnQCRwKtgw10wUiwntkjZraBqSgClnML5iiPe10sJ+JHWFgMfq740BCZTqB56eDAh01LQ3Ev/zGgn4J86Ah3ECLKSTh/xEYIjwKBLc4pJREH1NCJVc/xXTDpGEgg4up0Owpk+eJfZh+bRs3RwVKhdpGlm0g/ZQEVnoGFXQFaoiG1H0iJ7RK3oznowX4934mIxmjHRnG/2B8fkDCbqWjw==</latexit><latexit sha1_base64="xxgf4TsL5DIM1cU20XqUhL3G4J0=">AAACBHicbVC7SgNBFJ2NrxhfUUstBoOQgIRdEdRCiNpYWER0TSBZl9nJbDJk9sHM3UAIaWz8FRsLFVs/ws6/cZJsoYkH7uVwzr3M3OPFgiswzW8jMze/sLiUXc6trK6tb+Q3t+5VlEjKbBqJSNY9opjgIbOBg2D1WDISeILVvO7lyK/1mFQ8Cu+gHzMnIO2Q+5wS0JKb360X4aBXOmuqJHDFwzU+d0URSre690puvmCWzTHwLLFSUkApqm7+q9mKaBKwEKggSjUsMwZnQCRwKtgw10wUiwntkjZraBqSgClnML5iiPe10sJ+JHWFgMfq740BCZTqB56eDAh01LQ3Ev/zGgn4J86Ah3ECLKSTh/xEYIjwKBLc4pJREH1NCJVc/xXTDpGEgg4up0Owpk+eJfZh+bRs3RwVKhdpGlm0g/ZQEVnoGFXQFaoiG1H0iJ7RK3oznowX4934mIxmjHRnG/2B8fkDCbqWjw==</latexit> S = A 1 X = WX<latexit sha1_base64="hdBfqgKQHcwT9cShfRXFwaIL0xM=">AAACEHicbZDNSsNAFIUn/tb6F3XpZrAI3VgSEdRFoerGZUVjA20sk+mkHTqZhJmJUEJewY2v4saFiluX7nwbJ20EbT0w8HHuvcy9x48Zlcqyvoy5+YXFpeXSSnl1bX1j09zavpVRIjBxcMQi4fpIEkY5cRRVjLixICj0GWn5w4u83ronQtKI36hRTLwQ9TkNKEZKW12z2gmRGvgBvK4XlJ7dpQd25mZ1+OO03KzcNStWzRoLzoJdQAUUanbNz04vwklIuMIMSdm2rVh5KRKKYkaycieRJEZ4iPqkrZGjkEgvHV+UwX3t9GAQCf24gmP390SKQilHoa878x3ldC03/6u1ExWceCnlcaIIx5OPgoRBFcE8HtijgmDFRhoQFlTvCvEACYSVDjEPwZ4+eRacw9ppzb46qjTOizRKYBfsgSqwwTFogEvQBA7A4AE8gRfwajwaz8ab8T5pnTOKmR3wR8bHN6tInHw=</latexit><latexit sha1_base64="hdBfqgKQHcwT9cShfRXFwaIL0xM=">AAACEHicbZDNSsNAFIUn/tb6F3XpZrAI3VgSEdRFoerGZUVjA20sk+mkHTqZhJmJUEJewY2v4saFiluX7nwbJ20EbT0w8HHuvcy9x48Zlcqyvoy5+YXFpeXSSnl1bX1j09zavpVRIjBxcMQi4fpIEkY5cRRVjLixICj0GWn5w4u83ronQtKI36hRTLwQ9TkNKEZKW12z2gmRGvgBvK4XlJ7dpQd25mZ1+OO03KzcNStWzRoLzoJdQAUUanbNz04vwklIuMIMSdm2rVh5KRKKYkaycieRJEZ4iPqkrZGjkEgvHV+UwX3t9GAQCf24gmP390SKQilHoa878x3ldC03/6u1ExWceCnlcaIIx5OPgoRBFcE8HtijgmDFRhoQFlTvCvEACYSVDjEPwZ4+eRacw9ppzb46qjTOizRKYBfsgSqwwTFogEvQBA7A4AE8gRfwajwaz8ab8T5pnTOKmR3wR8bHN6tInHw=</latexit><latexit sha1_base64="hdBfqgKQHcwT9cShfRXFwaIL0xM=">AAACEHicbZDNSsNAFIUn/tb6F3XpZrAI3VgSEdRFoerGZUVjA20sk+mkHTqZhJmJUEJewY2v4saFiluX7nwbJ20EbT0w8HHuvcy9x48Zlcqyvoy5+YXFpeXSSnl1bX1j09zavpVRIjBxcMQi4fpIEkY5cRRVjLixICj0GWn5w4u83ronQtKI36hRTLwQ9TkNKEZKW12z2gmRGvgBvK4XlJ7dpQd25mZ1+OO03KzcNStWzRoLzoJdQAUUanbNz04vwklIuMIMSdm2rVh5KRKKYkaycieRJEZ4iPqkrZGjkEgvHV+UwX3t9GAQCf24gmP390SKQilHoa878x3ldC03/6u1ExWceCnlcaIIx5OPgoRBFcE8HtijgmDFRhoQFlTvCvEACYSVDjEPwZ4+eRacw9ppzb46qjTOizRKYBfsgSqwwTFogEvQBA7A4AE8gRfwajwaz8ab8T5pnTOKmR3wR8bHN6tInHw=</latexit><latexit sha1_base64="hdBfqgKQHcwT9cShfRXFwaIL0xM=">AAACEHicbZDNSsNAFIUn/tb6F3XpZrAI3VgSEdRFoerGZUVjA20sk+mkHTqZhJmJUEJewY2v4saFiluX7nwbJ20EbT0w8HHuvcy9x48Zlcqyvoy5+YXFpeXSSnl1bX1j09zavpVRIjBxcMQi4fpIEkY5cRRVjLixICj0GWn5w4u83ronQtKI36hRTLwQ9TkNKEZKW12z2gmRGvgBvK4XlJ7dpQd25mZ1+OO03KzcNStWzRoLzoJdQAUUanbNz04vwklIuMIMSdm2rVh5KRKKYkaycieRJEZ4iPqkrZGjkEgvHV+UwX3t9GAQCf24gmP390SKQilHoa878x3ldC03/6u1ExWceCnlcaIIx5OPgoRBFcE8HtijgmDFRhoQFlTvCvEACYSVDjEPwZ4+eRacw9ppzb46qjTOizRKYBfsgSqwwTFogEvQBA7A4AE8gRfwajwaz8ab8T5pnTOKmR3wR8bHN6tInHw=</latexit>
  41. 41. ML models for fMRI denoising !27 • Achieved by optimising for non Gaussianty e.g. FastICA Hyvärinen 1999 Two stages: Pre-whitening Iterative component extraction
  42. 42. ML models for fMRI denoising !28 • Centre (demean) • Whiten • Transform X to s.t • Performed using eigenvalue decomposition of covariance matrix (PCA) Pre-whitening xij xij 1 M X j0 xij <latexit sha1_base64="U/3HNgp1UEJGXQJLcK7I/uPYa+8=">AAACHXicbVDLSsNAFJ34rPUVdelmsIhuLIko6q7oxo1QwdhCE8JkOmmnnTyYmahlyJe48VfcuFBx4Ub8G6dtFtp64MLhnHu5954gZVRIy/o2Zmbn5hcWS0vl5ZXVtXVzY/NWJBnHxMEJS3gzQIIwGhNHUslIM+UERQEjjaB/MfQbd4QLmsQ3cpASL0KdmIYUI6kl3zx+8BXt5dBlJJSI8+QeFsqBG3KElZ2rq9wVWeSr3l5emL5ZsarWCHCa2AWpgAJ13/x02wnOIhJLzJAQLdtKpacQlxQzkpfdTJAU4T7qkJamMYqI8NTovRzuaqUNw4TriiUcqb8nFIqEGESB7oyQ7IpJbyj+57UyGZ56isZpJkmMx4vCjEGZwGFWsE05wZINNEGYU30rxF2kU5E60bIOwZ58eZo4h9Wzqn19VKmdF2mUwDbYAfvABiegBi5BHTgAg0fwDF7Bm/FkvBjvxse4dcYoZrbAHxhfPyZ/o1U=</latexit><latexit sha1_base64="U/3HNgp1UEJGXQJLcK7I/uPYa+8=">AAACHXicbVDLSsNAFJ34rPUVdelmsIhuLIko6q7oxo1QwdhCE8JkOmmnnTyYmahlyJe48VfcuFBx4Ub8G6dtFtp64MLhnHu5954gZVRIy/o2Zmbn5hcWS0vl5ZXVtXVzY/NWJBnHxMEJS3gzQIIwGhNHUslIM+UERQEjjaB/MfQbd4QLmsQ3cpASL0KdmIYUI6kl3zx+8BXt5dBlJJSI8+QeFsqBG3KElZ2rq9wVWeSr3l5emL5ZsarWCHCa2AWpgAJ13/x02wnOIhJLzJAQLdtKpacQlxQzkpfdTJAU4T7qkJamMYqI8NTovRzuaqUNw4TriiUcqb8nFIqEGESB7oyQ7IpJbyj+57UyGZ56isZpJkmMx4vCjEGZwGFWsE05wZINNEGYU30rxF2kU5E60bIOwZ58eZo4h9Wzqn19VKmdF2mUwDbYAfvABiegBi5BHTgAg0fwDF7Bm/FkvBjvxse4dcYoZrbAHxhfPyZ/o1U=</latexit><latexit sha1_base64="U/3HNgp1UEJGXQJLcK7I/uPYa+8=">AAACHXicbVDLSsNAFJ34rPUVdelmsIhuLIko6q7oxo1QwdhCE8JkOmmnnTyYmahlyJe48VfcuFBx4Ub8G6dtFtp64MLhnHu5954gZVRIy/o2Zmbn5hcWS0vl5ZXVtXVzY/NWJBnHxMEJS3gzQIIwGhNHUslIM+UERQEjjaB/MfQbd4QLmsQ3cpASL0KdmIYUI6kl3zx+8BXt5dBlJJSI8+QeFsqBG3KElZ2rq9wVWeSr3l5emL5ZsarWCHCa2AWpgAJ13/x02wnOIhJLzJAQLdtKpacQlxQzkpfdTJAU4T7qkJamMYqI8NTovRzuaqUNw4TriiUcqb8nFIqEGESB7oyQ7IpJbyj+57UyGZ56isZpJkmMx4vCjEGZwGFWsE05wZINNEGYU30rxF2kU5E60bIOwZ58eZo4h9Wzqn19VKmdF2mUwDbYAfvABiegBi5BHTgAg0fwDF7Bm/FkvBjvxse4dcYoZrbAHxhfPyZ/o1U=</latexit><latexit sha1_base64="U/3HNgp1UEJGXQJLcK7I/uPYa+8=">AAACHXicbVDLSsNAFJ34rPUVdelmsIhuLIko6q7oxo1QwdhCE8JkOmmnnTyYmahlyJe48VfcuFBx4Ub8G6dtFtp64MLhnHu5954gZVRIy/o2Zmbn5hcWS0vl5ZXVtXVzY/NWJBnHxMEJS3gzQIIwGhNHUslIM+UERQEjjaB/MfQbd4QLmsQ3cpASL0KdmIYUI6kl3zx+8BXt5dBlJJSI8+QeFsqBG3KElZ2rq9wVWeSr3l5emL5ZsarWCHCa2AWpgAJ13/x02wnOIhJLzJAQLdtKpacQlxQzkpfdTJAU4T7qkJamMYqI8NTovRzuaqUNw4TriiUcqb8nFIqEGESB7oyQ7IpJbyj+57UyGZ56isZpJkmMx4vCjEGZwGFWsE05wZINNEGYU30rxF2kU5E60bIOwZ58eZo4h9Wzqn19VKmdF2mUwDbYAfvABiegBi5BHTgAg0fwDF7Bm/FkvBjvxse4dcYoZrbAHxhfPyZ/o1U=</latexit> Y = WWX<latexit sha1_base64="0LhMhzBZ47SDxhG+nUZ0f7ZYIz8=">AAAB+HicbVBNS8NAFHypX7V+RT16WSyCp5KIoB6EohePFYyttCFstpt26WYTdjeFEvpPvHhQ8epP8ea/cdPmoK0DC8PMe7zZCVPOlHacb6uysrq2vlHdrG1t7+zu2fsHjyrJJKEeSXgiOyFWlDNBPc00p51UUhyHnLbD0W3ht8dUKpaIBz1JqR/jgWARI1gbKbDtXoz1MIzyp+t20O5Ma4FddxrODGiZuCWpQ4lWYH/1+gnJYio04Vipruuk2s+x1IxwOq31MkVTTEZ4QLuGChxT5eez5FN0YpQ+ihJpntBopv7eyHGs1CQOzWSRUy16hfif1810dOnnTKSZpoLMD0UZRzpBRQ2ozyQlmk8MwUQykxWRIZaYaFNWUYK7+OVl4p01rhru/Xm9eVO2UYUjOIZTcOECmnAHLfCAwBie4RXerNx6sd6tj/loxSp3DuEPrM8fMkCS6g==</latexit><latexit sha1_base64="0LhMhzBZ47SDxhG+nUZ0f7ZYIz8=">AAAB+HicbVBNS8NAFHypX7V+RT16WSyCp5KIoB6EohePFYyttCFstpt26WYTdjeFEvpPvHhQ8epP8ea/cdPmoK0DC8PMe7zZCVPOlHacb6uysrq2vlHdrG1t7+zu2fsHjyrJJKEeSXgiOyFWlDNBPc00p51UUhyHnLbD0W3ht8dUKpaIBz1JqR/jgWARI1gbKbDtXoz1MIzyp+t20O5Ma4FddxrODGiZuCWpQ4lWYH/1+gnJYio04Vipruuk2s+x1IxwOq31MkVTTEZ4QLuGChxT5eez5FN0YpQ+ihJpntBopv7eyHGs1CQOzWSRUy16hfif1810dOnnTKSZpoLMD0UZRzpBRQ2ozyQlmk8MwUQykxWRIZaYaFNWUYK7+OVl4p01rhru/Xm9eVO2UYUjOIZTcOECmnAHLfCAwBie4RXerNx6sd6tj/loxSp3DuEPrM8fMkCS6g==</latexit><latexit sha1_base64="0LhMhzBZ47SDxhG+nUZ0f7ZYIz8=">AAAB+HicbVBNS8NAFHypX7V+RT16WSyCp5KIoB6EohePFYyttCFstpt26WYTdjeFEvpPvHhQ8epP8ea/cdPmoK0DC8PMe7zZCVPOlHacb6uysrq2vlHdrG1t7+zu2fsHjyrJJKEeSXgiOyFWlDNBPc00p51UUhyHnLbD0W3ht8dUKpaIBz1JqR/jgWARI1gbKbDtXoz1MIzyp+t20O5Ma4FddxrODGiZuCWpQ4lWYH/1+gnJYio04Vipruuk2s+x1IxwOq31MkVTTEZ4QLuGChxT5eez5FN0YpQ+ihJpntBopv7eyHGs1CQOzWSRUy16hfif1810dOnnTKSZpoLMD0UZRzpBRQ2ozyQlmk8MwUQykxWRIZaYaFNWUYK7+OVl4p01rhru/Xm9eVO2UYUjOIZTcOECmnAHLfCAwBie4RXerNx6sd6tj/loxSp3DuEPrM8fMkCS6g==</latexit><latexit sha1_base64="0LhMhzBZ47SDxhG+nUZ0f7ZYIz8=">AAAB+HicbVBNS8NAFHypX7V+RT16WSyCp5KIoB6EohePFYyttCFstpt26WYTdjeFEvpPvHhQ8epP8ea/cdPmoK0DC8PMe7zZCVPOlHacb6uysrq2vlHdrG1t7+zu2fsHjyrJJKEeSXgiOyFWlDNBPc00p51UUhyHnLbD0W3ht8dUKpaIBz1JqR/jgWARI1gbKbDtXoz1MIzyp+t20O5Ma4FddxrODGiZuCWpQ4lWYH/1+gnJYio04Vipruuk2s+x1IxwOq31MkVTTEZ4QLuGChxT5eez5FN0YpQ+ihJpntBopv7eyHGs1CQOzWSRUy16hfif1810dOnnTKSZpoLMD0UZRzpBRQ2ozyQlmk8MwUQykxWRIZaYaFNWUYK7+OVl4p01rhru/Xm9eVO2UYUjOIZTcOECmnAHLfCAwBie4RXerNx6sd6tj/loxSp3DuEPrM8fMkCS6g==</latexit> Cov(Y) = I<latexit sha1_base64="nF8UTxFTGSjaXUfDM+2Psmjxn1E=">AAACA3icbZDNSsNAFIUn9a/Wv6jLbgaLUDclEUFdCMVudFfB2EobymQ6aYdOJmFmUighCze+ihsXKm59CXe+jZM2grYeGPg4917m3uNFjEplWV9GYWl5ZXWtuF7a2Nza3jF39+5kGAtMHByyULQ9JAmjnDiKKkbakSAo8BhpeaNGVm+NiZA05LdqEhE3QANOfYqR0lbPLDfCcbUbIDX0/OQ+Pbr44eu01DMrVs2aCi6CnUMF5Gr2zM9uP8RxQLjCDEnZsa1IuQkSimJG0lI3liRCeIQGpKORo4BIN5kekcJD7fShHwr9uIJT9/dEggIpJ4GnO7MV5XwtM/+rdWLln7kJ5VGsCMezj/yYQRXCLBHYp4JgxSYaEBZU7wrxEAmElc4tC8GeP3kRnOPaec2+OanUL/M0iqAMDkAV2OAU1MEVaAIHYPAAnsALeDUejWfjzXiftRaMfGYf/JHx8Q1gUpd7</latexit><latexit sha1_base64="nF8UTxFTGSjaXUfDM+2Psmjxn1E=">AAACA3icbZDNSsNAFIUn9a/Wv6jLbgaLUDclEUFdCMVudFfB2EobymQ6aYdOJmFmUighCze+ihsXKm59CXe+jZM2grYeGPg4917m3uNFjEplWV9GYWl5ZXWtuF7a2Nza3jF39+5kGAtMHByyULQ9JAmjnDiKKkbakSAo8BhpeaNGVm+NiZA05LdqEhE3QANOfYqR0lbPLDfCcbUbIDX0/OQ+Pbr44eu01DMrVs2aCi6CnUMF5Gr2zM9uP8RxQLjCDEnZsa1IuQkSimJG0lI3liRCeIQGpKORo4BIN5kekcJD7fShHwr9uIJT9/dEggIpJ4GnO7MV5XwtM/+rdWLln7kJ5VGsCMezj/yYQRXCLBHYp4JgxSYaEBZU7wrxEAmElc4tC8GeP3kRnOPaec2+OanUL/M0iqAMDkAV2OAU1MEVaAIHYPAAnsALeDUejWfjzXiftRaMfGYf/JHx8Q1gUpd7</latexit><latexit sha1_base64="nF8UTxFTGSjaXUfDM+2Psmjxn1E=">AAACA3icbZDNSsNAFIUn9a/Wv6jLbgaLUDclEUFdCMVudFfB2EobymQ6aYdOJmFmUighCze+ihsXKm59CXe+jZM2grYeGPg4917m3uNFjEplWV9GYWl5ZXWtuF7a2Nza3jF39+5kGAtMHByyULQ9JAmjnDiKKkbakSAo8BhpeaNGVm+NiZA05LdqEhE3QANOfYqR0lbPLDfCcbUbIDX0/OQ+Pbr44eu01DMrVs2aCi6CnUMF5Gr2zM9uP8RxQLjCDEnZsa1IuQkSimJG0lI3liRCeIQGpKORo4BIN5kekcJD7fShHwr9uIJT9/dEggIpJ4GnO7MV5XwtM/+rdWLln7kJ5VGsCMezj/yYQRXCLBHYp4JgxSYaEBZU7wrxEAmElc4tC8GeP3kRnOPaec2+OanUL/M0iqAMDkAV2OAU1MEVaAIHYPAAnsALeDUejWfjzXiftRaMfGYf/JHx8Q1gUpd7</latexit><latexit sha1_base64="nF8UTxFTGSjaXUfDM+2Psmjxn1E=">AAACA3icbZDNSsNAFIUn9a/Wv6jLbgaLUDclEUFdCMVudFfB2EobymQ6aYdOJmFmUighCze+ihsXKm59CXe+jZM2grYeGPg4917m3uNFjEplWV9GYWl5ZXWtuF7a2Nza3jF39+5kGAtMHByyULQ9JAmjnDiKKkbakSAo8BhpeaNGVm+NiZA05LdqEhE3QANOfYqR0lbPLDfCcbUbIDX0/OQ+Pbr44eu01DMrVs2aCi6CnUMF5Gr2zM9uP8RxQLjCDEnZsa1IuQkSimJG0lI3liRCeIQGpKORo4BIN5kekcJD7fShHwr9uIJT9/dEggIpJ4GnO7MV5XwtM/+rdWLln7kJ5VGsCMezj/yYQRXCLBHYp4JgxSYaEBZU7wrxEAmElc4tC8GeP3kRnOPaec2+OanUL/M0iqAMDkAV2OAU1MEVaAIHYPAAnsALeDUejWfjzXiftRaMfGYf/JHx8Q1gUpd7</latexit> WW = E 1 D 1/2 ET <latexit sha1_base64="bk/ezXMm8F8IJFoa8tdF299bHM0=">AAACCHicbVDLSsNAFJ34rPEVdelmsAhurEkR1IVQ1ILLCq0ptGmYTCft0MmDmYlQQrZu/BU3LlTc+gnu/BsnbRbaemCGwzn3cu89XsyokKb5rS0sLi2vrJbW9PWNza1tY2f3XkQJx6SFIxbxtocEYTQkLUklI+2YExR4jNje6Dr37QfCBY3CphzHxAnQIKQ+xUgqyTVgN0By6Pmp7dqX9V56bGU3+X9Szeq9ZqbrrlE2K+YEcJ5YBSmDAg3X+Or2I5wEJJSYISE6lhlLJ0VcUsxIpncTQWKER2hAOoqGKCDCSSeXZPBQKX3oR1y9UMKJ+rsjRYEQ48BTlfneYtbLxf+8TiL9cyelYZxIEuLpID9hUEYwjwX2KSdYsrEiCHOqdoV4iDjCUoWXh2DNnjxPWtXKRcW6Oy3Xroo0SmAfHIAjYIEzUAO3oAFaAINH8AxewZv2pL1o79rHtHRBK3r2wB9onz8JgJgs</latexit><latexit sha1_base64="bk/ezXMm8F8IJFoa8tdF299bHM0=">AAACCHicbVDLSsNAFJ34rPEVdelmsAhurEkR1IVQ1ILLCq0ptGmYTCft0MmDmYlQQrZu/BU3LlTc+gnu/BsnbRbaemCGwzn3cu89XsyokKb5rS0sLi2vrJbW9PWNza1tY2f3XkQJx6SFIxbxtocEYTQkLUklI+2YExR4jNje6Dr37QfCBY3CphzHxAnQIKQ+xUgqyTVgN0By6Pmp7dqX9V56bGU3+X9Szeq9ZqbrrlE2K+YEcJ5YBSmDAg3X+Or2I5wEJJSYISE6lhlLJ0VcUsxIpncTQWKER2hAOoqGKCDCSSeXZPBQKX3oR1y9UMKJ+rsjRYEQ48BTlfneYtbLxf+8TiL9cyelYZxIEuLpID9hUEYwjwX2KSdYsrEiCHOqdoV4iDjCUoWXh2DNnjxPWtXKRcW6Oy3Xroo0SmAfHIAjYIEzUAO3oAFaAINH8AxewZv2pL1o79rHtHRBK3r2wB9onz8JgJgs</latexit><latexit sha1_base64="bk/ezXMm8F8IJFoa8tdF299bHM0=">AAACCHicbVDLSsNAFJ34rPEVdelmsAhurEkR1IVQ1ILLCq0ptGmYTCft0MmDmYlQQrZu/BU3LlTc+gnu/BsnbRbaemCGwzn3cu89XsyokKb5rS0sLi2vrJbW9PWNza1tY2f3XkQJx6SFIxbxtocEYTQkLUklI+2YExR4jNje6Dr37QfCBY3CphzHxAnQIKQ+xUgqyTVgN0By6Pmp7dqX9V56bGU3+X9Szeq9ZqbrrlE2K+YEcJ5YBSmDAg3X+Or2I5wEJJSYISE6lhlLJ0VcUsxIpncTQWKER2hAOoqGKCDCSSeXZPBQKX3oR1y9UMKJ+rsjRYEQ48BTlfneYtbLxf+8TiL9cyelYZxIEuLpID9hUEYwjwX2KSdYsrEiCHOqdoV4iDjCUoWXh2DNnjxPWtXKRcW6Oy3Xroo0SmAfHIAjYIEzUAO3oAFaAINH8AxewZv2pL1o79rHtHRBK3r2wB9onz8JgJgs</latexit><latexit sha1_base64="bk/ezXMm8F8IJFoa8tdF299bHM0=">AAACCHicbVDLSsNAFJ34rPEVdelmsAhurEkR1IVQ1ILLCq0ptGmYTCft0MmDmYlQQrZu/BU3LlTc+gnu/BsnbRbaemCGwzn3cu89XsyokKb5rS0sLi2vrJbW9PWNza1tY2f3XkQJx6SFIxbxtocEYTQkLUklI+2YExR4jNje6Dr37QfCBY3CphzHxAnQIKQ+xUgqyTVgN0By6Pmp7dqX9V56bGU3+X9Szeq9ZqbrrlE2K+YEcJ5YBSmDAg3X+Or2I5wEJJSYISE6lhlLJ0VcUsxIpncTQWKER2hAOoqGKCDCSSeXZPBQKX3oR1y9UMKJ+rsjRYEQ48BTlfneYtbLxf+8TiL9cyelYZxIEuLpID9hUEYwjwX2KSdYsrEiCHOqdoV4iDjCUoWXh2DNnjxPWtXKRcW6Oy3Xroo0SmAfHIAjYIEzUAO3oAFaAINH8AxewZv2pL1o79rHtHRBK3r2wB9onz8JgJgs</latexit> ⌃ = Cov(X) = EDE 1 <latexit sha1_base64="FJLFEGwaxLrbuqQdcgcRFuyRDnM=">AAACG3icbZDLSsNAFIYn9VbjLerSTbAIdWFJiqAuCsVacFnR2EITy2Q6aYfOJGFmUigh7+HKR3ElKOJOXPg2Jr2Atv4w8PGfc5hzfjekREjD+FZyS8srq2v5dXVjc2t7R9vduxdBxBG2UEAD3nKhwJT42JJEUtwKOYbMpbjpDmpZvTnEXJDAv5OjEDsM9nziEQRlanW0ss2g7LtebN+SHoNJpRYMizOvlRxXZly/qj/EJ2aSqB2tYJSMsfRFMKdQAFM1Otqn3Q1QxLAvEYVCtE0jlE4MuSSI4kS1I4FDiAawh9sp+pBh4cTj2xL9KHW6uhfw9PlSH7u/J2LIhBgxN+3MNhXztcz8r9aOpHfuxMQPI4l9NPnIi6guAz0LSu8SjpGkoxQg4iTdVUd9yCGSaZxqloI5f/MiWOXSRcm8OS1UL6dx5MEBOARFYIIzUAXXoAEsgMAjeAav4E15Ul6Ud+Vj0ppTpjP74I+Urx+JHqD5</latexit><latexit sha1_base64="FJLFEGwaxLrbuqQdcgcRFuyRDnM=">AAACG3icbZDLSsNAFIYn9VbjLerSTbAIdWFJiqAuCsVacFnR2EITy2Q6aYfOJGFmUigh7+HKR3ElKOJOXPg2Jr2Atv4w8PGfc5hzfjekREjD+FZyS8srq2v5dXVjc2t7R9vduxdBxBG2UEAD3nKhwJT42JJEUtwKOYbMpbjpDmpZvTnEXJDAv5OjEDsM9nziEQRlanW0ss2g7LtebN+SHoNJpRYMizOvlRxXZly/qj/EJ2aSqB2tYJSMsfRFMKdQAFM1Otqn3Q1QxLAvEYVCtE0jlE4MuSSI4kS1I4FDiAawh9sp+pBh4cTj2xL9KHW6uhfw9PlSH7u/J2LIhBgxN+3MNhXztcz8r9aOpHfuxMQPI4l9NPnIi6guAz0LSu8SjpGkoxQg4iTdVUd9yCGSaZxqloI5f/MiWOXSRcm8OS1UL6dx5MEBOARFYIIzUAXXoAEsgMAjeAav4E15Ul6Ud+Vj0ppTpjP74I+Urx+JHqD5</latexit><latexit sha1_base64="FJLFEGwaxLrbuqQdcgcRFuyRDnM=">AAACG3icbZDLSsNAFIYn9VbjLerSTbAIdWFJiqAuCsVacFnR2EITy2Q6aYfOJGFmUigh7+HKR3ElKOJOXPg2Jr2Atv4w8PGfc5hzfjekREjD+FZyS8srq2v5dXVjc2t7R9vduxdBxBG2UEAD3nKhwJT42JJEUtwKOYbMpbjpDmpZvTnEXJDAv5OjEDsM9nziEQRlanW0ss2g7LtebN+SHoNJpRYMizOvlRxXZly/qj/EJ2aSqB2tYJSMsfRFMKdQAFM1Otqn3Q1QxLAvEYVCtE0jlE4MuSSI4kS1I4FDiAawh9sp+pBh4cTj2xL9KHW6uhfw9PlSH7u/J2LIhBgxN+3MNhXztcz8r9aOpHfuxMQPI4l9NPnIi6guAz0LSu8SjpGkoxQg4iTdVUd9yCGSaZxqloI5f/MiWOXSRcm8OS1UL6dx5MEBOARFYIIzUAXXoAEsgMAjeAav4E15Ul6Ud+Vj0ppTpjP74I+Urx+JHqD5</latexit><latexit sha1_base64="FJLFEGwaxLrbuqQdcgcRFuyRDnM=">AAACG3icbZDLSsNAFIYn9VbjLerSTbAIdWFJiqAuCsVacFnR2EITy2Q6aYfOJGFmUigh7+HKR3ElKOJOXPg2Jr2Atv4w8PGfc5hzfjekREjD+FZyS8srq2v5dXVjc2t7R9vduxdBxBG2UEAD3nKhwJT42JJEUtwKOYbMpbjpDmpZvTnEXJDAv5OjEDsM9nziEQRlanW0ss2g7LtebN+SHoNJpRYMizOvlRxXZly/qj/EJ2aSqB2tYJSMsfRFMKdQAFM1Otqn3Q1QxLAvEYVCtE0jlE4MuSSI4kS1I4FDiAawh9sp+pBh4cTj2xL9KHW6uhfw9PlSH7u/J2LIhBgxN+3MNhXztcz8r9aOpHfuxMQPI4l9NPnIi6guAz0LSu8SjpGkoxQg4iTdVUd9yCGSaZxqloI5f/MiWOXSRcm8OS1UL6dx5MEBOARFYIIzUAXXoAEsgMAjeAav4E15Ul6Ud+Vj0ppTpjP74I+Urx+JHqD5</latexit>
  43. 43. ML models for fMRI denoising !29 • Optimise for negative entropy • a gaussian variable has the largest entropy among all random variables of equal variance (Cover and Thomas, 1991); • Computationally difficult to estimate; therefore approximate as Component Estimation J(y) = H(ygauss) H(y)<latexit sha1_base64="nGBPMfEXSg7/uQojXGhQ56c9htk=">AAACAXicbVDLSsNAFJ34rPUVdSVuBovQLiyJCOpCKLoprioYW2hDmEwn7dDJJMxMhBCKG3/FjQsVt/6FO//GSZuFth64cOace5l7jx8zKpVlfRsLi0vLK6ultfL6xubWtrmzey+jRGDi4IhFouMjSRjlxFFUMdKJBUGhz0jbH13nfvuBCEkjfqfSmLghGnAaUIyUljxz/6aa1i6b1dTLBiiRclyDx1A/a2XPrFh1awI4T+yCVECBlmd+9foRTkLCFWZIyq5txcrNkFAUMzIu9xJJYoRHaEC6mnIUEulmkxPG8EgrfRhEQhdXcKL+nshQKGUa+rozRGooZ71c/M/rJio4dzPK40QRjqcfBQmDKoJ5HrBPBcGKpZogLKjeFeIhEggrnVoegj178jxxTuoXdfv2tNK4KtIogQNwCKrABmegAZqgBRyAwSN4Bq/gzXgyXox342PaumAUM3vgD4zPHx5TlN8=</latexit><latexit sha1_base64="nGBPMfEXSg7/uQojXGhQ56c9htk=">AAACAXicbVDLSsNAFJ34rPUVdSVuBovQLiyJCOpCKLoprioYW2hDmEwn7dDJJMxMhBCKG3/FjQsVt/6FO//GSZuFth64cOace5l7jx8zKpVlfRsLi0vLK6ultfL6xubWtrmzey+jRGDi4IhFouMjSRjlxFFUMdKJBUGhz0jbH13nfvuBCEkjfqfSmLghGnAaUIyUljxz/6aa1i6b1dTLBiiRclyDx1A/a2XPrFh1awI4T+yCVECBlmd+9foRTkLCFWZIyq5txcrNkFAUMzIu9xJJYoRHaEC6mnIUEulmkxPG8EgrfRhEQhdXcKL+nshQKGUa+rozRGooZ71c/M/rJio4dzPK40QRjqcfBQmDKoJ5HrBPBcGKpZogLKjeFeIhEggrnVoegj178jxxTuoXdfv2tNK4KtIogQNwCKrABmegAZqgBRyAwSN4Bq/gzXgyXox342PaumAUM3vgD4zPHx5TlN8=</latexit><latexit sha1_base64="nGBPMfEXSg7/uQojXGhQ56c9htk=">AAACAXicbVDLSsNAFJ34rPUVdSVuBovQLiyJCOpCKLoprioYW2hDmEwn7dDJJMxMhBCKG3/FjQsVt/6FO//GSZuFth64cOace5l7jx8zKpVlfRsLi0vLK6ultfL6xubWtrmzey+jRGDi4IhFouMjSRjlxFFUMdKJBUGhz0jbH13nfvuBCEkjfqfSmLghGnAaUIyUljxz/6aa1i6b1dTLBiiRclyDx1A/a2XPrFh1awI4T+yCVECBlmd+9foRTkLCFWZIyq5txcrNkFAUMzIu9xJJYoRHaEC6mnIUEulmkxPG8EgrfRhEQhdXcKL+nshQKGUa+rozRGooZ71c/M/rJio4dzPK40QRjqcfBQmDKoJ5HrBPBcGKpZogLKjeFeIhEggrnVoegj178jxxTuoXdfv2tNK4KtIogQNwCKrABmegAZqgBRyAwSN4Bq/gzXgyXox342PaumAUM3vgD4zPHx5TlN8=</latexit><latexit sha1_base64="nGBPMfEXSg7/uQojXGhQ56c9htk=">AAACAXicbVDLSsNAFJ34rPUVdSVuBovQLiyJCOpCKLoprioYW2hDmEwn7dDJJMxMhBCKG3/FjQsVt/6FO//GSZuFth64cOace5l7jx8zKpVlfRsLi0vLK6ultfL6xubWtrmzey+jRGDi4IhFouMjSRjlxFFUMdKJBUGhz0jbH13nfvuBCEkjfqfSmLghGnAaUIyUljxz/6aa1i6b1dTLBiiRclyDx1A/a2XPrFh1awI4T+yCVECBlmd+9foRTkLCFWZIyq5txcrNkFAUMzIu9xJJYoRHaEC6mnIUEulmkxPG8EgrfRhEQhdXcKL+nshQKGUa+rozRGooZ71c/M/rJio4dzPK40QRjqcfBQmDKoJ5HrBPBcGKpZogLKjeFeIhEggrnVoegj178jxxTuoXdfv2tNK4KtIogQNwCKrABmegAZqgBRyAwSN4Bq/gzXgyXox342PaumAUM3vgD4zPHx5TlN8=</latexit> Hyvärinen, 1998b New approximations of differential entropy for independent component analysis and projection pursuit. NIPS (10) pages 273–279.MIT Press J(y) / [E{G(y)} E{G(v)}]2 <latexit sha1_base64="nyDpkNJ8POM2gPuesD04XOCiQK8=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAV6sKSFEHdFUUUVxWMLSSxTKaTdujkwsykEEJfwI2v4saFilsfwJ1v4zTNQqs/DHz85xzOnN+LGRXSML600tz8wuJSebmysrq2vqFvbt2JKOGYWDhiEe94SBBGQ2JJKhnpxJygwGOk7Q3PJ/X2iHBBo/BWpjFxA9QPqU8xksrq6nvXtfQAOjGPYhlB+8LJLpXhjA9zGily7xtdvWrUjVzwL5gFVEGhVlf/dHoRTgISSsyQELZpxNLNEJcUMzKuOIkgMcJD1Ce2whAFRLhZfs0Y7iunB/2IqxdKmLs/JzIUCJEGnuoMkByI2drE/K9mJ9I/cTMaxokkIZ4u8hMG1eGTaGCPcoIlSxUgzKn6K8QDxBGWKsCKCsGcPfkvWI36ad28Oao2z4o0ymAH7IIaMMExaIIr0AIWwOABPIEX8Ko9as/am/Y+bS1pxcw2+CXt4xvXJpm5</latexit><latexit sha1_base64="nyDpkNJ8POM2gPuesD04XOCiQK8=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAV6sKSFEHdFUUUVxWMLSSxTKaTdujkwsykEEJfwI2v4saFilsfwJ1v4zTNQqs/DHz85xzOnN+LGRXSML600tz8wuJSebmysrq2vqFvbt2JKOGYWDhiEe94SBBGQ2JJKhnpxJygwGOk7Q3PJ/X2iHBBo/BWpjFxA9QPqU8xksrq6nvXtfQAOjGPYhlB+8LJLpXhjA9zGily7xtdvWrUjVzwL5gFVEGhVlf/dHoRTgISSsyQELZpxNLNEJcUMzKuOIkgMcJD1Ce2whAFRLhZfs0Y7iunB/2IqxdKmLs/JzIUCJEGnuoMkByI2drE/K9mJ9I/cTMaxokkIZ4u8hMG1eGTaGCPcoIlSxUgzKn6K8QDxBGWKsCKCsGcPfkvWI36ad28Oao2z4o0ymAH7IIaMMExaIIr0AIWwOABPIEX8Ko9as/am/Y+bS1pxcw2+CXt4xvXJpm5</latexit><latexit sha1_base64="nyDpkNJ8POM2gPuesD04XOCiQK8=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAV6sKSFEHdFUUUVxWMLSSxTKaTdujkwsykEEJfwI2v4saFilsfwJ1v4zTNQqs/DHz85xzOnN+LGRXSML600tz8wuJSebmysrq2vqFvbt2JKOGYWDhiEe94SBBGQ2JJKhnpxJygwGOk7Q3PJ/X2iHBBo/BWpjFxA9QPqU8xksrq6nvXtfQAOjGPYhlB+8LJLpXhjA9zGily7xtdvWrUjVzwL5gFVEGhVlf/dHoRTgISSsyQELZpxNLNEJcUMzKuOIkgMcJD1Ce2whAFRLhZfs0Y7iunB/2IqxdKmLs/JzIUCJEGnuoMkByI2drE/K9mJ9I/cTMaxokkIZ4u8hMG1eGTaGCPcoIlSxUgzKn6K8QDxBGWKsCKCsGcPfkvWI36ad28Oao2z4o0ymAH7IIaMMExaIIr0AIWwOABPIEX8Ko9as/am/Y+bS1pxcw2+CXt4xvXJpm5</latexit><latexit sha1_base64="nyDpkNJ8POM2gPuesD04XOCiQK8=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAV6sKSFEHdFUUUVxWMLSSxTKaTdujkwsykEEJfwI2v4saFilsfwJ1v4zTNQqs/DHz85xzOnN+LGRXSML600tz8wuJSebmysrq2vqFvbt2JKOGYWDhiEe94SBBGQ2JJKhnpxJygwGOk7Q3PJ/X2iHBBo/BWpjFxA9QPqU8xksrq6nvXtfQAOjGPYhlB+8LJLpXhjA9zGily7xtdvWrUjVzwL5gFVEGhVlf/dHoRTgISSsyQELZpxNLNEJcUMzKuOIkgMcJD1Ce2whAFRLhZfs0Y7iunB/2IqxdKmLs/JzIUCJEGnuoMkByI2drE/K9mJ9I/cTMaxokkIZ4u8hMG1eGTaGCPcoIlSxUgzKn6K8QDxBGWKsCKCsGcPfkvWI36ad28Oao2z4o0ymAH7IIaMMExaIIr0AIWwOABPIEX8Ko9as/am/Y+bS1pxcw2+CXt4xvXJpm5</latexit> G(u) = e u2 /2 <latexit sha1_base64="VMgP1CG/cr4xdMFwfUL16RwnklA=">AAAB+HicbVDLSsNAFJ3UV62vqEs3g0Woi9akCOpCKLrQZQVjC21aJtNJO3QyCfMolNA/ceNCxa2f4s6/cdpmoa0HLhzOuZd77wkSRqVynG8rt7K6tr6R3yxsbe/s7tn7B08y1gITD8csFs0AScIoJ56iipFmIgiKAkYawfB26jdGREga80c1TogfoT6nIcVIGalr23clfXpdJp20rDvVs+qkaxedijMDXCZuRoogQ71rf7V7MdYR4QozJGXLdRLlp0goihmZFNpakgThIeqTlqEcRUT66ezyCTwxSg+GsTDFFZypvydSFEk5jgLTGSE1kIveVPzPa2kVXvop5YlWhOP5olAzqGI4jQH2qCBYsbEhCAtqboV4gATCyoRVMCG4iy8vE69auaq4D+fF2k2WRh4cgWNQAi64ADVwD+rAAxiMwDN4BW9War1Y79bHvDVnZTOH4A+szx+Zx5Hk</latexit><latexit sha1_base64="VMgP1CG/cr4xdMFwfUL16RwnklA=">AAAB+HicbVDLSsNAFJ3UV62vqEs3g0Woi9akCOpCKLrQZQVjC21aJtNJO3QyCfMolNA/ceNCxa2f4s6/cdpmoa0HLhzOuZd77wkSRqVynG8rt7K6tr6R3yxsbe/s7tn7B08y1gITD8csFs0AScIoJ56iipFmIgiKAkYawfB26jdGREga80c1TogfoT6nIcVIGalr23clfXpdJp20rDvVs+qkaxedijMDXCZuRoogQ71rf7V7MdYR4QozJGXLdRLlp0goihmZFNpakgThIeqTlqEcRUT66ezyCTwxSg+GsTDFFZypvydSFEk5jgLTGSE1kIveVPzPa2kVXvop5YlWhOP5olAzqGI4jQH2qCBYsbEhCAtqboV4gATCyoRVMCG4iy8vE69auaq4D+fF2k2WRh4cgWNQAi64ADVwD+rAAxiMwDN4BW9War1Y79bHvDVnZTOH4A+szx+Zx5Hk</latexit><latexit sha1_base64="VMgP1CG/cr4xdMFwfUL16RwnklA=">AAAB+HicbVDLSsNAFJ3UV62vqEs3g0Woi9akCOpCKLrQZQVjC21aJtNJO3QyCfMolNA/ceNCxa2f4s6/cdpmoa0HLhzOuZd77wkSRqVynG8rt7K6tr6R3yxsbe/s7tn7B08y1gITD8csFs0AScIoJ56iipFmIgiKAkYawfB26jdGREga80c1TogfoT6nIcVIGalr23clfXpdJp20rDvVs+qkaxedijMDXCZuRoogQ71rf7V7MdYR4QozJGXLdRLlp0goihmZFNpakgThIeqTlqEcRUT66ezyCTwxSg+GsTDFFZypvydSFEk5jgLTGSE1kIveVPzPa2kVXvop5YlWhOP5olAzqGI4jQH2qCBYsbEhCAtqboV4gATCyoRVMCG4iy8vE69auaq4D+fF2k2WRh4cgWNQAi64ADVwD+rAAxiMwDN4BW9War1Y79bHvDVnZTOH4A+szx+Zx5Hk</latexit><latexit sha1_base64="VMgP1CG/cr4xdMFwfUL16RwnklA=">AAAB+HicbVDLSsNAFJ3UV62vqEs3g0Woi9akCOpCKLrQZQVjC21aJtNJO3QyCfMolNA/ceNCxa2f4s6/cdpmoa0HLhzOuZd77wkSRqVynG8rt7K6tr6R3yxsbe/s7tn7B08y1gITD8csFs0AScIoJ56iipFmIgiKAkYawfB26jdGREga80c1TogfoT6nIcVIGalr23clfXpdJp20rDvVs+qkaxedijMDXCZuRoogQ71rf7V7MdYR4QozJGXLdRLlp0goihmZFNpakgThIeqTlqEcRUT66ezyCTwxSg+GsTDFFZypvydSFEk5jgLTGSE1kIveVPzPa2kVXvop5YlWhOP5olAzqGI4jQH2qCBYsbEhCAtqboV4gATCyoRVMCG4iy8vE69auaq4D+fF2k2WRh4cgWNQAi64ADVwD+rAAxiMwDN4BW9War1Y79bHvDVnZTOH4A+szx+Zx5Hk</latexit>
  44. 44. ML models for fMRI denoising !30 FastICA is then optimised as: 1. Choose an initial (e.g. random) weight vector w 2. Let 3. Let 4. If not converged, go back to 2. Component Estimation ! = !+ /||!+ ||<latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit> Hyvärinen, A.; Oja, E. (2000). "Independent component analysis: Algorithms and applications". Neural Networks. 13 (4–5): 411–430.  !+ = E{xg(!T x)} E{g0 (!T x)}!<latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit>
  45. 45. ML models for fMRI denoising !30 FastICA is then optimised as: 1. Choose an initial (e.g. random) weight vector w 2. Let 3. Let 4. If not converged, go back to 2. Component Estimation ! = !+ /||!+ ||<latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit> Hyvärinen, A.; Oja, E. (2000). "Independent component analysis: Algorithms and applications". Neural Networks. 13 (4–5): 411–430.  !+ = E{xg(!T x)} E{g0 (!T x)}!<latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit> * is first derivative of G(.) * is second derivative g(!T x)<latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit> g0 (!T x)<latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit>
  46. 46. ML models for fMRI denoising !30 FastICA is then optimised as: 1. Choose an initial (e.g. random) weight vector w 2. Let 3. Let 4. If not converged, go back to 2. Component Estimation ! = !+ /||!+ ||<latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit> Hyvärinen, A.; Oja, E. (2000). "Independent component analysis: Algorithms and applications". Neural Networks. 13 (4–5): 411–430.  !+ = E{xg(!T x)} E{g0 (!T x)}!<latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit> * is first derivative of G(.) * is second derivative g(!T x)<latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit><latexit sha1_base64="24tfcyyTNRPz2Byp2jgq5V27EPY=">AAACA3icbVC7TsMwFHV4lvIKMHaJqJDKUiUICdgqWBiL1NBKTagcx02tOnZkO4gqysDCr7AwAGLlJ9j4G5w2A7QcyfLROffq3nuChBKpbPvbWFpeWV1br2xUN7e2d3bNvf1byVOBsIs45aIXQIkpYdhVRFHcSwSGcUBxNxhfFX73HgtJOOuoSYL9GEaMDAmCSksDsxY1vIDTUE5i/WUej3EE77JO/pAfD8y63bSnsBaJU5I6KNEemF9eyFEaY6YQhVL2HTtRfgaFIojivOqlEicQjWGE+5oyGGPpZ9MjcutIK6E15EI/pqyp+rsjg7EsttSVMVQjOe8V4n9eP1XDcz8jLEkVZmg2aJhSS3GrSMQKicBI0YkmEAmid7XQCAqIlM6tqkNw5k9eJO5J86Lp3JzWW5dlGhVQA4egARxwBlrgGrSBCxB4BM/gFbwZT8aL8W58zEqXjLLnAPyB8fkDtQeYVw==</latexit> g0 (!T x)<latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit><latexit sha1_base64="7rI5Yp4ZMpNDKnPKm3I0mlBBirI=">AAACBHicbVC7TsMwFHXKq5RXgBGGiApRlipBSMBWwcJYpIZWakLlOE5q1Ykj20FUURYWfoWFARArH8HG3+C0GaDlSJaPzrlX997jJZQIaZrfWmVhcWl5pbpaW1vf2NzSt3duBUs5wjZilPGeBwWmJMa2JJLiXsIxjDyKu97oqvC795gLwuKOHCfYjWAYk4AgKJU00PfDo4bjMeqLcaS+zGERDuFd1skf8uOBXjeb5gTGPLFKUgcl2gP9y/EZSiMcS0ShEH3LTKSbQS4JojivOanACUQjGOK+ojGMsHCzyRW5cagU3wgYVy+WxkT93ZHBSBRbqsoIyqGY9QrxP6+fyuDczUicpBLHaDooSKkhmVFEYviEYyTpWBGIOFG7GmgIOURSBVdTIVizJ88T+6R50bRuTuutyzKNKtgDB6ABLHAGWuAatIENEHgEz+AVvGlP2ov2rn1MSyta2bML/kD7/AEZnpiI</latexit> Maxima of the approximation of the negentropy of wTx are obtained at certain optima of E{G(wTx)}, which under constraints: E{xg(!T x)} ! = 0<latexit sha1_base64="pEuPR6pigWO5ZsW3dVHDaA3LpUM=">AAACM3icbVDLSgMxFM3UV62vqks3wSLUhWVGBHUhFEUQ3FRobaFTSya9bUMzkyHJSMswH+XGD3EjggsVt/6D6WNRWw+EnJx7Lrn3eCFnStv2m5VaWFxaXkmvZtbWNza3sts790pEkkKFCi5kzSMKOAugopnmUAslEN/jUPV6V8N69RGkYiIo60EIDZ90AtZmlGgjNbO3127seoK31MA3F+7jTn7qHbvChw55KON+cugm+Ai7HmiCpyxjB77AdjObswv2CHieOBOSQxOUmtkXtyVo5EOgKSdK1R071I2YSM0ohyTjRgpCQnukA3VDA+KDasSjpRN8YJQWbgtpTqDxSJ3uiImvhhMap090V83WhuJ/tXqk22eNmAVhpCGg44/aEcda4GGCuMUkUM0HhhAqmZkV0y6RhGqTc8aE4MyuPE8qx4XzgnN3kiteTtJIoz20j/LIQaeoiG5QCVUQRU/oFX2gT+vZere+rO+xNWVNenbRH1g/v038qsk=</latexit><latexit sha1_base64="pEuPR6pigWO5ZsW3dVHDaA3LpUM=">AAACM3icbVDLSgMxFM3UV62vqks3wSLUhWVGBHUhFEUQ3FRobaFTSya9bUMzkyHJSMswH+XGD3EjggsVt/6D6WNRWw+EnJx7Lrn3eCFnStv2m5VaWFxaXkmvZtbWNza3sts790pEkkKFCi5kzSMKOAugopnmUAslEN/jUPV6V8N69RGkYiIo60EIDZ90AtZmlGgjNbO3127seoK31MA3F+7jTn7qHbvChw55KON+cugm+Ai7HmiCpyxjB77AdjObswv2CHieOBOSQxOUmtkXtyVo5EOgKSdK1R071I2YSM0ohyTjRgpCQnukA3VDA+KDasSjpRN8YJQWbgtpTqDxSJ3uiImvhhMap090V83WhuJ/tXqk22eNmAVhpCGg44/aEcda4GGCuMUkUM0HhhAqmZkV0y6RhGqTc8aE4MyuPE8qx4XzgnN3kiteTtJIoz20j/LIQaeoiG5QCVUQRU/oFX2gT+vZere+rO+xNWVNenbRH1g/v038qsk=</latexit><latexit sha1_base64="pEuPR6pigWO5ZsW3dVHDaA3LpUM=">AAACM3icbVDLSgMxFM3UV62vqks3wSLUhWVGBHUhFEUQ3FRobaFTSya9bUMzkyHJSMswH+XGD3EjggsVt/6D6WNRWw+EnJx7Lrn3eCFnStv2m5VaWFxaXkmvZtbWNza3sts790pEkkKFCi5kzSMKOAugopnmUAslEN/jUPV6V8N69RGkYiIo60EIDZ90AtZmlGgjNbO3127seoK31MA3F+7jTn7qHbvChw55KON+cugm+Ai7HmiCpyxjB77AdjObswv2CHieOBOSQxOUmtkXtyVo5EOgKSdK1R071I2YSM0ohyTjRgpCQnukA3VDA+KDasSjpRN8YJQWbgtpTqDxSJ3uiImvhhMap090V83WhuJ/tXqk22eNmAVhpCGg44/aEcda4GGCuMUkUM0HhhAqmZkV0y6RhGqTc8aE4MyuPE8qx4XzgnN3kiteTtJIoz20j/LIQaeoiG5QCVUQRU/oFX2gT+vZere+rO+xNWVNenbRH1g/v038qsk=</latexit><latexit sha1_base64="pEuPR6pigWO5ZsW3dVHDaA3LpUM=">AAACM3icbVDLSgMxFM3UV62vqks3wSLUhWVGBHUhFEUQ3FRobaFTSya9bUMzkyHJSMswH+XGD3EjggsVt/6D6WNRWw+EnJx7Lrn3eCFnStv2m5VaWFxaXkmvZtbWNza3sts790pEkkKFCi5kzSMKOAugopnmUAslEN/jUPV6V8N69RGkYiIo60EIDZ90AtZmlGgjNbO3127seoK31MA3F+7jTn7qHbvChw55KON+cugm+Ai7HmiCpyxjB77AdjObswv2CHieOBOSQxOUmtkXtyVo5EOgKSdK1R071I2YSM0ohyTjRgpCQnukA3VDA+KDasSjpRN8YJQWbgtpTqDxSJ3uiImvhhMap090V83WhuJ/tXqk22eNmAVhpCGg44/aEcda4GGCuMUkUM0HhhAqmZkV0y6RhGqTc8aE4MyuPE8qx4XzgnN3kiteTtJIoz20j/LIQaeoiG5QCVUQRU/oFX2gT+vZere+rO+xNWVNenbRH1g/v038qsk=</latexit> E{(!T x)2 } = ||!||2 = 1<latexit sha1_base64="Ld2L1TfjtL2FPO4WiWBFKcHTMgo=">AAACJnicbVDLSsNAFJ34rPUVdelmsAh1U5IiqItCUQSXFRpbaNoymU7boZNMmJmIJc3XuPJTXAkWEXd+ipM2C209MMzhnHu59x4vZFQqy/oyVlbX1jc2c1v57Z3dvX3z4PBB8khg4mDOuGh6SBJGA+IoqhhphoIg32Ok4Y1uUr/xSISkPKircUjaPhoEtE8xUlrqmpVbNy66Hmc9Ofb1F7vcJwPUqcOn5KxTdpMKnEyW/WQy6ZQrdtcsWCVrBrhM7IwUQIZa15y6PY4jnwQKMyRly7ZC1Y6RUBQzkuTdSJIQ4REakJamAfKJbMezMxN4qpUe7HOhX6DgTP3dESNfplvqSh+poVz0UvE/rxWp/mU7pkEYKRLg+aB+xKDiMM0M9qggWLGxJggLqneFeIgEwkonm8/rFOzFm5eJUy5dlez780L1OosjB47BCSgCG1yAKrgDNeAADJ7BK3gHU+PFeDM+jM956YqR9RyBPzC+fwBdaaY8</latexit><latexit sha1_base64="Ld2L1TfjtL2FPO4WiWBFKcHTMgo=">AAACJnicbVDLSsNAFJ34rPUVdelmsAh1U5IiqItCUQSXFRpbaNoymU7boZNMmJmIJc3XuPJTXAkWEXd+ipM2C209MMzhnHu59x4vZFQqy/oyVlbX1jc2c1v57Z3dvX3z4PBB8khg4mDOuGh6SBJGA+IoqhhphoIg32Ok4Y1uUr/xSISkPKircUjaPhoEtE8xUlrqmpVbNy66Hmc9Ofb1F7vcJwPUqcOn5KxTdpMKnEyW/WQy6ZQrdtcsWCVrBrhM7IwUQIZa15y6PY4jnwQKMyRly7ZC1Y6RUBQzkuTdSJIQ4REakJamAfKJbMezMxN4qpUe7HOhX6DgTP3dESNfplvqSh+poVz0UvE/rxWp/mU7pkEYKRLg+aB+xKDiMM0M9qggWLGxJggLqneFeIgEwkonm8/rFOzFm5eJUy5dlez780L1OosjB47BCSgCG1yAKrgDNeAADJ7BK3gHU+PFeDM+jM956YqR9RyBPzC+fwBdaaY8</latexit><latexit sha1_base64="Ld2L1TfjtL2FPO4WiWBFKcHTMgo=">AAACJnicbVDLSsNAFJ34rPUVdelmsAh1U5IiqItCUQSXFRpbaNoymU7boZNMmJmIJc3XuPJTXAkWEXd+ipM2C209MMzhnHu59x4vZFQqy/oyVlbX1jc2c1v57Z3dvX3z4PBB8khg4mDOuGh6SBJGA+IoqhhphoIg32Ok4Y1uUr/xSISkPKircUjaPhoEtE8xUlrqmpVbNy66Hmc9Ofb1F7vcJwPUqcOn5KxTdpMKnEyW/WQy6ZQrdtcsWCVrBrhM7IwUQIZa15y6PY4jnwQKMyRly7ZC1Y6RUBQzkuTdSJIQ4REakJamAfKJbMezMxN4qpUe7HOhX6DgTP3dESNfplvqSh+poVz0UvE/rxWp/mU7pkEYKRLg+aB+xKDiMM0M9qggWLGxJggLqneFeIgEwkonm8/rFOzFm5eJUy5dlez780L1OosjB47BCSgCG1yAKrgDNeAADJ7BK3gHU+PFeDM+jM956YqR9RyBPzC+fwBdaaY8</latexit><latexit sha1_base64="Ld2L1TfjtL2FPO4WiWBFKcHTMgo=">AAACJnicbVDLSsNAFJ34rPUVdelmsAh1U5IiqItCUQSXFRpbaNoymU7boZNMmJmIJc3XuPJTXAkWEXd+ipM2C209MMzhnHu59x4vZFQqy/oyVlbX1jc2c1v57Z3dvX3z4PBB8khg4mDOuGh6SBJGA+IoqhhphoIg32Ok4Y1uUr/xSISkPKircUjaPhoEtE8xUlrqmpVbNy66Hmc9Ofb1F7vcJwPUqcOn5KxTdpMKnEyW/WQy6ZQrdtcsWCVrBrhM7IwUQIZa15y6PY4jnwQKMyRly7ZC1Y6RUBQzkuTdSJIQ4REakJamAfKJbMezMxN4qpUe7HOhX6DgTP3dESNfplvqSh+poVz0UvE/rxWp/mU7pkEYKRLg+aB+xKDiMM0M9qggWLGxJggLqneFeIgEwkonm8/rFOzFm5eJUy5dlez780L1OosjB47BCSgCG1yAKrgDNeAADJ7BK3gHU+PFeDM+jM956YqR9RyBPzC+fwBdaaY8</latexit>
  47. 47. ML models for fMRI denoising !31 FastICA is then optimised as: 1. Choose an initial (e.g. random) weight vector w 2. Let 3. Let 4. If not converged, go back to 2. Component Estimation ! = !+ /||!+ ||<latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit> Hyvärinen, A.; Oja, E. (2000). "Independent component analysis: Algorithms and applications". Neural Networks. 13 (4–5): 411–430.  !+ = E{xg(!T x)} E{g0 (!T x)}!<latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit>
  48. 48. ML models for fMRI denoising !31 FastICA is then optimised as: 1. Choose an initial (e.g. random) weight vector w 2. Let 3. Let 4. If not converged, go back to 2. Component Estimation ! = !+ /||!+ ||<latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit><latexit sha1_base64="99vFKgnJTWxwf9CqRWTgwDu4IQg=">AAACK3icbVDLSsNAFJ34rPUVdelmsAiCUBMR1IVQ7MZlBWMLTSyTyaQdOsmEmYlQ0n6QG39FEBdW3PofTtostO2BYQ7n3Mu99/gJo1JZ1thYWl5ZXVsvbZQ3t7Z3ds29/UfJU4GJgznjouUjSRiNiaOoYqSVCIIin5Gm36/nfvOZCEl5/KAGCfEi1I1pSDFSWuqYddfnLJCDSH/Q5RHpohs4rz2dng2HC3U4HHbMilW1JoDzxC5IBRRodMx3N+A4jUisMENStm0rUV6GhKKYkVHZTSVJEO6jLmlrGqOISC+bHDuCx1oJYMiFfrGCE/VvR4Yime+oKyOkenLWy8VFXjtV4ZWX0ThJFYnxdFCYMqg4zJODARUEKzbQBGFB9a4Q95BAWOl8yzoEe/bkeeKcV6+r9v1FpXZbpFECh+AInAAbXIIauAMN4AAMXsAb+ARj49X4ML6M72npklH0HIB/MH5+AeYhqDM=</latexit> Hyvärinen, A.; Oja, E. (2000). "Independent component analysis: Algorithms and applications". Neural Networks. 13 (4–5): 411–430.  For one component !+ = E{xg(!T x)} E{g0 (!T x)}!<latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit><latexit sha1_base64="hIR5qzUv5xPlOEj1aeTWIlXqbcE=">AAACYHicfVHPS8MwGE3r1G3+2KY3vRSHOBFHK4J6EIYieFSwOljnSLO0C0ubkqSyUfJPevPixX/EdOtBN/GDkMd77yPf9+InlAhp2x+GuVJaXVsvV6obm1vbtXpj51mwlCPsIkYZ7/pQYEpi7EoiKe4mHMPIp/jFH9/m+ssb5oKw+ElOE9yPYBiTgCAoNTWoTzyf0aGYRvrKPBbhEL5mJ0pd33mZF0E58oNsosLWX7YnNVHHnjrV1vDoP8eypKqDetNu27OyloFTgCYo6mFQf/eGDKURjiWiUIieYyeyn0EuCaJYVb1U4ASiMQxxT8MYRlj0s1lCyjrUzNAKGNcnltaM/dmRwUjkA2pnvrRY1HLyL62XyuCyn5E4SSWO0fyhIKWWZFYetzUkHCNJpxpAxIme1UIjyCGS+lPyEJzFlZeBe9a+ajuP583OTZFGGeyDA9ACDrgAHXAPHoALEPg0SsaWsW18mRWzZjbmVtMoenbBrzL3vgHiiLnu</latexit>

×