Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Timo Honkela: From Patterns of Movement to Subjectivity of Understanding


Published on

Human visual system interprets information obtained through eyes to build a model of the surrounding world. This channel is our main source for understanding the world. Walking on a street, reading a book, or watching a movie all rely on our visual system. The relationship between movement, visual perception and language is complex. Movement is a specific focus of this presentation for several reasons. It is a fundamental part of human activities that ground our understanding of the world. Abstract meanings are often constructed as metaphoric extensions of movement schemas. As there is an increasing amount of video and motion tracking data available, formation of semantic models based on movement using computational methods is becoming feasible. In addition to movement, multilinguality and subjectivity of understanding are also addressed.

Published in: Technology
  • Be the first to like this

Timo Honkela: From Patterns of Movement to Subjectivity of Understanding

  1. 1. From Patterns of Movement to Subjectivity of Understanding Timo Honkela Aalto University (former Helsinki University of Technology)Department of Information and Computer Science Cognitive Systems research group Finland UC Santa Barbara, 4 Dec 2012
  2. 2. 0 Background and(un)Related Topics
  3. 3. Natural language database interfacewith dependency-based compositional semantics● H. Jäppinen, T. Honkela, H. Hyötyniemi & A. Lehtola (1988): A Multilevel Natural Language Processing Model. Nordic Journal of Linguistics 11:69-87. What is the turnover of the ten largest stock exchange companies in forestry? Morphological analysis Dependency parsing Logical analysis Database query formation Result from the SQL database
  4. 4. Classical example: Learning meaning from context: Maps of words in Grimm fairy tales Honkela, Pulkki & Kohonen 1995
  5. 5. Map of Finnish ScienceChemistry MedicineBiosciences Culture and Physics and society engineering
  6. 6. WordICATimo Honkela, Aapo Hyvärinen, and Jaakko Väyrynen. WordICA - Emergence oflinguistic representations for words by independent component analysis.Natural Language Engineering, 16(3):277–308, 2010.Jaakko J. Väyrynen, Lasse Lindqvist, and Timo Honkela. Sparse distributedrepresentations for words with thresholded independent componentanalysis. In Proceedings of IJCNN07, pages 1031–1036, 2007.
  7. 7. Learning taxonomiesMari-Sanna Paukkeri, Alberto Pérez García-Plaza,Víctor Fresno, Raquel Martínez Unanue and TimoHonkela (2012). Learning a taxonomy from a setof text documents. Applied Soft Computing,12(3), pp. 1138--1148.
  8. 8. Text mining for peer support Users User modelingDiscussion forum input and analysis of postings, etc. feedback (Honkela, Izzatdust, Lagus 2012) STYLE TOPIC ANALYSIS SENTIMENT ANALYSIS ANALYSIS MULTICRITERIA SELECTION PROCESS Selected stories EVALUATION
  9. 9. ICA of wellbeing-related terms in Reddit texts (Honkela, Izzatdust, Lagus 2012)
  10. 10. Analyzing Complexity of Languages Markus Sadeniemi, Kimmo Kettunen, Tiina Lindh-Knuutila, and Timo Honkela. Complexity of European Union languages: A comparative approach. Journal of Quantitative Linguistics, 15(2):185– 211, 2008.
  11. 11. META-NETNetwork of Excellence
  12. 12. MultilingualWeb
  13. 13. Concept Formation and Communication - General Theory ξ: si ∈ Si → C Ci: N­dimensional  An individual  metric concept  mapping function  space  from symbols to  concepts Observing f1 and after symbol  S: symbol space, selection process, agent 1  The vocabulary of an communicates a symbol s* φi: Si → D agent that consists of  to agent 2 as signal d.  When agent  discrete symbols An individual  mapping from agent  2 observes d, it maps it  to some s2  λ : Ci × Cj → R, i ≠ j is vocabulary to the  ∈ S2  by using the function φ ­11.    A distance between  signal space D and Then it maps the symbol to some  two points in the  an inverse mapping φ­ point in its concept space by using  concept spaces of  1  i from the signal  ξ2.  If this point is close to its  different agents space to the symbol  observation f2 in the sense of λ, the  space communication process has  succeeded.Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of conceptformation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
  14. 14. 1Patterns ofMovement
  15. 15. Why brains?● What are the central differences between plants and animals? “The original need for a nervous system was to coordinate movement, so an organism could go find food, instead of waiting for the food to come to it.”● An extreme example: A sea squirt transforms from an “animal” to a “plant”. It absorbs its own cerebral ganglion that it used to swim about and find its attachment place.
  16. 16. Human movement
  17. 17. David Baileys thesis (1997):Verbs related to hand movement
  18. 18. Point of view from cognitive linguistics● The meaning of linguistic symbols in the mind of the language users derives from the users sensory perceptions, their actions with the world and with each other.● For example: the meaning of the word walk involves ● what walking looks like ● what it feels like to walk and after having walked ● how the world looks when walking (e.g. objects approach at a certain speed, etc.). ● ...
  19. 19. Abstract vs concrete grounding Ronald Langacker
  20. 20. Motion capture OptiTrack Image analysis Animation KinectVideo analysis Reinforcement learning Klaus Förger Tapio Takala Xi Chen Markus Koskela Paul Wagner Jorma Laaksonen Machine learning Timo Honkela Harri ValpolaRobotics Oskar Kohonen Language learning Symbol grounding Socio-cognitive modeling Learning relations
  21. 21. Multimodally Grounded Language TechnologyA project funded by Academy of Finland2011-2014Timo Honkela as the Principal InvestigatorA collaboration betweendepartments of* Information and Computer Science, and* Media Technology
  22. 22. Earliertoday: Turk on Gesture Interaction Potential uses of gesture interaction technologies: ● mouse replacement for user interaction ● video game control ● navigation for visualization ● sign language recognition ● automatic transcription of communication ● medical diagnosis and rehabilitation ● sculpting ● conducting and playing music Matthew Turk, UCSB ● interactive art
  23. 23. Labeling movementsFrom an unpublished manuscript. Experiments by Klaus Förger.
  24. 24. Linking between modalities
  25. 25. Potential uses of the emerging technologies● Multimodally grounded natural language interaction and machine translation● Animation based on linguistic instruction● Automated skill instruction (playing an instrument, learning some sports, etc.)● Video annotation● Addressing some of the fundamental issues in traditional AI, cognitive science and philosophy
  26. 26. 2Contextuality and Subjectivity of Understanding
  27. 27. Meaning is contextual red wine red skin red shirtGärdenfors: Conceptual SpacesHardin: Color for Philosophers
  28. 28. Meaning is contextualWHITESNOW -WHITE?
  29. 29. Meaning is contextual● “Small”, “big” Fuzziness● “White house”● “Get”● “Every” - “Every Swede is tall/blond”● etc. etc. Another comment: Strict compositionality cannot be assumed
  30. 30. Learning meaning from context● Self-Organizing Semantic Maps● Latent Semantic Analysis● Latent Dirichlet Allocation● WordICA● etc. etc.
  31. 31. Classical example: Learning meaning from context: Maps of words in Grimm fairy tales Honkela, Pulkki & Kohonen 1995
  32. 32. Meaning is subjective
  33. 33. Meaning is subjective● Good● Fair● Useful A proper theory of● Scientific meaning has to take● Democratic this into account● Sustainable● etc.
  34. 34. Gary B. Fogel11th of June, 2012 WCCI 2012
  35. 35. 2bMeasuringSubjectivity ofUnderstanding
  36. 36. User-specific difficulty assessmentBasic architecture of the method
  37. 37. GICA: Grounded Intersubjective Concept AnalysisDescription of the method
  38. 38. Publication:Timo Honkela, Juha Raitio, Krista Lagus, Ilari T.Nieminen, Nina Honkela, and Mika Pantzar.Subjects on objects in contexts: Using GICAmethod to quantify epistemologicalsubjectivity.Proceedings of IJCNN 2012, International JointConference on Neural Networks, pp. 2875-2883, 2012.
  39. 39. Subjectifying: adding subjectiveviews into object-context matricesOutcome: Subject-Object-Context (SOC) Tensors
  40. 40. Potential sources for subjectification● Conceptual surveys: ● individual assessment of contextual appropriateness● Text mining: ● statistics of word/phrase-context patterns● Empirical psychology: ● reaction times, etc.● Brain research
  41. 41. Flattening: unfolding 3-way tensor for traditional 2-way analysis
  42. 42. GICA: GroundedIntersubjective Concept AnalysisExamples of use
  43. 43. Data collection CONTEXTS:OBJECTS:RelaxationHappinessFitnessWellbeingSUBJECTS:Event participants
  44. 44. MDS: Objects x Subjects Fitness
  45. 45. NeRV: Objects x Subjects FitnessNeRV:J. Venna, J. Peltonen, K. Nybo, H. Aidos, and S. Kaski. Information Retrieval Perspective to NonlinearDimensionality Reduction for Data Visualization. Journal of Machine Learning Research, 11:451-490, 2010.
  46. 46. SOM: Objects x Subjects
  47. 47. Case 2: State of the Union Addresses● In this case, text mining is used for populating the Subject-Object-Context tensor● This took place by calculating the frequencies on how often a subject uses an object word in the context of a context word ● Context window of 30 words
  48. 48. Analysis of the word health
  49. 49. Conclusions (1)● Languages, including formal languages, should be considered as tools for coordination, storing and sharing knowledge in a compressed form – approximate and relative to the point of view taken● Constructing a language or symbol system (such as an ontology) is an investment and spreading the language into use in a community is even a larger one Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
  50. 50. Conclusions (2)● Making people aware of the differences in the conceptual systems among them may have different applications, e.g., ● Helping in conflict resolution ● Promoting interdisciplinary communication ● Enhancing participatory processes and democracy
  51. 51. From TEDxAALTO presentation “Measuring Subjectivity of Meaning –and How it may change our life” with illustrations by Nelli Honkela
  52. 52. 1+2Movement and Subjectivity
  53. 53. / UZnvH
  54. 54. Survey / UZnvH
  55. 55. Thankyou!