Timo Honkela: From Patterns of Movement to Subjectivity of Understanding
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Timo Honkela: From Patterns of Movement to Subjectivity of Understanding

  • 342 views
Uploaded on

Human visual system interprets information obtained through eyes to build a model of the surrounding world. This channel is our main source for understanding the world. Walking on a street, reading......

Human visual system interprets information obtained through eyes to build a model of the surrounding world. This channel is our main source for understanding the world. Walking on a street, reading a book, or watching a movie all rely on our visual system. The relationship between movement, visual perception and language is complex. Movement is a specific focus of this presentation for several reasons. It is a fundamental part of human activities that ground our understanding of the world. Abstract meanings are often constructed as metaphoric extensions of movement schemas. As there is an increasing amount of video and motion tracking data available, formation of semantic models based on movement using computational methods is becoming feasible. In addition to movement, multilinguality and subjectivity of understanding are also addressed.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to like this
No Downloads

Views

Total Views
342
On Slideshare
342
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
1
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. From Patterns of Movement to Subjectivity of Understanding Timo Honkela Aalto University (former Helsinki University of Technology)Department of Information and Computer Science Cognitive Systems research group Finland UC Santa Barbara, 4 Dec 2012
  • 2. 0 Background and(un)Related Topics
  • 3. Natural language database interfacewith dependency-based compositional semantics● H. Jäppinen, T. Honkela, H. Hyötyniemi & A. Lehtola (1988): A Multilevel Natural Language Processing Model. Nordic Journal of Linguistics 11:69-87. What is the turnover of the ten largest stock exchange companies in forestry? Morphological analysis Dependency parsing Logical analysis Database query formation Result from the SQL database
  • 4. Classical example: Learning meaning from context: Maps of words in Grimm fairy tales Honkela, Pulkki & Kohonen 1995
  • 5. Map of Finnish ScienceChemistry MedicineBiosciences Culture and Physics and society engineering
  • 6. WordICATimo Honkela, Aapo Hyvärinen, and Jaakko Väyrynen. WordICA - Emergence oflinguistic representations for words by independent component analysis.Natural Language Engineering, 16(3):277–308, 2010.Jaakko J. Väyrynen, Lasse Lindqvist, and Timo Honkela. Sparse distributedrepresentations for words with thresholded independent componentanalysis. In Proceedings of IJCNN07, pages 1031–1036, 2007.
  • 7. Learning taxonomiesMari-Sanna Paukkeri, Alberto Pérez García-Plaza,Víctor Fresno, Raquel Martínez Unanue and TimoHonkela (2012). Learning a taxonomy from a setof text documents. Applied Soft Computing,12(3), pp. 1138--1148.
  • 8. Text mining for peer support Users User modelingDiscussion forum input and analysis of postings, etc. feedback (Honkela, Izzatdust, Lagus 2012) STYLE TOPIC ANALYSIS SENTIMENT ANALYSIS ANALYSIS MULTICRITERIA SELECTION PROCESS Selected stories EVALUATION
  • 9. ICA of wellbeing-related terms in Reddit texts (Honkela, Izzatdust, Lagus 2012)
  • 10. Analyzing Complexity of Languages Markus Sadeniemi, Kimmo Kettunen, Tiina Lindh-Knuutila, and Timo Honkela. Complexity of European Union languages: A comparative approach. Journal of Quantitative Linguistics, 15(2):185– 211, 2008.
  • 11. META-NETNetwork of Excellence
  • 12. MultilingualWeb
  • 13. Concept Formation and Communication - General Theory ξ: si ∈ Si → C Ci: N­dimensional  An individual  metric concept  mapping function  space  from symbols to  concepts Observing f1 and after symbol  S: symbol space, selection process, agent 1  The vocabulary of an communicates a symbol s* φi: Si → D agent that consists of  to agent 2 as signal d.  When agent  discrete symbols An individual  mapping from agent  2 observes d, it maps it  to some s2  λ : Ci × Cj → R, i ≠ j is vocabulary to the  ∈ S2  by using the function φ ­11.    A distance between  signal space D and Then it maps the symbol to some  two points in the  an inverse mapping φ­ point in its concept space by using  concept spaces of  1  i from the signal  ξ2.  If this point is close to its  different agents space to the symbol  observation f2 in the sense of λ, the  space communication process has  succeeded.Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of conceptformation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
  • 14. 1Patterns ofMovement
  • 15. Why brains?● What are the central differences between plants and animals? “The original need for a nervous system was to coordinate movement, so an organism could go find food, instead of waiting for the food to come to it.” http://www.fi.edu/learn/brain/● An extreme example: A sea squirt transforms from an “animal” to a “plant”. It absorbs its own cerebral ganglion that it used to swim about and find its attachment place. http://goodheartextremescience.wordpress.com/2010/01/27/meet-the-creature-that-eats-its-own-brain/
  • 16. Human movement
  • 17. David Baileys thesis (1997):Verbs related to hand movement
  • 18. Point of view from cognitive linguistics● The meaning of linguistic symbols in the mind of the language users derives from the users sensory perceptions, their actions with the world and with each other.● For example: the meaning of the word walk involves ● what walking looks like ● what it feels like to walk and after having walked ● how the world looks when walking (e.g. objects approach at a certain speed, etc.). ● ...
  • 19. Abstract vs concrete grounding Ronald Langacker
  • 20. Motion capture OptiTrack Image analysis Animation KinectVideo analysis Reinforcement learning Klaus Förger Tapio Takala Xi Chen Markus Koskela Paul Wagner Jorma Laaksonen Machine learning Timo Honkela Harri ValpolaRobotics Oskar Kohonen Language learning Symbol grounding Socio-cognitive modeling Learning relations
  • 21. Multimodally Grounded Language TechnologyA project funded by Academy of Finland2011-2014Timo Honkela as the Principal InvestigatorA collaboration betweendepartments of* Information and Computer Science, and* Media Technology
  • 22. Earliertoday: Turk on Gesture Interaction Potential uses of gesture interaction technologies: ● mouse replacement for user interaction ● video game control ● navigation for visualization ● sign language recognition ● automatic transcription of communication ● medical diagnosis and rehabilitation ● sculpting ● conducting and playing music Matthew Turk, UCSB ● interactive art
  • 23. Labeling movementsFrom an unpublished manuscript. Experiments by Klaus Förger.
  • 24. Linking between modalities
  • 25. Potential uses of the emerging technologies● Multimodally grounded natural language interaction and machine translation● Animation based on linguistic instruction● Automated skill instruction (playing an instrument, learning some sports, etc.)● Video annotation● Addressing some of the fundamental issues in traditional AI, cognitive science and philosophy
  • 26. 2Contextuality and Subjectivity of Understanding
  • 27. Meaning is contextual red wine red skin red shirtGärdenfors: Conceptual SpacesHardin: Color for Philosophers
  • 28. Meaning is contextualWHITESNOW -WHITE?
  • 29. Meaning is contextual● “Small”, “big” Fuzziness● “White house”● “Get”● “Every” - “Every Swede is tall/blond”● etc. etc. Another comment: Strict compositionality cannot be assumed
  • 30. Learning meaning from context● Self-Organizing Semantic Maps● Latent Semantic Analysis● Latent Dirichlet Allocation● WordICA● etc. etc.
  • 31. Classical example: Learning meaning from context: Maps of words in Grimm fairy tales Honkela, Pulkki & Kohonen 1995
  • 32. Meaning is subjective
  • 33. Meaning is subjective● Good● Fair● Useful A proper theory of● Scientific meaning has to take● Democratic this into account● Sustainable● etc.
  • 34. Gary B. Fogel11th of June, 2012 WCCI 2012
  • 35. 2bMeasuringSubjectivity ofUnderstanding
  • 36. User-specific difficulty assessmentBasic architecture of the method
  • 37. GICA: Grounded Intersubjective Concept AnalysisDescription of the method
  • 38. Publication:Timo Honkela, Juha Raitio, Krista Lagus, Ilari T.Nieminen, Nina Honkela, and Mika Pantzar.Subjects on objects in contexts: Using GICAmethod to quantify epistemologicalsubjectivity.Proceedings of IJCNN 2012, International JointConference on Neural Networks, pp. 2875-2883, 2012.
  • 39. Subjectifying: adding subjectiveviews into object-context matricesOutcome: Subject-Object-Context (SOC) Tensors
  • 40. Potential sources for subjectification● Conceptual surveys: ● individual assessment of contextual appropriateness● Text mining: ● statistics of word/phrase-context patterns● Empirical psychology: ● reaction times, etc.● Brain research
  • 41. Flattening: unfolding 3-way tensor for traditional 2-way analysis
  • 42. GICA: GroundedIntersubjective Concept AnalysisExamples of use
  • 43. Data collection CONTEXTS:OBJECTS:RelaxationHappinessFitnessWellbeingSUBJECTS:Event participants
  • 44. MDS: Objects x Subjects Fitness
  • 45. NeRV: Objects x Subjects FitnessNeRV:J. Venna, J. Peltonen, K. Nybo, H. Aidos, and S. Kaski. Information Retrieval Perspective to NonlinearDimensionality Reduction for Data Visualization. Journal of Machine Learning Research, 11:451-490, 2010.
  • 46. SOM: Objects x Subjects
  • 47. Case 2: State of the Union Addresses● In this case, text mining is used for populating the Subject-Object-Context tensor● This took place by calculating the frequencies on how often a subject uses an object word in the context of a context word ● Context window of 30 words
  • 48. Analysis of the word health
  • 49. Conclusions (1)● Languages, including formal languages, should be considered as tools for coordination, storing and sharing knowledge in a compressed form – approximate and relative to the point of view taken● Constructing a language or symbol system (such as an ontology) is an investment and spreading the language into use in a community is even a larger one Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
  • 50. Conclusions (2)● Making people aware of the differences in the conceptual systems among them may have different applications, e.g., ● Helping in conflict resolution ● Promoting interdisciplinary communication ● Enhancing participatory processes and democracy
  • 51. From TEDxAALTO presentation “Measuring Subjectivity of Meaning –and How it may change our life” with illustrations by Nelli Honkela
  • 52. 1+2Movement and Subjectivity
  • 53. goo.gl / UZnvH
  • 54. Survey address:goo.gl / UZnvH
  • 55. Thankyou!