• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Random Indexing
 

Random Indexing

on

  • 1,363 views

On space and meaning.

On space and meaning.

Statistics

Views

Total Views
1,363
Views on SlideShare
1,359
Embed Views
4

Actions

Likes
1
Downloads
10
Comments
0

1 Embed 4

http://www.slideshare.net 4

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Random Indexing Random Indexing Presentation Transcript

    • Random indexing: On space and meaning Simon Belak
    • Order of the day
      • Meaning
        • Philosophy
        • Neuroscience
        • Computer science
      • Space
        • Words as points in space
        • On dimensionality
      • Random indexing
    • What’s the meaning of meaning ?
    • Philosophers say:
      • “ Meaning just is use.”
      • – Wittgenstein
    • Neuroscientists say:
      • Episodic memory  semantic memory
        • (concrete event  abstract concept)
      • Hebbian process
    • Computer scientists say:
      • LSA
      • semantic networks
      • HAL
      • TLC
      • SAM
      • ACT-R
      • ontology
    • Projecting meaning into space
    • Adjacent words closely related
    • Movement
      • Co-occurrences
      • Hebbian process
        • Self-organisation
        • Clustering
      • Evolution of language
        • Coach ( Kocs  carriage  train  car)
    • Problem: homonym s
      • Table
      • 1.
      • a. An article of furniture supported by one or more vertical legs and having a flat horizontal surface.
      • b. The objects laid out for a meal on this article of furniture.
      • 2. The food and drink served at meals; fare: kept an excellent table.
      • 3. The company of people assembled around a table, as for a meal.
      • 4 A plateau or tableland.
      • 5 .
      • a. A flat facet cut across the top of a precious stone.
      • b . A stone or gem cut in this fashion.
      • 6 . Music
      • a. The front part of the body of a stringed instrument.
      • b. The sounding board of a harp.
      • 7 . Architecture
      • a. A raised or sunken rectangular panel on a wall.
      • b. A raised horizontal surface or continuous band on an exterior wall; a stringcourse.
      • 8 . A part of the human palm framed by four lines, analyzed in palmistry.
      • 9 . An orderly arrangement of data, especially one in which the data are arranged in columns and rows in an essentially rectangular form.
      • 1 0 . An abbreviated list, as of contents; a synopsis.
      • 1 1 . An engraved slab or tablet bearing an inscription or a device.
      • 1 2 . Anatomy The inner or outer flat layer of bones of the skull separated by the dipole.
    • Solution: high dimensionality
      • One dimension per word
      • Table extends into food , furniture , music ,... dimensions
    • Problem: synonyms
      • amazing , stupefying , staggering , awesome , awful , awe-inspiring , awing , astonishing , astounding
    • Solution: latent meaning
      • Reduced dimensionality
      • Closely related words fold into one
      • “ Higher-order” meaning
    • Random indexing
    • The idea
      • Word is the sum of it’s contexts
      • Context is the sum of it’s words
      • Grounding?
    • The algorithm
      • Take a context of words
      • Generate a context index vector
      • Add index to all the word vectors
      • Go to 1)
      • Episodic memory (2) + Hebbian process (3)
    • Dimensionality reduction
      • Sparse high-dimensional ternary index
      • (a small number of randomly distributed +1s and -1s)
      • N early orthogonal
        • Distances approximately preserved
    • The good
      • Fast, scalable
      • Trivially parallelised
        • Per word
        • Addition is associative, commutative
      • Stable
        • Words are independent
        • Integer arithmetics
      • Incremental
    • The bad
      • Memory hungry
        • Caching (Zipf’s law)
    • Uses
      • Comparing words to words
        • Query expnasion
      • Comparing documents to documents
        • Clustering
        • Search
        • Recomendations
      • Comparing documents to words
        • Keyword extraction
    • Key points
      • Meaning is use
      • Words in space
      • Multiple meanings, multiple dimensions
      • Random indexing
        • Cognitive rationale
        • Simple
        • Fast, scalable
    • Questions?
    • References
      • http://www.sics.se/~mange/papers/KarlgrenSahlgren2001.pdf
      • http://www.kfs.org/~jonathan/witt/tlph.html
      • http://www.mtsu.edu/~sschmidt/Cognitive/semantic/semantic.html
      • http://memory.syr.edu/marc/papers/HowaAddiJingKaha-LSAChap-doc.pdf
      • http://memory.psych.upenn.edu/research/research_episodic_memory.php