Spectral Composition of Semantic Spaces

779 views
693 views

Published on

Spectral theory in mathematics is key to the success of as diverse application domains as quantum mechanics and latent semantic indexing, both relying on eigenvalue decomposition for the localization of their respective entities in observation space. This points at some implicit "energy" inherent in semantics and in need of quantification. We show how the structure of atomic emission spectra, and meaning in concept space, go back to the same compositional principle, plus propose a tentative solution for the computation of term, document and collection "energy" content.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
779
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Spectral Composition of Semantic Spaces

  1. 1. Spectral Composition of Semantic Spaces Spectral Composition of Semantic Spaces ´ ´ Peter Wittek Sandor Daranyi Swedish School of Library and Information Science University of Boras˚ 27/06/11
  2. 2. Spectral Composition of Semantic SpacesOutline 1 Spectroscopy 2 Semantic Spaces 3 Spectral Composition of Semantic Spaces 4 Evolving Semantics
  3. 3. Spectral Composition of Semantic Spaces SpectroscopyEmission Spectra of Individual Atoms Each element’s emission spectrum is unique. Spectroscopy identifies the elements in a compound of unknown composition. ¨ Solutions to the time-independent Schrodinger wave equation are commonly used to calculate the energy levels in the emission spectrum. Figure: The emission spectrum of hydrogen
  4. 4. Spectral Composition of Semantic Spaces SpectroscopyEmission Spectra of Compounds A spectrogram is a spectral representation of an electromagnetic signal that shows the spectral density of the signal. The continuum of energy levels called “spectral bands”. Band spectra are the combinations of many different spectral lines, resulting from rotational, vibrational and electronic transitions.
  5. 5. Spectral Composition of Semantic Spaces Semantic SpacesExamples Semantic spaces are algebraic models for representing terms as vectors. HAL VSM Latent semantic indexing Random indexing Term co-occurrence models
  6. 6. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesSemantic Spaces as Observables The semantic space must be a self-adjoint operator Different solutions exist to make a semantic space self-adjoint: Since semantic spaces are typically real-valued, self-adjoint simply means symmetric Padding a TD matrix HAL: H + H T Term co-occurrence matrix
  7. 7. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesSemantic Spectrum Eigendecomposition of a term co-occurrence matrix. Decomposing a semantic space results in a concept space or a topic model. We identify this latent topic mixture in LSI with the energy eigenstructure. More prevalent hidden topics correspond to higher energy states of atoms and molecules. In a metaphoric sense, words in an eigendecomposition are similar to chemical compounds: as both are composed of doses of latent constituents, the dosimetric view applies to them. Since the term co-occurrence matrix does not have an underlying physical meaning, we mapped the eigenvalues to the visible spectrum.
  8. 8. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesThe Spectrum of the Reuters Collection The semantic spectrum of the collection is a composite, a sum of spectra of elementary components, which would correspond to individual elements in a chemical compound in spectrophotometry.
  9. 9. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesThe Spectrum of the Term Japan We match spectral components to terms based on their proximity to latent variables.
  10. 10. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesThe Spectrum of the Term Courage
  11. 11. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesThe Spectrum of the Term Male
  12. 12. Spectral Composition of Semantic Spaces Spectral Composition of Semantic SpacesEnergy Level for a Sentence Work required to create an utterance. The less likely to encounter a certain sense, the more energy required to construct a sentence with the term in that particular sense.
  13. 13. Spectral Composition of Semantic Spaces Evolving SemanticsLanguage Change An attempt to formalize corpus dynamics. External forces leading to expansion. Inherent quality in terms and their agglomerates called their meaning. Evolving vector spaces of terms and documents follow directly from variable matrix spectra.
  14. 14. Spectral Composition of Semantic Spaces Evolving SemanticsHamiltonian The Hamiltonian which describes the energy stored in a system. H =T +V T is the potential energy and V is the kinetic energy of a system. The change in T goes back partly to changes in document collection content reflected by different index term occurrence rates.
  15. 15. Spectral Composition of Semantic Spaces Evolving SemanticsSummary http://www.squalar.org/publications.html Compositional semantics versus spectral bands. A representation richer than a simple vector space. Connecting QM and language by the concept of energy Intellectual work stored in documents. The total energy of a system changes over time. Bottlenecks Not every word has a nice intuitive spectrum.

×