Introducing Scalable Quantum Approaches in Language Representation         Introducing Scalable Quantum Approaches in     ...
Introducing Scalable Quantum Approaches in Language RepresentationOutline       1    SQUALAR       2    Quantum Approaches...
Introducing Scalable Quantum Approaches in Language Representation  SQUALARFundamental Ideas                              ...
Introducing Scalable Quantum Approaches in Language Representation  SQUALARWhy Do We Need It?              Language indepe...
Introducing Scalable Quantum Approaches in Language Representation  Quantum ApproachesExamples              Focus: vector ...
Introducing Scalable Quantum Approaches in Language Representation  ScalabilityHeterogeneous Computing                Focu...
Introducing Scalable Quantum Approaches in Language Representation  ScalabilityThe Case for GPUs        Figure: Floating-P...
Introducing Scalable Quantum Approaches in Language Representation  ScalabilityWhy Is It Hard to Program GPUs?            ...
Introducing Scalable Quantum Approaches in Language Representation  ScalabilityThe Case for Distributed GPU systems       ...
Introducing Scalable Quantum Approaches in Language Representation  ScalabilityTools That Help                GPU programm...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEBEAGLE              Simultaneously...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEComputational steps              R...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEExperiment details              Co...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEResults: Random Indexing          ...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEResults: Random Indexing          ...
Introducing Scalable Quantum Approaches in Language Representation  A Case Study: BEAGLEResults: Convolution              ...
Introducing Scalable Quantum Approaches in Language Representation  ConclusionsSummary                http://www.squalar.o...
Upcoming SlideShare
Loading in …5
×

Introducing Scalable Quantum Approaches in Language Representation

336 views
285 views

Published on

High-performance computational resources and distributed systems are crucial for the success of real-world language technology applications.
The novel paradigm of general-purpose computing on graphics processors (GPGPU) offers a feasible and economical alternative: it has already become
a common phenomenon in scientific computation, with many algorithms adapted to the new paradigm. However, applications in language technology do not readily adapt to this approach. Recent advances show the applicability of quantum metaphors in language representation, and many algorithms in quantum mechanics have already been adapted to GPGPU computing. SQUALAR aims to match quantum algorithms with heterogeneous computing to develop new formalisms of information representation for natural language processing in quantum

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
336
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Introducing Scalable Quantum Approaches in Language Representation

  1. 1. Introducing Scalable Quantum Approaches in Language Representation Introducing Scalable Quantum Approaches in Language Representation ´ ´ Peter Wittek Sandor Daranyi Swedish School of Library and Information Science University of Boras˚ 27/06/11
  2. 2. Introducing Scalable Quantum Approaches in Language RepresentationOutline 1 SQUALAR 2 Quantum Approaches 3 Scalability 4 A Case Study: BEAGLE 5 Conclusions
  3. 3. Introducing Scalable Quantum Approaches in Language Representation SQUALARFundamental Ideas Language Technology Quantum High-performance Theory Computing
  4. 4. Introducing Scalable Quantum Approaches in Language Representation SQUALARWhy Do We Need It? Language independence Linguistic processing is typically a multi-level, bottom-up approach The processing starts at word level and then attempts to identify higher level grammatical or semantic structures Scalability Many algorithms do not readily lend themselves to parallelization. Stanford shallow parser processes about eight sentences per second running on a single core.
  5. 5. Introducing Scalable Quantum Approaches in Language Representation Quantum ApproachesExamples Focus: vector space based models (semantic spaces). Quantum phenomena in languages may be present at different levels: Sub-word level, terms, combination of concepts, and documents. Encoding term order, operator algebra.
  6. 6. Introducing Scalable Quantum Approaches in Language Representation ScalabilityHeterogeneous Computing Focus: distributed computing with graphics hardware. A heterogeneous node: multicore CPUs and GPU accelerator cores. CPU: task parallel GPU: data parallel GPU instances are now available by several cloud providers
  7. 7. Introducing Scalable Quantum Approaches in Language Representation ScalabilityThe Case for GPUs Figure: Floating-Point Operations per Second for the CPU and GPU
  8. 8. Introducing Scalable Quantum Approaches in Language Representation ScalabilityWhy Is It Hard to Program GPUs? Control ALU ALU ALU ALU Cache DR AM DR AM CP U GPU Streaming hardware Explicit memory management
  9. 9. Introducing Scalable Quantum Approaches in Language Representation ScalabilityThe Case for Distributed GPU systems GPU programming enforces minimal communication. This is in sharp contrast with traditional HPC approaches such as MPI. On the other, MapReduce also discourages internode communication. Memory limitations of the GPU beg for distributed systems.
  10. 10. Introducing Scalable Quantum Approaches in Language Representation ScalabilityTools That Help GPU programming Low level: OpenCL, CUDA Standard libraries: CuFFT, CuBLAS, CuSparse Experimental frameworks: MapReduce on GPUs (MARS) Distributed computing MPI... MapReduce framework: Hadoop Experimental: Heterogeneous systems: GPMR (distributed MapReduce on GPUs using MPI)
  11. 11. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEBEAGLE Simultaneously encodes word order and context information within semantic vectors. The outer product of two vectors compressed (Holographic Reduced Representation) n Circular convolution: zi = j=0 xj(modn+1) y(i−j)(modn+1) Complexity: O(n2 ), or O(n log n) with Fourier transformation. Two different vectors for each word w: An environmental vector A memory vector
  12. 12. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEComputational steps Random indexing Convolution FFT Pointwise multiplication IFFT
  13. 13. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEExperiment details Corpus Approx. 18,000 documents and 340,000 terms Hardware CPU: Xeon E5620, 2.4GHz GPU: Tesla C2050, 448 Cuda cores, 1.15GHz Software Lucene with StandardAnalyzer SemanticVectors (SVN)
  14. 14. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEResults: Random Indexing 14.5x slow-down Figure: Time taken in ms for random indexing based on a Lucene index
  15. 15. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEResults: Random Indexing I/O-bound Graphics hardware underutilized Is it a lost cause? MARS and GPMR, tfidf indexing already adapted to MapReduce Even though I/O-bound, word counting can be accelerated 11x speed-up on a single GPU
  16. 16. Introducing Scalable Quantum Approaches in Language Representation A Case Study: BEAGLEResults: Convolution Number of convolutions: 12,380,531 CPU: 1,281s GPU: 408s Na¨ve implementation without optimization ı Theoretical maximum is around 20x for vectors of 200 dimensions over a single-threaded CPU implemantation 10x over a four-thread CPU implementation
  17. 17. Introducing Scalable Quantum Approaches in Language Representation ConclusionsSummary http://www.squalar.org/publications.html Overall acceleration for BEAGLE with a non-optimized implementation: 2.78x. All operations should be performed on the GPU. Distributed systems are a small step forward from single GPU systems. Hoping to attract interest.

×