• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Ubiquitous Multimedia Computing
 

Ubiquitous Multimedia Computing

on

  • 2,201 views

 

Statistics

Views

Total Views
2,201
Views on SlideShare
2,201
Embed Views
0

Actions

Likes
1
Downloads
20
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Ubiquitous Multimedia Computing Ubiquitous Multimedia Computing Presentation Transcript

  • Ubiquitous Multimedia Computing and Communication: Challenges and Future Trends C.-C. Jay Kuo University of Southern California
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Mobile Multimedia
    • Features
      • Portability
        • Pervasive computing
        • Ubiquitous computing
      • Communication capability
        • 2G/2.5G/3G
        • WLAN, Bluetooth, UWB
        • Overlay networks
      • Multimedia capability
        • A/V capturing
        • A/V display
      • Marketing
        • Consumer electronics oriented
        • Rather than PC-oriented
    Embedded processors
  • Main Challenges 00111010001111010110100 wireless channels power management security and rights issue Poor Channel Conditions: Low SNR Multi-path Fading Doppler Fading
  • Embedded Computing Systems
    • Two-module solution
      • Communication module
        • RF-Modem
        • Baseband processing-channel decoding, despreading, etc.
      • Multimedia module
        • Codec, DRM, etc.
    • Embedded Media Processor Architecture
    CPU DSP Image Co-Processor On-Chip Memory CPU Image Co-Processor On-Chip Memory
  • Standards of Mobile Communications Resource: http://plus.ric.co.jp/wireless/wl003_01_0410.html Effective Distance Data Rate
  • Trends of Mobile Communications (2)
    • O rthogonal F requency D ivision M odulation
    • Wireless Systems
      • Wi-Fi ( IEEE 802.11a/g/n )
      • WiMax ( IEEE 802.16 )
      • DVB-H Broadcasting Systems
  • Trends of Mobile Multimedia Applications
    • Two-way communication
      • Video telephony
      • 3G provides the solution
    • Web access
      • Short video clips for on-demand service
        • Download or streaming
      • Video gaming
      • Something better than 3G is desirable
    • Live video broadcasting
      • DVB-H or DMB
        • Vision: Bring live TV programs to your cellular phone
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Tradeoff between Different Architectures Dedicated coprocessors ASIC Specific processors SIMD, VLIW General processors CISC or RISC Power consumption, chip area Flexibility
  • CISC versus RISC
    • Main difference
      • CISC: multi-clock complex instructions
        • Minimizing the no. of instructions per program, while sacrificing the no. of cycles per instruction
        • Example: Intel Pentium
      • RISC: one-clock reduced instructions
        • Go to the opposite direction of CISC
        • Example: ARM
    • Advantages of RISC design
      • Lower power (suitable for mobile applications)
      • Lower chip area (for lower cost)
  • Single Instruction Multiple Data (SIMD)
    • Why is SIMD?
      • Multimedia data’s low-precision
        • 8-bit pixels for image/video application
        • 16-bit samples for audio application
        • Challenges: representation, storage and processing
      • Multimedia algorithm’s inherit data parallelism
        • Add, subtract, and simple forms of multiplication and division are common operations
    • First developed by UIUC
      • Used as imaging processing engine (CM series) in early days
    • Popular engine: Intel MMX, TI iMX
  • Examples of Media Processors
    • Texas Instruments (TI)
      • DSC-25, DM-270, DM-320
      • OMAP for cellular phone
      • C64xx series
    • Intel
      • XScale Processor
    • Trimedia: TM1300
      • Speech/Image/Video
      • Somehow, not well received
    • Equator media processor
  • Integrated Solution to Video Coding
    • RISC (ARM processor)
      • Irregularity operations
      • Sequential computation (no inherent parallelism)
      • e.g. streaming parsing & entropy decoding
    • SIMD (Image co-processor)
      • Semi-regular operations
      • e.g. Intra prediction, quantization, motion prediction/compensation
    • ASIC (dedicated coprocessors)
      • Regular yet computational intensive operations
      • To save SOC area and power
      • e.g. DCT, loop filtering, half-pel interpolation etc.
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
    • Paradigm shift from hardware to software
      • Faster algorithm adoption
      • Easier for add-on functionalities
      • Reuse of platform independent codes
    • Increased use of programmable processor core in system-on-chip (SoC)
      • Easier use of high level languages such as C, C++
    Advantages of Software Solution
  • Compiler for Embedded Processors
    • Need more sophisticated compiler to generate codes to meet stringent embedded application requirements
    Relative importance of embedded software development tools Use of C code versus assembly language by design team at Northern Telecom area
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Why Multi-Codec Design? Joint S/W & H/W Decoder S/W Components Entropy decoding Selected mode decoding H/W Components Inverse transform Inverse quantization Motion search De-blocking filter MPEG-2 Stream H.264/MPEG- 4 AVC Stream VC-1 Stream Output Video/Audio
  • Flexible Design (1): Processor Consideration
    • Bitstream parsing
      • Sequential processing
      • Mutimedia standards define different bitstream formats
    • Solution
      • Programmable architecture
        • Pros: flexible, easy to be upgraded to compliance multiple standards
        • Cons: not cost-effective on power and area
      • Dedicated architecture
        • Pros: Save power, chip area
        • Cons: fixed, not easy to be modified
  • Flexible Design (2): Bus Consideration
    • Bus architecture
      • Integrating components on multimedia SoC
      • Multi-format codec is usually a hybrid SoC
        • RISC, SIMD, ASIC, on-chip memory
      • Interfacing with off-chip memory
    • How to integrate efficiently?
      • Common bus architecture
        • Flexible, accommodate to modules from all parties
        • Example: AMBA (used in ARM)
      • Dedicated bus architecture
        • High utilization of bus bandwidth, cost-effective
  • Flexible Design (3) ---- A Case Study “ Real-time Audio/Video Decoders for Digital Multimedia Broadcasting” Victor H. S. Ha, Samsung, IWSOC’04 Common bus AMBA High bandwidth Dedicated bus
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Confidentiality for Multimedia Data
    • What separates multimedia data from traditional alpha numeric data?
      • Large in file size
      • May require real-time processing (especially for continuous media)
      • Portable and mobile applications
  • Approaches to Multimedia Encryption
    • Signal scrambling
      • Historical approach
      • Not compatible with modern multimedia compression
      • Fast speed but low security
    • Total encryption with cryptographic ciphers
      • Trivial solution
      • High security but slow speed
    • Selective encryption
      • Most popular approach today
      • Limited in its range of application
    • Integrating encryption into entropy coding
      • Complementary to selective encryption
      • Very fast computation speed
    • Others
  • Selective Encryption
    • Select the most important coefficients and then encrypt them with traditional ciphers such as DES
    • Advantages
      • Lower complexity
      • High security level provided by traditional cryptology
      • Less error correction coding redundancy
      • Compatible with existing software and hardware modules
    Media Compression System Coefficient Selection Cryptographic Cipher Error Correction Coding Digitized Audiovisual data Coefficients Selected Coefficients Non-selected Coefficients Transmission channel or storage media
  • Example: Selective Encryption for G.723.1 Speech Coder
    • ITU-T Recommendation G.723.1
      • A popular low bit rate speech codec
    • Based on the human voice generation model
      • Vocoder
      • Decoder synthesizes speech using the model
    LSP Decoder Pitch Decoder Excitation Decoder + Synthesis Filter Pitch Postfilter LSP Interpolator Formant Postfilter Gain Scaling Unit LSP codebook indices Lag of pitch predictors Gain vectors Fixed codebook gains and others
  • Effect of Encrypting Different Coefficients on Speech Intelligibility Original Speech
  • Effective Selective Encryption Scheme
    • Encrypt the most significant bits of all important coefficients given below
    • The best restoration approach
      • replace encrypted bits with the average value of its range
    Encrypt 37 bits/frame roughly 20% of total encryption
  • Randomized Huffman Table Encryption 0 0 0 0 1 1 1 1 1 A 0 1 B C D E F G 0 1 0 1 1 0 1 1 0 A 0 1 B C D E F G 0 0 BADCAEFG Huffman code #0 Huffman code #1 100011001010110111101111 1 1 0011001 1 1011011 0 1 1 1111 isomorphic tree! 00000000 1 00 11 0 1 0
  • Multimedia Encryption with Randomized Entropy Coder
    • Select a good PRBG
    • Select an r -bit random seed s (encryption key)
    • Pseudo-random sequence output from PRBG(s) becomes the key hoping sequence (KHS)
    Entropy Coder Input symbol 110001… 101 1110 PRBG s KHS = 011000110 … 1110
  • Simulation Results
    • Pentium III CPU, 600MHz, 256MB RAM
    • Video compression standard: H.264
    • Cipher: the Randomized Huffman Table (RHT) encryption
    • PRBG: emulated by 128-bit MD5 hash function
    • Test video clip: “foreman”
      • CIF size: 352 x 288
      • YUV 4:2:0 format
      • The first 10 frames encrypted using key 0x246CCA6B103C95
  • Encoding Speed 0.24% 0.59% 0.28% 0.85% 0.85% 1.41% 1.21% 1.68% 1.11% 1.09% 415 ms 340 351 354 361 355 334 362 364 369 414 ms 338 350 351 358 350 330 356 360 365 47848 bits 12712 15232 14776 16744 15384 11864 15640 15928 18184 47848 bits 12712 15232 14776 16744 15384 11864 15640 15928 18184 1 2 3 4 5 6 7 8 9 10 Time increase Encoding time (encryption) Encoding time (no encryption) Bit-stream size (encryption) Bit-stream size (no encryption) Frame No.
  • Video Clip “Foreman”
  • Standard H.264 Decoding
  • Attacking with Random Seed 0x17460FD05B9EDF
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Research Motivation
    • There is no trivial relationship between quality (R-D performance) and complexity
      • Does the variable block size motion estimation always help?
        • From 16x16 to 2 modes 16x16 and 8x8 (MPEG-4)
        • From 16x16 to 7 modes 16x16, 8x16, 16x8, 8x8, 4x8, 8x4 and 4x4
      • Does the subpel motion search always help?
        • The subpel interpolation is one of the most time-consuming jobs in the decoder implementation
      • Does the deblocking filter always help?
        • The deblocking filer is another time-consuming job in the decoder implementation
      • Does the long-term memory always help?
        • This is probably not to be used in mobile video
    • The answer:
      • It is content dependent
  • Decoding-Friendly Encoder Design
    • The encoder selects the decoding-friendly modes
      • Stick to the integer motion pel
      • Stick to the option of no deblocking filter
      • Stick to the 1-reference frame
      • Stick to blocks of larger sizes
      • if the other “fancier” choices do not help much
    • How to cast these in a formal framework
      • Rate-Distortion-Complexity optimization
  • Content
    • Introduction
    • Review of embedded system design
      • Choice of RISC, VLIW and ASIC
      • Software or hardware?
      • Low power design
    • Several new design issues
      • Design of multi-format codec
      • Joint compression/encryption algorithm
      • Joint R-D-C optimization
    • Conclusion
  • Conclusion (1)
    • Mobile multimedia communication is the major trend
      • Convergence of IT, CE, telecom, gaming, etc.
    • Requirements from consumers
      • Low power
      • Low cost
      • Broadband access
      • Reliability (quality)
      • Mobility
    • Requirements from company executives
      • Short design cycle, fast turn-around time
      • Fast adaptation to new markets
  • Conclusion (2)
    • Requirements from content owners
      • Digital Rights Management
    • Technology barriers and R&D opportunities
      • More flexible architectures
        • Example: multi-format video decoder design (MPEG-2,H.264 and VC-1)
      • Lightweight encryption algorithms
        • Examples: selective speech encryption, randomized Huffman entropy coder
      • Decoder-friendly coding methods
        • Example: Joint R-D-C optimization
        • Complexity is introduced effectively and in a controlled fashion
  • Conclusion (3)
    • Technology barriers and R&D opportunities (Cont’d)
      • Broadband wireless communication
        • More spatial diversity (MIMO) to be exploited
          • MIMO-OFDM
        • New standard activities
          • 4G & 802.11n
      • Cross-layer design
        • No clear layer boundary as observed in wired communication systems
        • Integrated QoS across physical, MAC, transport, application layers