Dali: A Multimedia Software Library

  • 304 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
304
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • add animation
  • add animation
  • add animation
  • add animation
  • add animation
  • add animation
  • 8 x 8
  • 8 x 8

Transcript

  • 1. Dalí A Multimedia Software Library Wei Tsang Ooi Brian Smith, Hsi Haye Chan, Matthew Chiu, Sugata Mukhopadhyay, Dan Rubinovitz, Jiesang Song, Steve Weiss.
  • 2. Motivations
  • 3. Current Software Solution Black Box C Source
  • 4. MPEG to RealVideo Black Box Real Video Encoder RGB MPEG Real OpenFile(f) GetNextFrame()
  • 5. MPEG to RealVideo RGB Decoder YUV to RGB Encoder RGB to YUV Real MPEG YUV compressed data OpenFile(f) GetNextFrame() YUV
  • 6. MPEG to RealVideo YUV Decoder YUV to RGB Encoder RGB to YUV Real MPEG compressed data
  • 7. Decode I-Frame only Black Box RGB MPEG ? OpenFile(f) GetNextFrame()
  • 8. Decode I-Frame only RGB MPEG ? Decoder YUV to RGB compressed data YUV OpenFile(f) GetNextFrame()
  • 9. Decode I-Frame only RGB MPEG ? Decoder YUV to RGB I Frame filter compressed data I-frames
  • 10. Others Motivations..
    • Read from network ?
    • Tune I/O ?
    • Run on handheld ?
    Black Box
  • 11. Current Software Solution Black Box C Source
  • 12. Dalí is in between.. Black Box C Source Dalí
  • 13. Example
  • 14. Example :
    • Process a subset of the gray scale version of each I-frame
  • 15. MPEG structures GOP hdr sequence hdr picture picture pic hdr pic data
  • 16. What we want to do
    • parse sequence header
  • 17. What we want to do
    • parse sequence header
    • while not eof do - find & parse picture header
  • 18. What we want to do
    • parse sequence header
    • while not eof do - find & parse picture header - if it is an I-frame then - parse the picture into DCT domain - IDCT into spatial domain
  • 19. What we want to do
    • parse sequence header
    • while not eof do - find & parse picture header - if it is an I-frame then - parse the picture into DCT domain - IDCT into spatial domain - take the gray scale version of the frame and process it
  • 20. Abstractions
  • 21. Abstractions
    • For input/output
    • For representing elements in MPEG
    • For storing images in DCT domain
    • For storing images in spatial domain
  • 22. BitStream & BitParser BitStream File BitParser BitParser
  • 23.
    • Parse
    • Encode
    MPEG Headers
    • Find
    • Skip
    • Dump
    MpegGopHdr MpegSeqHdr MpegPicHdr Picture Data
  • 24.
    • Parse
    • Encode
    Find
    • Find
    • Skip
    • Dump
    MpegGopHdr MpegSeqHdr MpegPicHdr Picture Data
  • 25.
    • Parse
    • Encode
    Skip
    • Find
    • Skip
    • Dump
    MpegGopHdr MpegSeqHdr MpegPicHdr Picture Data
  • 26.
    • Parse
    • Encode
    Dump
    • Find
    • Skip
    • Dump
    MpegGopHdr MpegSeqHdr MpegPicHdr Picture Data MpegGopHdr
  • 27.
    • Parse
    • Encode
    Parse
    • Find
    • Skip
    • Dump
    MpegGopHdr MpegSeqHdr MpegPicHdr Picture Data width: 320 height: 240 pic rate: 30 fps etc
  • 28.
    • Parse
    • Encode
    Encode
    • Find
    • Skip
    • Dump
    MpegSeqHdr width: 320 height: 240 pic rate: 30 fps etc
  • 29. ScImage ScBlock :- { short dc; char index[63]; int ac[63]; } represents 8x8 pixels
  • 30. ByteImage unsigned char 0..255 represents a pixel
  • 31. The Code
  • 32. Example MpegSeqHdr *seqHdr = MpegSeqHdrNew (); MpegPicHdr *picHdr = MpegPicHdrNew (); BitStream *bs = BitStreamNew (65536); BitStream
  • 33. Example BitParser *bp = BitParserNew (); BitParserAttach (bp, bs); BitStream BitParser
  • 34. Example BitStreamReadFromFile (bs, f, 0); BitStream BitParser File MpegSeqHdr MpegGopHdr MpegPicHdr
  • 35. Example MpegSeqHdrFind (bp); BitParser MpegSeqHdr MpegGopHdr MpegPicHdr
  • 36. Example MpegSeqHdrParse (bp, seqHdr); w = (seqHdr->width + 7)/8; h = (seqHdr->height + 7)/8; BitParser MpegSeqHdr MpegGopHdr MpegPicHdr
  • 37. Example ScImage *y = ScNew (w, h); ScImage *u = ScNew (w/2, h/2); ScImage *v = ScNew (w/2, h/2); y u v
  • 38. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); status = MpegPicHdrFind (bp); }
  • 39. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { } status = MpegPicHdrFind (bp); }
  • 40. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : } status = MpegPicHdrFind (bp); }
  • 41. Example status = MpegPicHdrFind (bp); while (status != NOT_FOUND) { MpegPicHdrParse (bp, picHdr); if (picHdr->type == I_FRAME) { MpegPicIParse ( bp,seqHdr,picHdr, scy,scu,scv); : } BitStreamFill(bp, f); status = MpegPicHdrFind (bp); }
  • 42. Refill the BitStream BitParser
  • 43. Refill the BitStream BitParser
  • 44. Refill the BitStream BitParser File
  • 45. Memory Mapped BitStream bs = BitStreamMMapReadNew(“tennis.mpg”); File
  • 46. Operate on a subset
  • 47. Physical ByteImage width: 8 height: 6 x: 0 y: 0 parentWidth: 8 isVirtual: 0 firstByte *
  • 48. Virtual ByteImage width: 5 height: 4 x: 1 y: 1 parentWidth: 8 isVirtual: 1 firstByte *
  • 49. Example ScImage *virt = ScClip (y,6,2,20,20); 6 20 blocks
  • 50. Example ScImage *virt = ScClip (y,6,2,20,20); ByteImage *byte = ByteNew (160, 160); ScToByte (virt, byte); 6 20 blocks
  • 51. Implementation
  • 52. Implementation
    • C library
    • Tcl bindings available
  • 53. Organization
    • Code are organized into packages
    • Basic
      • ByteImage, ScImage, etc.
    • MPEG
      • MPEG-1 Video/Audio/System
    • Vision
      • Useful vision routines
  • 54. Other Packages
    • AVI
    • WAVE
    • JPEG
    • GIF
  • 55. Availability
    • http://www.cs.cornell.edu/dali
    • open source
    • binaries for linux/win32/sunos
    • examples
    • documentations
  • 56. The End