Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Graphics in Mobile Terminals: Principles and Applications Tomi Aarnio Nokia Research Center Visual Communications Lab Sept...
What Is Computer Graphics? <ul><li>Computer graphics concerns visual synthesis of real or imaginary objects from their com...
Graphics in Mobile Terminals <ul><li>2D Vector Graphics </li></ul><ul><ul><li>What is it? </li></ul></ul><ul><ul><li>Appli...
2D Vector Graphics
2D Vector Graphics <ul><li>Raster  images are composed of individual pixels (e.g., 320x240) </li></ul><ul><ul><li>The colo...
Rasterization <ul><li>Conversion of vector images into pixel arrays </li></ul><ul><ul><li>Vector images have infinite reso...
Benefits of 2D Vector Graphics <ul><li>Zooming.  Uncovers new details instead of magnifying pixels. </li></ul><ul><li>Disp...
2D Applications: Interactive Maps <ul><li>Very little network traffic when downloading </li></ul><ul><li>No network traffi...
2D Applications: Cartoons, Micromovies <ul><li>Vector graphics often a better format than video </li></ul><ul><li>TV-quali...
2D Applications: Multimedia Messaging <ul><li>Animation effects combined with photos, audio, text </li></ul><ul><li>Messag...
2D Applications: Graphical User Interfaces <ul><li>Lots of freedom in UI design, compared to raster graphics </li></ul><ul...
2D Applications: Graphics Overlays for Video <ul><li>Video can be overlaid with text, buttons, logos, etc. </li></ul><ul><...
2D Applications: Presentations, e-Books <ul><li>Flexible page layout: capabilities far beyond HTML </li></ul><ul><li>Effic...
More Applications for 2D Vector Graphics <ul><li>Personalization </li></ul><ul><ul><li>Compact animated UI elements </li><...
2D Vector Graphics Infrastructure <ul><li>Vector graphics data can come from </li></ul><ul><ul><li>Web sites, WAP sites (m...
2D Vector Graphics Formats: Flash <ul><li>Proprietary format from Macromedia </li></ul><ul><ul><li>Flash Player installed ...
2D Vector Graphics Formats: SVG <ul><li>Scalable Vector Graphics by W3C </li></ul><ul><ul><li>Open standard from the World...
2D Vector Graphics Formats: VML <ul><li>Vector Markup Language by Microsoft </li></ul><ul><ul><li>Office2000 native format...
2D Vector Graphics Formats: MPEG-4 <ul><li>ISO/IEC standard for &quot;Coding of Audio-Visual Objects&quot; </li></ul><ul><...
Flash vs. SVG <ul><li>Flash is geared for multimedia presentations </li></ul><ul><ul><li>Best choice for non-interactive c...
3D Graphics
Modeling Objects in 3D <ul><li>Geometric descriptions of objects in 3D space </li></ul><ul><li>Usually only the surface of...
Lighting <ul><li>Lighting makes objects look three-dimensional </li></ul><ul><li>Approximates the behavior of light in the...
Texture Mapping <ul><li>Photograph or some other raster image projected on a surface </li></ul><ul><li>Practical alternati...
Environment Mapping <ul><li>Object textured with an image of its environment </li></ul><ul><li>Relatively cheap method to ...
Scene Graph <ul><li>Hierarchic data structure of objects in a visual scene and their relationships </li></ul><ul><li>Relat...
3D Applications: Personal Navigation <ul><li>Location sensitive map viewed from an upwards angle </li></ul><ul><li>&quot;U...
3D Applications: Cartoons, Micromovies <ul><li>The 3rd dimension gives a lot of freedom to authors </li></ul><ul><li>More ...
3D Applications: Multimedia Messaging <ul><li>3D animation gives more possibilities than 2D animation </li></ul><ul><li>An...
3D Applications: Games <ul><li>Visual element will be important also in mobile gaming </li></ul><ul><li>Real-time graphics...
3D Applications: E-Commerce <ul><li>Interactive product visualization </li></ul><ul><li>3D models used instead of photogra...
3D Applications: Augmented Reality <ul><li>Virtual Reality Overlaid on Reality </li></ul><ul><li>Virtual objects placed in...
3D Graphics Formats: VRML / X3D <ul><li>Virtual Reality Modeling Language </li></ul><ul><ul><li>An ISO standard from mid 1...
3D Graphics Formats: MetaStream <ul><li>Proprietary technology from MetaCreations </li></ul><ul><ul><li>Web browser plug-i...
3D Graphics Formats: Pulse3D <ul><li>Proprietary technology from Pulse Entertainment </li></ul><ul><ul><li>Web browser plu...
Graphics Software Architecture
The Role of a Graphics API <ul><li>&quot;Application Programming Interface&quot; </li></ul><ul><li>Provides a defined meth...
Components of a Graphics System Graphics Hardware ? Hardware Specific API Low-Level Generic API High-Level API / Applicati...
Existing Graphics Systems: Case 2D Windows GDI, Epoc GDI, Unix Xlib 2D vector graphics rendering library, 2D Game Engine F...
Existing Graphics Systems: Case 3D OpenGL, Direct3D IRIS Performer, 3D Game Engine VRML, MetaStream, Pulse3D (format + vie...
Graphics in Mobile Terminals: Principles and Applications Tomi Aarnio Nokia Research Center Visual Communications Lab Sept...
Upcoming SlideShare
Loading in …5
×

Graphics in mobile terminals

1,603 views

Published on

Slides from a lecture I gave about 2D/3D graphics in mobile devices, pitching the case for open standard formats and APIs, such as OpenGL and SVG.

Dated September 15, 2000.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Graphics in mobile terminals

  1. 1. Graphics in Mobile Terminals: Principles and Applications Tomi Aarnio Nokia Research Center Visual Communications Lab September 15, 2000
  2. 2. What Is Computer Graphics? <ul><li>Computer graphics concerns visual synthesis of real or imaginary objects from their computer-based models </li></ul><ul><li>Includes creation, storage and manipulation of models and images of objects, often in an interactive way </li></ul><ul><li>Closely related to image processing, which treats analysis and manipulation of existing images </li></ul>
  3. 3. Graphics in Mobile Terminals <ul><li>2D Vector Graphics </li></ul><ul><ul><li>What is it? </li></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><li>Technologies </li></ul></ul><ul><li>3D Graphics </li></ul><ul><ul><li>What is it? </li></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><li>Technologies </li></ul></ul><ul><li>Graphics Software Architecture </li></ul>
  4. 4. 2D Vector Graphics
  5. 5. 2D Vector Graphics <ul><li>Raster images are composed of individual pixels (e.g., 320x240) </li></ul><ul><ul><li>The color of each pixel is stored separately </li></ul></ul><ul><ul><li>Good for natural images (photographs and video) </li></ul></ul><ul><li>Vector images are composed of shapes: lines, curves, arcs, etc. </li></ul><ul><ul><li>Example: A circle is defined by its center and radius </li></ul></ul><ul><ul><li>Good for hand-drawn and computer-generated images </li></ul></ul><ul><li>Vector graphics usually means mixed vector/raster graphics </li></ul><ul><ul><li>Shapes may be filled or blended with raster images </li></ul></ul>
  6. 6. Rasterization <ul><li>Conversion of vector images into pixel arrays </li></ul><ul><ul><li>Vector images have infinite resolution, current displays do not </li></ul></ul><ul><ul><li>Images are sampled at some resolution (e.g., 320x240, 64k colors) </li></ul></ul><ul><li>Rasterization is done only when the image needs to be displayed </li></ul><ul><ul><li>Terminal decides the resolution and number of colors </li></ul></ul><ul><ul><li>Gives optimal quality for any display </li></ul></ul>
  7. 7. Benefits of 2D Vector Graphics <ul><li>Zooming. Uncovers new details instead of magnifying pixels. </li></ul><ul><li>Display independence. Optimal quality on any screen or printer. </li></ul><ul><li>Bandwidth efficiency. Orders of magnitude savings in some cases. </li></ul><ul><li>Interactivity. Allows terminal-side manipulation of individual shapes. </li></ul>Zoom
  8. 8. 2D Applications: Interactive Maps <ul><li>Very little network traffic when downloading </li></ul><ul><li>No network traffic when zooming or panning </li></ul><ul><li>Can be overlaid with arrows, paths, text, etc. </li></ul><ul><li>Can be printed out with perfect quality </li></ul>
  9. 9. 2D Applications: Cartoons, Micromovies <ul><li>Vector graphics often a better format than video </li></ul><ul><li>TV-quality South Park episodes at ~25 kbits/s </li></ul><ul><li>Scalable to any screen resolution with no loss of quality </li></ul><ul><li>Many popular sites on the web already (cartoons, music videos) </li></ul>Vector graphic image Compressed with JPEG
  10. 10. 2D Applications: Multimedia Messaging <ul><li>Animation effects combined with photos, audio, text </li></ul><ul><li>Messages composed on the terminal from simple elements </li></ul><ul><li>Collections of animation clips and effects on a server </li></ul><ul><li>Building on the success of SMS and first generation MMS </li></ul>
  11. 11. 2D Applications: Graphical User Interfaces <ul><li>Lots of freedom in UI design, compared to raster graphics </li></ul><ul><li>Innovative, more intuitive UIs possible for mobile devices also </li></ul><ul><li>Efficient use of screen space with zooming, panning, antialiasing </li></ul><ul><li>Example: Apple's Aqua is based on real-time vector graphics </li></ul>
  12. 12. 2D Applications: Graphics Overlays for Video <ul><li>Video can be overlaid with text, buttons, logos, etc. </li></ul><ul><li>Encoding overlays as video is a serious waste of bandwidth </li></ul><ul><li>Solution: transmit video and overlays separately </li></ul><ul><li>Use vector graphics for the overlays </li></ul>
  13. 13. 2D Applications: Presentations, e-Books <ul><li>Flexible page layout: capabilities far beyond HTML </li></ul><ul><li>Efficient screen space utilization with vector fonts </li></ul><ul><li>Text can be rotated, scaled, slanted, put on a curve, etc. </li></ul><ul><li>Hyperlinks and other interactive elements can be used </li></ul>
  14. 14. More Applications for 2D Vector Graphics <ul><li>Personalization </li></ul><ul><ul><li>Compact animated UI elements </li></ul></ul><ul><ul><li>Animated &quot;personal assistants&quot; </li></ul></ul><ul><li>Charts, graphs, figures </li></ul><ul><ul><li>Considerable bandwidth savings </li></ul></ul><ul><ul><li>Can be zoomed to uncover details </li></ul></ul><ul><ul><li>Interactive pop-up elements, etc. </li></ul></ul><ul><li>Web site intro animations </li></ul><ul><ul><li>Bandwidth efficiency is key </li></ul></ul><ul><ul><li>Could be used in WAP as well </li></ul></ul><ul><li>Server-based games </li></ul><ul><ul><li>No graphical games can be done in HTML/WML </li></ul></ul><ul><ul><li>Vector graphics & scripting provide a solution </li></ul></ul>
  15. 15. 2D Vector Graphics Infrastructure <ul><li>Vector graphics data can come from </li></ul><ul><ul><li>Web sites, WAP sites (maps, cartoons, games, etc.) </li></ul></ul><ul><ul><li>Other terminals (animation messages, etc.) </li></ul></ul><ul><ul><li>Local storage (logos, fonts, preloaded maps) </li></ul></ul><ul><li>Vector graphics engine can be integrated into </li></ul><ul><ul><li>Web/WAP browser </li></ul></ul><ul><ul><li>Operating system </li></ul></ul><ul><ul><li>Application program </li></ul></ul><ul><li>Two methods to deliver long animations </li></ul><ul><ul><li>Download the animation as one big file, then play it back </li></ul></ul><ul><ul><li>Perform playback while downloading (streaming) </li></ul></ul>
  16. 16. 2D Vector Graphics Formats: Flash <ul><li>Proprietary format from Macromedia </li></ul><ul><ul><li>Flash Player installed in over 90 % of web browsers </li></ul></ul><ul><ul><li>Good authoring tool support (Adobe, Corel, Macromedia) </li></ul></ul><ul><li>Multimedia presentations - not just graphics </li></ul><ul><ul><li>Poor integration with other web content (HTML) </li></ul></ul><ul><ul><li>Frame-based animation with synchronized audio </li></ul></ul><ul><ul><li>Streamable, but only with a reliable transport </li></ul></ul><ul><ul><li>Binary format with very simple compression </li></ul></ul><ul><ul><li>Limited client-side interactivity </li></ul></ul><ul><li>Macromedia aims to get control of the web </li></ul><ul><ul><li>Web sites built with Flash instead of HTML </li></ul></ul><ul><ul><li>Licence fees from everyone but the end users </li></ul></ul>
  17. 17. 2D Vector Graphics Formats: SVG <ul><li>Scalable Vector Graphics by W3C </li></ul><ul><ul><li>Open standard from the World Wide Web Consortium (W3C) </li></ul></ul><ul><ul><li>SVG viewer available as a browser plug-in from Adobe </li></ul></ul><ul><ul><li>Supported in authoring tools from Adobe, Corel, others </li></ul></ul><ul><li>SVG is a part of the &quot;next generation web&quot; </li></ul><ul><ul><li>XML-based text format (no compression) </li></ul></ul><ul><ul><li>Excellent integration with other web standards </li></ul></ul><ul><ul><li>Multimedia presentations built with SVG+SMIL </li></ul></ul><ul><li>Technically more advanced than Flash </li></ul><ul><ul><li>Image post-processing (filter effects) </li></ul></ul><ul><ul><li>Sophisticated animation model </li></ul></ul><ul><ul><li>Full client-side interactivity </li></ul></ul>
  18. 18. 2D Vector Graphics Formats: VML <ul><li>Vector Markup Language by Microsoft </li></ul><ul><ul><li>Office2000 native format for vector drawings </li></ul></ul><ul><ul><li>Only supported in Office2000 and Internet Explorer </li></ul></ul><ul><ul><li>Microsoft's current plans for VML are unknown </li></ul></ul><ul><li>One of the initial proposals that led to SVG </li></ul><ul><ul><li>XML-based text format, no compression </li></ul></ul><ul><ul><li>Good integration with other web content </li></ul></ul><ul><ul><li>Full client-side interactivity </li></ul></ul><ul><li>Technically somewhat challenged </li></ul><ul><ul><li>Geared towards typical office applications </li></ul></ul><ul><ul><li>No animation capabilities - still images only </li></ul></ul><ul><ul><li>No image post-processing capabilities </li></ul></ul>
  19. 19. 2D Vector Graphics Formats: MPEG-4 <ul><li>ISO/IEC standard for &quot;Coding of Audio-Visual Objects&quot; </li></ul><ul><ul><li>No support in authoring tools </li></ul></ul><ul><ul><li>No commercial viewers/players </li></ul></ul><ul><li>A presentation environment similar to Flash, but far larger </li></ul><ul><ul><li>Supports 2D/3D graphics, video, audio, etc. </li></ul></ul><ul><ul><li>The only format that is designed for streaming </li></ul></ul><ul><ul><li>Binary format with advanced compression </li></ul></ul><ul><ul><li>No compatibility with web standards </li></ul></ul><ul><li>2D vector graphics features a bit lacking </li></ul><ul><ul><li>No arbitrary clipping paths </li></ul></ul><ul><ul><li>No image post-processing </li></ul></ul>
  20. 20. Flash vs. SVG <ul><li>Flash is geared for multimedia presentations </li></ul><ul><ul><li>Best choice for non-interactive content </li></ul></ul><ul><ul><li>Published presentations cannot be changed </li></ul></ul><ul><ul><li>Simple to implement, low complexity </li></ul></ul><ul><ul><li>Owned by Macromedia </li></ul></ul><ul><li>SVG is for vector graphics content in general </li></ul><ul><ul><li>Best choice for interactive content </li></ul></ul><ul><ul><li>All content is readable and reusable </li></ul></ul><ul><ul><li>Very advanced graphics capabilities </li></ul></ul><ul><ul><li>Open standard, licence-free </li></ul></ul><ul><ul><li>Excellent web integration </li></ul></ul><ul><li>SVG and Flash will probably coexist for a long time </li></ul>
  21. 21. 3D Graphics
  22. 22. Modeling Objects in 3D <ul><li>Geometric descriptions of objects in 3D space </li></ul><ul><li>Usually only the surface of an object is modeled </li></ul><ul><li>Surfaces are most often represented with triangle meshes </li></ul>
  23. 23. Lighting <ul><li>Lighting makes objects look three-dimensional </li></ul><ul><li>Approximates the behavior of light in the real world </li></ul><ul><li>More complex lighting equations yield better visual quality </li></ul>
  24. 24. Texture Mapping <ul><li>Photograph or some other raster image projected on a surface </li></ul><ul><li>Practical alternative to modeling surfaces with millimeter accuracy </li></ul>
  25. 25. Environment Mapping <ul><li>Object textured with an image of its environment </li></ul><ul><li>Relatively cheap method to simulate reflective surfaces </li></ul>
  26. 26. Scene Graph <ul><li>Hierarchic data structure of objects in a visual scene and their relationships </li></ul><ul><li>Relationships may include transformation, visibility, and many other kinds of attributes </li></ul>House Living Room Kitchen Table TV Set Sofa Teapot
  27. 27. 3D Applications: Personal Navigation <ul><li>Location sensitive map viewed from an upwards angle </li></ul><ul><li>&quot;Up&quot; on the screen is always &quot;forward&quot; in the real world </li></ul><ul><li>Buildings, characters, arrows, etc. can be placed on the map </li></ul><ul><li>Quicker and easier to navigate than a static 2D map </li></ul>
  28. 28. 3D Applications: Cartoons, Micromovies <ul><li>The 3rd dimension gives a lot of freedom to authors </li></ul><ul><li>More life-like animation, more movie-like camera angles </li></ul><ul><li>Modest (and adjustable) bandwidth requirements </li></ul>
  29. 29. 3D Applications: Multimedia Messaging <ul><li>3D animation gives more possibilities than 2D animation </li></ul><ul><li>Animation sequences and 3D models stored on a server </li></ul><ul><li>Example: Virtual Jay Leno at http://www.nbc.com </li></ul>
  30. 30. 3D Applications: Games <ul><li>Visual element will be important also in mobile gaming </li></ul><ul><li>Real-time graphics is needed to go beyond Minesweeper and Solitaire </li></ul><ul><li>Multi-player networked games more interesting than solo games </li></ul>
  31. 31. 3D Applications: E-Commerce <ul><li>Interactive product visualization </li></ul><ul><li>3D models used instead of photographs </li></ul><ul><li>Customers get a better view of the product </li></ul>
  32. 32. 3D Applications: Augmented Reality <ul><li>Virtual Reality Overlaid on Reality </li></ul><ul><li>Virtual objects placed in your physical surroundings </li></ul><ul><li>Viewed through a Head-Mounted Display (HMD) </li></ul>
  33. 33. 3D Graphics Formats: VRML / X3D <ul><li>Virtual Reality Modeling Language </li></ul><ul><ul><li>An ISO standard from mid 1990's </li></ul></ul><ul><ul><li>Implemented as web browser plug-ins </li></ul></ul><ul><li>VRML never became popular </li></ul><ul><ul><li>Aimed to be a generic platform for all applications </li></ul></ul><ul><ul><li>Had several flaws in design and was too complex </li></ul></ul><ul><ul><li>No complete and correct implementations exist </li></ul></ul><ul><ul><li>Poor integration with other web content </li></ul></ul><ul><li>Efforts continued under the name X3D (Extensible 3D) </li></ul><ul><ul><li>A reformulation of VRML97 in XML </li></ul></ul><ul><ul><li>Carries the legacy of VRML to maintain compatibility </li></ul></ul><ul><ul><li>Will probably not succeed - should have started from scratch </li></ul></ul>
  34. 34. 3D Graphics Formats: MetaStream <ul><li>Proprietary technology from MetaCreations </li></ul><ul><ul><li>Web browser plug-in required (~100 kbytes) </li></ul></ul><ul><ul><li>Geared for product visualization in e-commerce </li></ul></ul><ul><li>Technologically impressive, but not perfect </li></ul><ul><ul><li>Nearly photorealistic rendering </li></ul></ul><ul><ul><li>Integrates well with other web content </li></ul></ul><ul><ul><li>XML-based scene descriptions </li></ul></ul><ul><ul><li>Compressed object definitions </li></ul></ul><ul><ul><li>Progressive downloading </li></ul></ul><ul><ul><li>Limited animation capabilities </li></ul></ul><ul><li>For more information and demos, see http://www.metastream.com </li></ul>
  35. 35. 3D Graphics Formats: Pulse3D <ul><li>Proprietary technology from Pulse Entertainment </li></ul><ul><ul><li>Web browser plug-in required (~100 kbytes) </li></ul></ul><ul><ul><li>Aimed for entertainment: cartoons, music videos, etc. </li></ul></ul><ul><ul><li>A lot of content already available on the web </li></ul></ul><ul><li>Technology not as advanced as MetaStream's </li></ul><ul><ul><li>Rendering quality not as impressive </li></ul></ul><ul><ul><li>Scalability and streaming features lacking </li></ul></ul><ul><ul><li>Limited integration with other web content </li></ul></ul><ul><ul><li>Good animation capabilities </li></ul></ul><ul><li>For more information and demos, see http://www.pulse3d.com </li></ul>
  36. 36. Graphics Software Architecture
  37. 37. The Role of a Graphics API <ul><li>&quot;Application Programming Interface&quot; </li></ul><ul><li>Provides a defined method of accessing graphics functions </li></ul><ul><li>Abstracts underlying hardware and software implementations </li></ul><ul><li>Allows easy development of graphics applications for different platforms </li></ul>?
  38. 38. Components of a Graphics System Graphics Hardware ? Hardware Specific API Low-Level Generic API High-Level API / Application Engine Presentation Platform Applications
  39. 39. Existing Graphics Systems: Case 2D Windows GDI, Epoc GDI, Unix Xlib 2D vector graphics rendering library, 2D Game Engine Flash, SVG, VML, PDF (format + viewer) Hardware Specific API Graphics Hardware Cartoons, MMS, e-Books, GUIs, Games, Web/WAP services
  40. 40. Existing Graphics Systems: Case 3D OpenGL, Direct3D IRIS Performer, 3D Game Engine VRML, MetaStream, Pulse3D (format + viewer) Hardware Specific API Graphics Hardware Games, Maps, Cartoons, GUIs, etc.
  41. 41. Graphics in Mobile Terminals: Principles and Applications Tomi Aarnio Nokia Research Center Visual Communications Lab September 15, 2000

×