• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Advanced Game Development with the Mobile 3D Graphics API

Advanced Game Development with the Mobile 3D Graphics API



Slides from the Java Mobile 3D tutorial we gave at JavaOne 2004 together with Kari Pulli.

Slides from the Java Mobile 3D tutorial we gave at JavaOne 2004 together with Kari Pulli.



Total Views
Views on SlideShare
Embed Views



1 Embed 3

http://localhost 3



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Advanced Game Development with the Mobile 3D Graphics API Advanced Game Development with the Mobile 3D Graphics API Presentation Transcript

    • Advanced Game Development with the Mobile 3D Graphics API Tomi Aarnio, Kari Pulli Nokia Research Center
        • Gain an insight to the Mobile 3D Graphics API and take your games to the next level
    • Prerequisites
      • Fundamentals of 3D graphics
      • OpenGL or some other modern 3D API
      • Java programming, preferably MIDP
    • Objectives
      • Get a quick overview of the whole API
      • Gain a deeper understanding of selected topics
      • Learn practical tricks not found in the spec
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Mobile 3D Graphics API (M3G)
      • Also known as JSR-184
      • Designed for mobile devices
        • Primarily CLDC / MIDP
        • But also CDC
      Mobile 3D Graphics API Java Virtual Machine (CLDC 1.1) MIDP Midlets Graphics Hardware
    • Overcome the performance barrier Benchmarked on an ARM9 processor Native (C/C++) vs. Java on mobiles
    • Leverage hardware acceleration
      • Benefit for developers
        • Mask hardware differences (3D HW, FPU, DSP,…)
        • Exploit any new hardware automatically
      • Benefit for hardware vendors
        • Gives a concrete target which features to offer
    • Speed up development cycles
      • Separate content from code
        • Quick prototyping without programming
        • Artists create looks and behavior
        • Export all in a binary file format
        • Load the file, populate scene graph
      • Commonly needed functionality built in
        • High level functionality such as scene graph, keyframe animation
    • Why a new standard?
      • OpenGL (ES) (and D3D) are too low-level
        • Lots of Java code needed for simple things
      • Java 3D is too bloated
        • A hundred times larger than M3G
        • Does not fit together with MIDP
        • Tried and failed, but…
      • Now knew what we wanted!
        • Basic Java 3D ideas: nodes, scene graph
        • Add file format, keyframe animation
        • Remain compatible with OpenGL ES
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Key classes World Graphics3D Loader 3D graphics context Performs all rendering Scene graph root node Can load individual objects and entire scene graphs (M3G and PNG files)
    • Graphics3D
      • Contains global state
        • Target surface, viewport, depth buffer
        • Camera, light sources
        • Rendering quality hints
      • Each renderable object has its own local state
        • Geometry and appearance (material, textures, etc.)
        • Transformation relative to parent or world
    • Graphics3D: Rendering modes
      • Retained mode
        • Render a scene graph, rooted by the World
        • Take the Camera and Lights from the World
      • Immediate mode
        • Render a branch or an individual node at a time
        • Explicitly give the Camera and Lights to Graphics3D
    • Graphics3D: How-To-Use
      • Bind a target to it, render, release the target
      • Tip: Never mess with a bound target
      • Tip: Graphics3D is a singleton (threads!)
      void paint(Graphics g) { myGraphics3D.bindTarget(g); myGraphics3D.render(world); myGraphics3D.releaseTarget(); }
    • Graphics3D: Rendering targets Graphics Canvas Image CustomItem Graphics3D Image2D World Can render to textures also M3G MIDP
      • Everything is synchronous
        • A method returns only when it’s done
        • No separate thread for renderer or loader
      • There are no callbacks
        • No abstract methods, interfaces, or events
        • Tip: Do not even try to override any methods
      • Scene update is decoupled from rendering
        • animate : updates the scene to the given time
        • align : applies auto-alignments, e.g., billboards
        • render : draws the scene, no side-effects
      Things to keep in mind
    • “Hello, World” import javax.microedition.midlet.MIDlet; import javax.microedition.lcdui.Display; import javax.microedition.lcdui.game.GameCanvas; import javax.microedition.m3g.*; public class Player extends MIDlet { public void pauseApp() {} public void destroyApp(boolean b) {} public void startApp() { PlayerCanvas player = new PlayerCanvas(true); Display.getDisplay(this).setCurrent(player); try { player.run(); } catch (Exception e) {} notifyDestroyed(); } } A simplified animation player
    • “ Hello, World” class PlayerCanvas extends GameCanvas { PlayerCanvas(boolean suppress){super(suppress);} public void run() throws Exception { Graphics3D g3d = Graphics3D.getInstance(); World w = (World) Loader.load(&quot;/file.m3g&quot;)[0]; long start, elapsed, time = 0; while (getKeyStates() == 0) { start = System.currentTimeMillis(); g3d.bindTarget(getGraphics()); try { w.animate(time); g3d.render(w); } finally { g3d.releaseTarget(); } flushGraphics(); elapsed = System.currentTimeMillis()-start; time += (elapsed < 100) ? 100 : (int)elapsed; if (elapsed < 100) Thread.sleep(100-elapsed); } } }
    • Demo
      • Animation playback
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Renderable Objects Mesh Sprite3D Made of triangle strips Base class for meshes (three types exist) 2D image placed in 3D space Good for labels, etc.
    • Mesh
      • Common buffer of vertex data
      • Submesh has triangle strip indices to vertices
      • One Appearance for each submesh
      Mesh VertexBuffer coordinates normals colors texcoords Composed of submeshes IndexBuffer Appearance
    • Appearance components CompositingMode Material colors for lighting Can track per-vertex colors PolygonMode Fog Texture2D Material Blending, depth buffering Alpha testing, masking Winding, culling, shading Perspective correction hint Fades colors based on distance Linear and exponential mode Texture matrix, blending, filtering Multitexturing: One Texture2D for each unit
    • Sprite3D
      • Scaled mode for billboards, trees, etc.
      • Unscaled mode for text labels, icons, etc.
      Image2D 2D image with a position in 3D space Sprite3D Appearance Image2D CompositingMode Fog
    • Rendering tricks
      • Use layers to impose rendering order
        • Appearance contains a layer index (integer)
        • Defines a global ordering for submeshes & sprites
        • Tip: Disable z-buffering for sky boxes, use layers
      • Tip: Maximize triangle strip length
        • Even if it requires adding degenerate triangles
        • Better benefits from vertex caching
    • Example: Rotating cube // Corners of a cube as (X,Y,Z) triplets static short[] cubeVertices = { -1, -1, 1, 1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, 1, -1 }; // A color for each corner as an (R,G,B) triplet static byte[] cubeColors = { (byte) 255, (byte) 0, (byte) 0, // red … (byte) 0, (byte) 255, (byte) 0, // green }; // Define the cube as a single triangle strip static int[] indices = { 6,7,4,5,1,7,3,6,2,4,0,1,2,3 }; static int[] stripLen = { 14 }; Define raw data for a cube
    • Example: Rotating cube // Fill in VertexArrays from short[] and byte[] int numVertices = vertices.length/3; VertexArray pos = new VertexArray(numVertices, 3, 2); VertexArray col = new VertexArray(numVertices, 3, 1); pos.set(0, numVertices, cubeVertices); col.set(0, numVertices, cubeColors); // Attach the VertexArrays to a VertexBuffer // Note the scale (1.0) and bias (0,0,0) vertices = new VertexBuffer(); vertices.setPositions(pos, 1.0f, null); vertices.setColors(col); // Fill in the triangle strip triangles = new TriangleStripArray(cubeIndices, stripLen); // Create a Mesh with default Appearance cube = new Mesh(vertices, triangles, new Appearance()); Copy raw data into Objects
    • Example: Rotating cube Camera cam = new Camera(); // 60-degree field of view, screen aspect ratio // Near clipping plane at 1.0, far plane at 1000 float aspect = (float) getWidth() / (float) getHeight(); cam.setPerspective(60.0f, aspect, 1.0f, 1000.0f); // Place the camera at z=5.0 in world space // View vector is along the negative Z axis transform = new Transform(); transform.postTranslate(0.0f, 0.0f, 5.0f); g3d = Graphics3D.getInstance(); g3d.setCamera(cam, transform); Set up a Camera
    • Example: Rotating cube g3d = Graphics3D.getInstance(); g3d.bindTarget(getGraphics()); try { g3d.clear(null); g3d.render(cube, transform); } finally { g3d.releaseTarget(); } xAngle += 1; yAngle += 2; zAngle += 3; transform.setIdentity(); transform.postRotate(xAngle, 1.0f, 0.0f, 0.0f); transform.postRotate(yAngle, 0.0f, 1.0f, 0.0f); transform.postRotate(zAngle, 0.0f, 0.0f, 1.0f); Clear the buffers, render, animate
    • Demo
      • Rotating cube
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Add bitmap graphics
      • 3D graphics is mostly vector graphics
        • mathematical shape definitions
        • define camera, lights, simulate
        • “ computer graphics look”, pretty boring
      • Bitmap graphics adds visual richness
        • background images
        • texture maps on surfaces
        • sprites
    • Texturing tricks
      • Tip: Use light maps for lighting effects
        • Usually faster than per-vertex lighting
        • Use luminance textures, not RGB
        • Multitexturing is your friend
      • Tip: Use the perspective correction hint
        • Almost always faster than adding more vertices
        • But first check if the implementation supports it!
    • More bitmap tricks
      • Tip: Use background images
        • Can be scaled, tiled and scrolled very flexibly
        • Generally much faster than sky boxes or similar
      • Tip: Use sprites as impostors, particles
        • Generally (much) faster than textured quads
        • Unscaled mode is (much) faster than scaled
      • Tip: Load your images with the Loader
        • Loader.load(“/img.png”) outputs Image2D
        • Going through MIDP Image is a waste of memory
    • Demo
      • Backgrounds and texturing
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • The scene graph SkinnedMesh Group Group Group Mesh Sprite Light World Group Camera Group MorphingMesh Actually, it’s just a tree Not allowed!
    • How to find an object in the scene VertexArray Background Image2D VertexBuffer IndexBuffer Appearance Image2D Texture2D Mesh Group Mesh Mesh Group Light Camera World UserID defaults to 0 find(4) Diagram courtesy of Sean Ellis, Superscape Mesh 4 0 1 2 3 4 4 5 6 1 0 6 7 8 9 10 11 UserIDs not necessarily unique
      • From this node to the parent node
        • Ignored for the root node
      • Composed of four parts
        • Translation T
        • Orientation R
        • Non-uniform scale S
        • Generic 3x4 matrix M
      • Composite: C = T R S M
      Node transformations Group Group Group Mesh Sprite C C C C C World C
    • Node transformations
      • Tip: Keep the transformations simple
        • Favor the T R S components over M
        • Avoid non-uniform scales in S
      • Tip: How to scale along an arbitrary axis
        • There is no direct support ( C = T R N -1 S N M)
        • Method 1: R’ = R N -1 and M’ = N M
        • Method 2: Use an extra Group node
      • Tip: How to rotate about an arbitrary point
        • Again, no direct support ( C = T P -1 R P S M )
        • Method 1: Use an extra Group node
        • Method 2: If S is identity, combine into T and M
    • Example: Set up a hierarchy private Group group1, group2; private Mesh cube1, cube2, cube3; … cube1 = new Mesh(cubeVB, triangles, appearance); cube2 = new Mesh(cubeVB, triangles, appearance); cube3 = new Mesh(cubeVB, triangles, appearance); group1 = new Group(); group2 = new Group(); group1.addChild(cube1); group1.addChild(cube2); group1.addChild(group2); group2.addChild(cube3); group2 group1 cube1 cube3 cube2
    • Animate and render the hierarchy g3d.render(group1, null); … cube1.setOrientation( yAngle, 0.0f, 1.0f, 0.0f ); cube1.setTranslation( 0.0f, 2.0f, 0.0f ); cube2.setOrientation(-yAngle, 0.0f, 1.0f, 0.0f ); cube2.setTranslation( 0.0f,-2.0f, 0.0f ); group2.setOrientation(-yAngle, 0.0f, 1.0f, 0.0f ); group2.setTranslation( -2.0f, 0.0f, 0.0f ); cube3.setOrientation( zAngle, 0.0f, 0.0f, 1.0f ); cube3.setTranslation( -2.0f, 0.0f, 0.0f ); cube3.setScale( 0.5f, 0.5f, 0.5f );
    • Demo
      • Hierarchic transformations
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • MorphingMesh
      • A base mesh and several morph targets
      • Result = weighted sum of morph deltas
      • Can morph any vertex attribute
        • Vertex positions
        • Colors
        • Normals
        • Texture coordinates
      For vertex morphing animation
    • MorphingMesh Example: Animating a rabbit’s face Base Target 1 eyes closed Target 2 mouth closed Animate eyes and mouth independently
    • SkinnedMesh
      • Stretch a mesh over a hierarchic “skeleton”
        • The skeleton consists of scene graph nodes
        • Each node (“bone”) defines a transformation
        • Each vertex is linked to one or more bones
      Articulated characters without cracks at joints
    • SkinnedMesh
      • Neutral pose, bones at rest
      Weighted skinning
    • SkinnedMesh
      • Bone B rotated 90 degrees
      Weighted skinning
    • SkinnedMesh Example: Animating an arm No skinning Local skinning one bone per vertex Smooth skinning two bones per vertex
    • Agenda
      • Why should I care?
      • Getting started
      • Low level features
      • Bitmaps and texturing
      • Scene graph
      • Dynamic meshes
      • Animation
    • Animation: Relevant classes KeyframeSequence AnimationController AnimationTrack A link between sequence, controller and target Object3D Base class for all objects that can be animated Controls the playback of one or more animations Storage for keyframes Defines interpolation mode
    • KeyframeSequence Keyframe is a time and the value of a property at that time Can store any number of keyframes Several keyframe interpolation modes Can be open or closed (looping) sequence time t v KeyframeSequence Diagram courtesy of Sean Ellis, Superscape
    • AnimationController Can control several keyframed animations together Determines relationship between world time and sequence time world time AnimationController Diagram courtesy of Sean Ellis, Superscape 0 d sequence time t 0 t s 0 d sequence time 0 d sequence time
    • Animation Relates animation controller, keyframe sequence, and object property together. Identifies animated property on this object Call to animate(worldTime) s v Calculate sequence time from world time Look up value at this sequence time Apply value to animated property Diagram courtesy of Sean Ellis, Superscape Object3D AnimationTrack AnimationController KeyframeSequence 0 d sequence time
    • Animation
      • Tip: You can read back the animated values
        • Use the animation engine for anything you want
        • Much faster than doing interpolation in Java
        • Especially in case of quaternions and splines
    • Example: Set up keyframes KeyframeSequence createKeyframeSequence() { // 10 keys, 3 components, spline interpolation KeyframeSequence ks; ks = new KeyframeSequence(10,3,KeyframeSequence.SPLINE); // x grows linearly, y wiggles up-down, z remains 0.0f float[] tb = { 0.0f, 0.0f, 0.0f }; tb[0] = -4.0f; tb[1] = 0.0f; ks.setKeyframe(0, 0, tb); tb[0] = -3.0f; tb[1] = 2.0f; ks.setKeyframe(1, 10, tb); … tb[0] = 5.0f; tb[1] = 0.0f; ks.setKeyframe(9, 90, tb); ks.setDuration(100); // sequence duration in local time ks.setValidRange(0, 9); // all 10 keyframes are valid ks.setRepeatMode(KeyframeSequence.LOOP); return ks; }
    • Example: Set up controller controller = new AnimationController(); // The animation is active between these world times controller.setActiveInterval(0, 1500); // Set speed, scale “around” given time (0) controller.setSpeed(2.0f, 0); // Create animation track with keyframes & controller ks = createKeyframeSequence(); at = new AnimationTrack(ks,AnimationTrack.TRANSLATION); at.setController(controller); // Attach it to our cube cube.addAnimationTrack(at);
    • Demo
      • Keyframe animation
    • Summary
      • M3G is the standard 3D API for J2ME ™
        • Will be supported in millions of devices
      • Not just an API
        • M3G also defines a binary file format
      • M3G is flexible
        • Using the immediate mode gives you full control
        • High-level features make your code faster & smaller
    • Demo
      • 3D snowboarding
    • For More Information
      • M3G specification
        • www.forum.nokia.com/java/jsr184
      • SDKs, additional documentation
        • www.forum.nokia.com/ (click “Series 40”)
        • developer.superscape.com/ (registration needed)
      • M3G content creation tools
        • www.discreet.com/
        • www.superscape.com/
    • Q&A
      • Thanks: Kari Kangas, Sean Ellis, Nokia M3G team, JSR-184 Expert Group
    • Advanced Game Development with the Mobile 3D Graphics API Tomi Aarnio, Kari Pulli Nokia Research Center