• Save
Java ME - 08 - Mobile 3D Graphics
Upcoming SlideShare
Loading in...5
×
 

Java ME - 08 - Mobile 3D Graphics

on

  • 6,992 views

Nearly all of today's PC and console games use 3D graphics. With mobile phones getting more and more powerful and some devices even boasting a hardware 3D acceleration, development using ...

Nearly all of today's PC and console games use 3D graphics. With mobile phones getting more and more powerful and some devices even boasting a hardware 3D acceleration, development using three-dimensional graphics is an important topic for mobile computing. This module covers the JSR 184 (Mobile 3D Graphics) and explores the possibilities offered by its retained mode (high level) API for defining a scene graph and modifying objects in the world. A detailed example walks you through the basics of the free, open source 3D package Blender. Using this powerful software, you can create a sample 3D scene that you can display on your mobile phone with just a few lines of code. The challenge is about extending a sample 3D game framework, featuring an exciting car racing game!

Contents:

* Mobile 3D - Overview
* JSR 184 (m3g)
* Scene graph
* Your first m3g file with Blender
* Display, load and modify the 3D scene
* Objects and materials
* Challenge: 3D car racing game

Statistics

Views

Total Views
6,992
Views on SlideShare
6,922
Embed Views
70

Actions

Likes
3
Downloads
0
Comments
1

5 Embeds 70

http://www.slideshare.net 61
http://www.techgig.com 5
http://127.0.0.1 2
http://translate.googleusercontent.com 1
http://static.slidesharecdn.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Mobile 3D Graphic
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Java ME - 08 - Mobile 3D Graphics Java ME - 08 - Mobile 3D Graphics Presentation Transcript

  • Java™ Platform, Micro Edition
    Part 8 – Mobile 3D Graphics
    v3.0a – 18 April 2009
    1
    Andreas Jakl, 2009
  • Disclaimer
    These slides are provided free of charge at http://www.symbianresources.com and are used during Java ME courses at the University of Applied Sciences in Hagenberg, Austria at the Mobile Computing department ( http://www.fh-ooe.at/mc )
    Respecting the copyright laws, you are allowed to use them:
    for your own, personal, non-commercial use
    in the academic environment
    In all other cases (e.g. for commercial training), please contact andreas.jakl@fh-hagenberg.at
    The correctness of the contents of these materials cannot be guaranteed. Andreas Jakl is not liable for incorrect information or damage that may arise from using the materials.
    This document contains copyright materials which are proprietary to Sun or various mobile device manufacturers, including Nokia, SonyEricsson and Motorola. Sun, Sun Microsystems, the Sun Logo and the Java™ Platform, Micro Edition are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.
    Andreas Jakl, 2009
    2
  • Agenda
    Mobile 3D – Overview
    JSR 184
    Scene graph
    Your first m3g file with Blender
    Display, load and modify the 3D scene
    Objects and materials
    Andreas Jakl, 2009
    3
  • Introduction
    Mobile 3D
    Andreas Jakl, 2009
    4
  • 3D Java Game Examples
    5
    Brothers in Arms: Earned in Blood (Gameloft)
    Planet Riders 3D (Fishlabs)
    Massive Snowboarding 3D (Gameloft)
    Tornado Mania! 3D (Digital Chocolate)
    For comparison: Current C++ / Open GL ES based N-Gage games (Symbian OS)(This is what is currently possible with mid-/higher class mobile phones)
    Blades & Magic 3D (Fishlabs)
    Hooked On: Creatures of the Deep (Infinite Dreams)
    Star Wars: The Force Unleashed (Universomo / LucasArts)
    One (Digital Legends)
    Andreas Jakl, 2009
  • Benchmark Your Phone
    FuturemarkSPMarkJava JSR 184http://www.futuremark.com/products/spmark/spmarkjavajsr184/
    JBenchmark 3D/HD:http://www.jbenchmark.com/
    Andreas Jakl, 2009
    6
  • 3D APIs for Java ME
    Mobile 3D Graphics API (JSR 184)
    Java based 3D graphics library
    Supports immediate and retained mode (low and high level)
    Optional package
    Open GL ES (JSR 239)
    Provide Java bindings to the Open GL ES native 3D graphics library
    OpenGL ES can also be used in native code (Symbian OS / C++)
    Mascot Capsule
    Like JSR 184, additionally supported by e.g. SonyEricsson
    (Java 3D from Java SE is too bloated for mobile phones: ~40 MB)
    7
    Andreas Jakl, 2009
  • 3D APIs
    M3G builds on the feature set of OpenGL ES
    Both APIs are designed concurrently
    Andreas Jakl, 2009
    8
    Native C / C++ Applications
    Java Applications
    M3G (JSR 184)
    OpenGL ES
    Graphics Hardware
  • Performance
    Java is slow on mobile phones
    KVM most widely used today
    Nokia (and others) migrating to HotSpot VM from Sun
    But HotSpot takes a lot of RAM  problematic in real life
    9
    Andreas Jakl, 2009
    Benchmarked on an ARM926EJ-S processor with hand-optimized Java and assembly code
    Diagram from TomiAarnio (see sources slide at the end)
  • The Basics of JSR 184
    Get started
    Andreas Jakl, 2009
    10
  • JSR 184
    Hardware independent
    Uses newer hardware + 3D accelerators if present
    Separate code from content
    m3g file format contains models, animation, appearance, textures, ...
    Control logic in source code
    Tightly integrated with LCDUI (from MIDP, requires floating point support of CLDC 1.1+)
    Built in high level & low level API
    Scene graphs, keyframe animation, bones, ...
    11
    Andreas Jakl, 2009
  • Modes
    Immediate Mode
    Create objects programmatically
    Uses triangles as primitives
    Low level
    Retained mode
    Create objects in 3D app
    Scene graphs
    Keeps graphics processing code in native code (no Java)!
    Mixing is possible
    (e.g. insert immediate object into a scene graph)
    12
    Andreas Jakl, 2009
  • Tools
    Blender (Freeware, http://www.blender.org/)
    m3g exporter plug-in: http://www.nelson-games.de/bl2m3g/default.html
    Blender: Powerful, but difficult to learn UI
    Written in Python
    Commercial
    3DS Max (integrated)
    Maya
    Lightwave
    13
    Andreas Jakl, 2009
  • JSR 184 – Synchronous Structure
    All APIs are synchronous
    Calls return when completed
    Create own thread for loading larger scenes!
    Structure
    No callbacks (interfaces, events, abstract methods)
    Do not override any methods
    14
    Andreas Jakl, 2009
  • m3g File Format
    Compact binary format
    Stores complete scene graph in file
    Objects are serialized
    Includes camera(s), lightning and objects
    Can be compressed
    Viewer:http://www.mascotcapsule.com/toolkit/m3g/en/index.php
    15
    Andreas Jakl, 2009
  • Scene Graph
    Andreas Jakl, 2009
    16
    World is a top-level node containing the whole scene
    Background defines the background against which the 3D scene is rendered
    World
    Background
    Groups allow the application to treat multiple nodes as a single unit
    Group
    Camera
    Light
    Groups can be nested inside other groups
    Sprite3D is a 2D image with a 3D position
    Light defines a light source in the scene
    Camera defines a viewpoint
    Sprite3D
    Group
    Group
    Mesh
    User object
    Mesh defines the 3D geometry of a visible object
    An arbitrary user object can be associated with any scene object
    MorphingMesh
    SkinnedMesh
    Morphing and skinned meshes are animated geometry objects
    Based on the scene graph image from the JSR 184 API documentation
  • Graph vs. Tree
    Scene graph has a tree structure
    Node can belong to at most one group at a time
    But components (VertexArrays, textures, ...) can be shared
    No cyclic structures allowed
    Andreas Jakl, 2009
    17
    Node
    Node
    Component
    Group
    Group
    Component
    Node
  • Scene Graph – Example
    Andreas Jakl, 2009
    18
    World root node
    Camera and light
    The car
    The landscape scene group
    Tree
    The tree consists of two meshes
    The tree has no texture image
    Grass block
    Grass texture
  • Node
    Abstract base class for all scene graph nodes
    Contains information about:
    Transformation
    Parent
    Alignment (also automatic!)
    Visibility
    ...
    Andreas Jakl, 2009
    19
    Node
    Camera
    Light
    Group
    Mesh
    Sprite3D
    World
  • Key Classes
    3D graphics context
    Handles all rendering
    Graphics3D
    Utility to load m3g and png files
    (entire scene graphs or single objects)
    Loader
    Root node of the scene graph
    World
    20
    Andreas Jakl, 2009
  • Graphics3D
    Stores global state
    Rendering target, viewport, depth buffer, back buffer
    Camera, light sources
    Rendering quality hints (Antialiasing, dithering, true color rendering)
    Each node has its own local state
    Geometry & appearance (material, textures, ...)
    Transformation relative to the parent or world
    21
    Andreas Jakl, 2009
    Graphics3D
  • Graphics3D – Rendering Modes
    Retained mode
    Render entire world (scenegraph)
    Uses lights and camera nodes from the world
    Immediate mode
    Render scene graph nodes or submesh
    Uses lights and camera from Graphics3D object
    Andreas Jakl, 2009
    22
  • Graphics3D – Render Targets
    Graphics3D can render to any Graphics object or Image2D
     possible to render to textures (eg. environment mapping)
    Andreas Jakl, 2009
    23
    World
    M3G(JSR 184)
    Graphics3D
    Image2D
    Graphics
    Canvas
    Image
    MIDP
    CustomItem
  • Graphics3D – Example
    Target has to be bound and released
    Do not modify the target from the outside in between
    Andreas Jakl, 2009
    24
    publicclassMyCanvasextendsCanvas
    {
    Graphics3D myG3D = Graphics3D.getInstance();
     
    publicvoidpaint(Graphics g) {
    try {
    myG3D.bindTarget(g);
    // ... update thescene ...
    // ... render thescene ...
    } finally {
    myG3D.releaseTarget();
    }
    }
  • World Node
    Top level node for a scene graph
    Contains
    Other nodes that compose the scene
    Background
    Camera, Light, Fog
    Requires an active camera for rendering
    Andreas Jakl, 2009
    25
    World
  • Example – Goal
    Create an m3g file with Blender
    Load and display the file in a MIDlet
    Rotate the object
    Andreas Jakl, 2009
    26
  • Blender
    The basics of
    Andreas Jakl, 2009
    27
  • Blender – Interface
    Andreas Jakl, 2009
    28
    3D Viewport
    Light
    Sample Cube
    Camera
    Buttons Window
  • Blender – Windows
    Andreas Jakl, 2009
    29
    The highlighted window receiveskeyboard inputs:
    Highlighted
    (slightly brightercolour)
    Change the window type:
    ... you can split, join and resize windows anddrag mini windows (with the )
  • Blender – 3D Viewport
    Andreas Jakl, 2009
    30
    Click + hold middle mouse button
     Rotate viewport
    Shift + middle mouse button
     Pan view (move sideways)
    Mouse wheel
     Zoom
    Z-key
     Toggle wireframe or solid mode
    NUM5 (with NUM LOCK on)
     Toggle Ortographic / Perspective
    NUM0 (with NUM LOCK on)
     Camera view (MMB will exit it)
    Choose viewport shading (more options than Z)
    NUM refers to the number pad – if you pressed the numbers above the normal keys instead, press 1 to return to the normal view (for viewing layer 1)
  • The 3D Cursor
    Andreas Jakl, 2009
    31
    Task: place the 3D cursor between the camera and the cube
    Make sure you are in object mode (press TAB to toggle)
    Disable the “3d transform manipulator” to make sure you can move the cursor without selecting the cube by accident
    Hit NUM7 to get to the top view
    Click with LMB between cube and camera
    Choose different view (NUM1 – front view or NUM3 – side view)
    Click between cube and camera with LMB
    Rotate the view to see if it was positioned correctly
    1
    2
    3
    4
    Note: we’re working in a 3D space but only have a 2D screen – therefore you need two views to set all three coordinates of the cursor!
    5
  • Deleting and Adding Objects
    Andreas Jakl, 2009
    32
    Task: replace the cube with a monkey
    Set “Object Mode”; switch off “3d transform manipulator” (see previous slide)
    Move the cursor to the center (Shift+CKEY)
    RMB on the cube to select it
    DELKEY to delete the object. Confirm in the prompt window (“Erase Selected”)
    Use the Add menu to add a new object. Here: Add  Mesh  Monkey
    Blender automatically switches to edit mode. Go to object mode (TAB), press CKEY to center the cursor on the screen, ZKEY to toggle solid and wireframe mode. Zoom in (Mouse wheel)
    3
    4
    5
    6
  • Material
    Andreas Jakl, 2009
    33
    2
    3
    Task: change the material of the monkey
    Select the monkey object (RMB); set “Object Mode”
    Press the shading button
    In contrast to the cube, the monkey doesn’t have a default material. Click the button next to “Add New” and choose “O Material” (same that was originally on the cube)
    In the “Material” tab, set any color you want for “Col” and “Spe” (specularcolor, for highlights)
    Press the button with the small car in the “Links and Pipeline” tab for an automatic material name
    Set the draw type of the 3D Viewport to “Shaded” for a more accurate preview
    4
    5
    6
  • IDs and m3g Export
    Andreas Jakl, 2009
    34
    Task: to access the nodes from the m3g file in source code, they need IDs.
    Set “Object Mode”
    Select the monkey with the RMB
    Add #01 at the end of the object name to give it the ID 1 in the m3g file
    Do the same for the light (#02) and the camera (#03)
    Task: export the m3g file (requires the m3g exporter plug-in)
    File  Export  M3G
    You can safely disable texturing
  • m3g File
    Andreas Jakl, 2009
    35
    Open the m3g file in the m3g viewer* and choose Display  Scene Graph View
    Our IDs have been saved in the file
    * http://www.mascotcapsule.com/toolkit/m3g/en/index.php
  • Display the 3D scene
    Switching to the Java side
    Andreas Jakl, 2009
    36
  • Loading the m3g File
    Create a simple MIDlet and Canvas-class (sample start project is provided)
    Add the monkey.m3g to the root of the .jar
    Init code snippetof the Canvas:
    Object3D[] roots = null;
    try {
    // Load the m3g file
    // The loader will return all root level objects of the file,
    // which are not referenced by any other objects.
    // Always state an absolute path ("/")!
    roots = Loader.load("/monkey.m3g");
    } catch (IOException ex) {
    // couldn't open the file, or invalid data in the file
    ex.printStackTrace();
    }
    // Usually the world node is the only and first node.
    // If loading unknown files, you should check it though.
    // Save the world node to an instance variable
    for (inti = 0; i < roots.length; ++i) {
    if (roots[i] instanceof World) {
    iWorld = (World) roots[i];
    }
    }
    Andreas Jakl, 2009
    37
  • Game loop in a GameCanvas
    Andreas Jakl, 2009
    38
    public void run() {
    // Get the singleton Graphics3D instance that is associated with this midlet.
    Graphics3D g3d = Graphics3D.getInstance();
    // Measure the time that has passed since the prev. frame
    long start, elapsed = 0; int time = 0;
    while (iIsActive) {
    start = System.currentTimeMillis();
    try {
    // Bind the 3D graphics context to the given MIDP Graphics object.
    g3d.bindTarget(getGraphics());
    // Update the world [...]
    iWorld.animate(time); // Animate the world
    // Render the view from the active camera
    g3d.render(iWorld);
    } finally {
    // Release the graphics context
    g3d.releaseTarget();
    }
    flushGraphics(); // Flush the rendered image to the screen
    // Give other threads a chance to run
    Thread.sleep(20); // This sample omits try and catch
    elapsed = System.currentTimeMillis() - start;
    time += elapsed; // Time that has passed since the last frame
    }
    }
  • MIDP Relationship
    Andreas Jakl, 2009
    39
    Mobile 3D
    MIDP 2
    binds
    Graphics3D
    Graphics
    renders to
    renders
    renders
    World
    Screen
  • The Camera
    The camera was too far away in the Blender scene
    Move it towards the object after loading the world:
    Andreas Jakl, 2009
    40
    Camera cam = iWorld.getActiveCamera();
    cam.setTranslation(-5.0f, 5.0f, -4.0f);
  • Node Transformations
    Each node: local coordinate system
    Node transformation: from this node to the parent node (ignored for root)
    Translation T
    Rotation R
    Non-uniform scale S
    Generic matrix M
    Composite C = TRSM
    Andreas Jakl, 2009
    41
    C
    World
    C
    Group
    C
    C
    Sprite3D
    Group
    C
    C
    Group
    Mesh
  • Modifying Objects
    Rotate the monkey
    Andreas Jakl, 2009
    42
    private static final int ID_MONKEY = 1;
    // Find searches through all children of this node
    Node monkey = (Node) iWorld.find(ID_MONKEY);
    switch (getKeyStates())
    {
    case LEFT_PRESSED:
    // Parameters: angle, x / y / z component of
    // the rotation axis. Here: all on z axis
    monkey.postRotate(5.0f, 0.0f, 0.0f, -1.0f);
    break;
    case RIGHT_PRESSED:
    monkey.postRotate(5.0f, 0.0f, 0.0f, 1.0f);
    break;
    }
  • Objects and Materials
    What is actually rendered?
    Andreas Jakl, 2009
    43
  • Textures
    Add visual richness
    Backgrounds
    Surface structure
    Sprites for “cheating”(no complex 3D models)
    Andreas Jakl, 2009
    44
    Character images copyright David Dang
    http://www.chi-3d.co.uk/
    Sprites in “Doom” by id Software
  • Perspective Correction
    Perspective correction
    Expensive, avoid if possible
    But still better than a lot more polygons
    Nokia: 2% fixed overhead, 20% in the worst case
    Andreas Jakl, 2009
    45
    Perspective correction
    http://en.wikipedia.org/wiki/Texture_mapping
  • Light Maps
    Lightning
    Use light maps instead of real light
    Pre-calculate light for non-textured (colored) objects: vertex coloring
    Andreas Jakl, 2009
    46
    light map(unfiltered)
    light map(filtered)
    w/o light maps
    with light maps
    Images from lecture slides of Stephen Chenney (see sources at the end)
  • Mobile Textures II
    Backgrounds
    Use background images, no skybox or real 3D
    Sprites
    Much faster than real geometry or textured quads
    But too much overhead for particle effects
    Images
    Load through Loader class (Image2D) – going through MIDP Image wastes memory
    Andreas Jakl, 2009
    47
    Sprites in a 3D scene
    Playman World Soccer (Mr.Goodliving)
    3D Sprites can be useful for 2D games as well (rotation!)
    Tower Bloxx (Digital Chocolate)
  • IndexBuffer
    IndexBuffer
    Appearance
    Appearance
    Mesh
    How is the object defined? (vertices, material, ...)
    Andreas Jakl, 2009
    48
    Mesh
    Vertex positions, normals, texture coordinates, ...
    VertexBuffer
    Defines a submesh (consisting of triangle strips) – reference data from the vertex buffer
    IndexBuffer
    Appearance
    1 Appearance per submesh. Defines color, texture, ...
  • Sprite3D
    2D image with 3D position
    Fast, but functionally restricted alternative to textured geometry
    Scaled mode: billboards, trees, ...
    Unscaled mode: text labels, icons, ...
    Andreas Jakl, 2009
    49
    Sprite3D
    Appearance
    Contains compositing and fogging attributes
    Image2D
  • Animation
    Short overview of
    Andreas Jakl, 2009
    50
  • Animations
    Defined in the m3g file
    Automatically played corresponding to the time by animate() method
    Andreas Jakl, 2009
    51
    Defines time within the animation, speed, weight for blending, ...
    Animated Object (Object3D)
    AnimationTrack
    AnimationController
    AnimationTrack
    KeyframeSequence
    Each track associates a property (e.g. position) with a controler & keyframe data.
    Multiple tracks may be blended.
    Values of the animated property (keyframes) + the interpolation mode.
    KeyframeSequence
  • KeyframeSequence
    Andreas Jakl, 2009
    52
    KeyframeSequence
    Keyframe: time & property value at that time
    Sequence:
    Can store multiple keyframes
    Several interpolation methods
    Can be looping
    property value
    sequence time
    v
    t
    Diagram based on Sean Ellis, Superscape
  • KeyframeSequence
    Keyframe: time & property value at that time
    Sequence:
    Can store multiple keyframes
    Several interpolation methods
    Can be looping
    Andreas Jakl, 2009
    53
    KeyframeSequence
    property value
    sequence time
    Diagram based on Sean Ellis, Superscape
  • KeyframeSequence
    Keyframe: time & property value at that time
    Sequence:
    Can store multiple keyframes
    Several interpolation methods
    Can be looping
    Andreas Jakl, 2009
    54
    KeyframeSequence
    property value
    sequence time
    Diagram based on Sean Ellis, Superscape
  • KeyframeSequence
    Andreas Jakl, 2009
    55
    KeyframeSequence
    Keyframe: time & property value at that time
    Sequence:
    Can store multiple keyframes
    Several interpolation methods
    Can be looping
    property value
    sequence time
    v
    t
    Diagram based on Sean Ellis, Superscape
  • AnimationController
    Controls position, speed and weight of an animation sequence
    Usually controls multiple AnimationTracks (properties) at the same time
    Defines mapping from world time to sequence time
    Andreas Jakl, 2009
    56
    AnimationController
    world time
    sequence time
    0
    s
    d
    t1
    t2
    Diagram based on Sean Ellis, Superscape
  • Morphing
    Andreas Jakl, 2009
    57
    Base
    Target 1eyes closed
    Target 2
    mouth closed
    Animate eyesand mouth
    independently
    Unfortunately, the animations are not exported to the pdf
    Mesh
    MorphingMesh
    Images from Aarnio and Pulli (see sources at the end)
  • Skinning
    Andreas Jakl, 2009
    58
    No skinning
    Local skinning
    one bone per vertex
    Smooth skinning
    two bones per vertex
    Unfortunately, the animations are not exported to the pdf
    Mesh
    SkinnedMesh
    Images from Aarnio and Pulli (see sources at the end)
  • Summary
    M3G enables real-time 3D on mobile phones
    Works both with software and hardware 3D
    Supported by millions of devices
    Mixture of low- and high level
    Allows easy creation of scenes
    But still gives the developer full control
    Flexible architecture
    Based on OpenGL ES features
    Defines a convenient file format
    Andreas Jakl, 2009
    59
  • Sources
    TomiAarnio and Kari Pulli: Advanced Game Development with the Mobile 3D Graphics API. JavaOne Conference, 2004.
    Andrew Davison: Special Topics in Info. Eng.: Introductionto J2ME Programming. Lectureslides, 2007
    Qusay H. Mahmoud: Getting Started With the Mobile 3D Graphics API for J2ME. http://developers.sun.com/mobility/apis/articles/3dgraphics/
    Carol Hamer: Creating a basic M3G file with Blender. http://bittyjava.wordpress.com/2007/01/04/creating-a-basic-m3g-file-with-blender/
    David Millet, Arthur Tombs et al.: Blender 3D: Noob to Pro. http://en.wikibooks.org/wiki/Blender_3D:_Noob_to_Pro
    MikaelBaros: 3D programming tutorial for mobile devices using M3G (JSR 184). http://developer.sonyericsson.com/site/global/techsupport/tipstrickscode/mobilejava3d/p_java3d_tutorial_part1_compliments_redikod.jsp
    Kari Pulli et al.: Developing Mobile 3D Applications With OpenGL ES and M3G. Eurographics 2006. http://people.csail.mit.edu/kapu/mobile_3D_course/index.html
    JanneHellsten: Building scalable 3D apps - Tips & tricks for developers. http://www.khronos.org/developers/library/seoul_04_2005/Hybrid_Tips-and-Tricks.ppt
    Stephen Chenney: Computer Game Technology. Lecture slides, 2001. http://www.cs.wisc.edu/graphics/Courses/638-f2001/
    Andreas Jakl, 2009
    60
  • Thanks for your attention
    That’s it!
    Andreas Jakl, 2009
    61