3D Graphics on iPhone

3D Graphics on iPad, iPhone, and iPodTouch Douglass Turner douglass.turner@gmail.com @dugla 781 775 3708

twitter hash tag: #iphone3dnycmu thats ... # iphone 3d nyc mu

A talk in three acts ...

• Intro to Computer Graphics • Computer Graphics in OpenGL • OpenGL and CocoaTouch Playing Nice Together

Intro to Computer Graphics (a trip down the graphics pipeline)

Build and Tesselate Model Copyright Steve Jubinville

Pose Model Copyright Steve Jubinville

Aim Camera

Render Scene

Cull & clip model triangles to viewing frustrum.

Rasterize From models to pixels. Scanline interpolation. • Depth Comparison • Apply Appearance - Vertex level. Pixel level. • Composite Elements

Present the Image

Update Model State

Rinse, repeat.

Computer Graphics in OpenGL

OpenGL Cocoa

OpenGL is Old School

You are ﬂipping levers on a state machine.

float foo[] glPushMatrix() and glPopMatrix() Data Structures are for wimps.

OpenGL is a dragster. Not a Prius. Just buckle up ...

... and enjoy the ride, baby!

iPhone 3GS by the Numbers (via AnandTech http://bit.ly/1pQaJO) ~7M triangles / second ~230,000 realtime triangles (30fps) iPhone display: 320 * 480 = 153,600 pixels 230,000 / 153,600 = ~1.5 realtime-triangles / pixel ;-)

OpenGL websites are a bit different than Cocoa websites ...

Isn’t that sweet ...

Um... WTF?

Can’t you just feel the love?

Dude. I think my eyes are bleeding.

No worries. safari.oreilly.com is your friend

Hierarchical Modeling. Transformation Stacks. Coordinate Frames. Pose Model

Pose Model

You are already familiar with hierarchical modeling and coordinate frames.

[self.parentView addSubview:childView]; or ... for (UIView *v in self.containerView.subviews) { CGRect bounds = [v convertRect:v.bounds toView:scrollView]; if (CGRectIntersectsRect(bounds, scrollView.bounds)) { // do stuff ... } else { // do other stuff ... } } // for (self.containerView.subviews) You are already familiar with hierarchical modeling and coordinate frames.

- (void)setTransform:(CGAffineTransform)newValue { // Scale along x-axis only CGAffineTransform constrainedTransform = CGAffineTransformScale(CGAffineTransformIdentity, newValue.a, 1.0); newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d; [super setTransform:newValue]; } You are already familiar with transformations.

HelloTeapot Demo Source: http://github.com/turner/HelloTeapot

HelloTeapot Code Walkthrough http://github.com/turner/HelloTeapot

http://github.com/turner/HelloTeapot

Cocoa Touch: [myView setNeedsDisplay] and [myView drawRect] OpenGL: - (void)startAnimation { animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES]; }

// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); // Position Model glPushMatrix(); JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix(); GLViewController.m

Pose Model // Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); // Position Model glPushMatrix(); JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix(); GLViewController.m

Light Scene // Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); // Position Model glPushMatrix(); JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix(); GLViewController.m

// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); Aim Camera ... // Position Model glPushMatrix(); Dude, where is the camera ...? JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix();

camera teapot wTc wTt world

cT t camera teapot ? glMatrixMode(GL_MODELVIEW); glLoadMatrixf(_openGLCameraInverseTransform);

cT t camera teapot -1 (wTc) wTt world cT t = cT w * wTt

cT t = cT w * wTt

// Position Camera glMatrixMode(GL_MODELVIEW); cT t = cT w * wTt glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); // Position Model glPushMatrix(); JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix(); GLViewController.m

- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up; GLViewController.m

Render float teapot_vertices [] = { 0.0663, 0.1178, 0.0, 0.0672, 0.1152, 0.0, 0.0639, 0.1178, 0.0178043, ... }; float teapot_normals[] = { -0.987635, -0.156768, 0, -0.902861, -0.429933, 0, -0.953562, -0.156989, -0.257047, ... }; short indicies[] = { // howmany vertices in vertex strip 26, // vertex strip indices 1122, 1243, 1272, 1242, ... ,1283, 1199, ... }; teapot.h

Render glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices); glEnableClientState(GL_NORMAL_ARRAY); glNormalPointer(GL_FLOAT, 0, teapot_normals); glEnable(GL_NORMALIZE); GLViewController.m

Render // Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform); // Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); glPopMatrix(); // Position Model glPushMatrix(); JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation); // Render for(int i = 0; i < num_teapot_indices; i += indicies[i] + 1) { glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]); } // for (num_teapot_indices) glPopMatrix(); GLViewController.m

invert( camera_transform ) red_light green_light blue_light push() camera_transform white_spotlight pop() push() teapot_transform draw() pop()

You’ll want to understand this stuff if you intend to do AR* apps like all the cool kids. *Augmented Reality - See http://layar.com

Design Appearance

Mipmapped textures are your friend. They will become your workhorse for visual complexity.

HelloTexture Demo http://github.com/turner/HelloTexture

glGenTextures(1, &_name); glBindTexture(GL_TEXTURE_2D, _name); glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexImage2D(GL_TEXTURE_2D, 0, glColor, _w, _h, 0, glColor, glFormat, data); glGenerateMipmapOES(GL_TEXTURE_2D); Class TEITexture.m Method initWithTextureFile:mipmap:

Two Words Texture. Atlas.

t s Texture Space

HelloParticleSystem Demo Texture Atlas Hacks

OpenGL ES 2.0 == GLSL (OpenGL Shading Language) Shaders Take it to Another Level ...

ZBrush Modeling Daniel Lieske Shaders Take it to Another Level

ZBrush Modeling Daniel Lieske Shaders Take it to Another Level Material Design Ralf Stumpf

Shaders Take it to Another Level Copyright: Stephen Molyneaux

GLSL is cool ... Sadly, OpenGL ES 2.0 (iPhone 3Gs) does not support hardware anti-aliasing. Stick with mipmapped textures until you really know what you are doing. Or don’t care about aliasing artifacts ...

1 sample per pixel 4 jittered/ﬁltered samples per pixel

Remember the Apple Imperitive ... Your iPhone app will be beautiful. Your iPad app will be georgous.

OpenGL and Cocoa Touch Playing Nice Together

MVC • OpenGL is mostly M • Keep V dumb • Lots of chatter between M and C See: • GLGravity - http://bit.ly/2dS8YI • Jeff LaMarche OpenGL ES Template - http://bit.ly/2IvZtV

Touch Sequence & Accelerometer

HelloParticleSystem Demo • Touch Sequence • Accelerometer

Touch Sequence On iTunes: Stanford CS 193P - Lecture 14

Initialize model state touchesBegan:withEvent: Evolve model state touchesMoved:withEvent: Clean up model state touchesEnded:withEvent: touchesCancelled:withEvent:

Accelerometer Note: accelerometer-simulator is your friend: http://code.google.com/p/accelerometer-simulator

Gravitynew = (RawAcc * k) + Gravityold * (1 - k) #define kFilteringFactor ( 0.1) - (void)accelerometer:(UIAccelerometer*)accelerometer didAccelerate:(UIAcceleration*)acceleration { // Compute "G". Use low-pass filter to attenuate instantaneous acceleration. self.accelerationValueX = acceleration.x * kFilteringFactor + self.accelerationValueX * (1.0 - kFilteringFactor); self.accelerationValueY = acceleration.y * kFilteringFactor + self.accelerationValueY * (1.0 - kFilteringFactor); self.accelerationValueZ = acceleration.z * kFilteringFactor + self.accelerationValueZ * (1.0 - kFilteringFactor); // ParticleSystem particles live in flatland. Use x and y compoments of "G". [ParticleSystem setGravity:CGPointMake(self.accelerationValueX, self.accelerationValueY)]; } UIAccelerometerDelegate

+ (void)setGravity:(CGPoint)gravityVector { // Normalize 2D “G” vector. Only care about direction. float length = sqrtf(gravityVector.x * gravityVector.x + gravityVector.y * gravityVector.y); ParticleSystemGravity.x = gravityVector.x / length; ParticleSystemGravity.y = gravityVector.y / length; // Flip y-component of gravity vector to be consistent with screen space coordinate // system used in ParticleSystem. ParticleSystemGravity.y = -(ParticleSystemGravity.y); } ParticleSystem - Class State

// dv = dv/dt * dt float dv_x = (ParticleSystemGravity.x * gravityScaleFactor * dt); float dv_y = (ParticleSystemGravity.y * gravityScaleFactor * dt); particle.velocity = CGPointMake(particle.velocity.x + dv_x, particle.velocity.y + dv_y); // dx = dx/dt * dt float dx = particle.velocity.x * dt; float dy = particle.velocity.y * dt; // x = x_old * dx particle.location = CGPointMake(particle.location.x + dx, particle.location.y + dy); ParticleSystem - Instance State

Cocoa Touch: [myView setNeedsDisplay] and [myView drawRect] OpenGL: - (void)startAnimation { animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES]; }

iPhone is an Expression Platform You Must See: Stanford cs 193p Lecture 17 Video | On iTunes: http://bit.ly/PUrGj Creating New Expressive Social Mediums on the iPhone Ge Wang | CCO & CTO | Smule

iPhone is a general purpose sensing platform • Touch • Sound • Proximity • Cloud • Camera • INU (almost) • Geography

iPhone encourages plausible, hyper-real, extensions of human experience. Be interesting. Be brilliant.

Resources • Website: http://theelasticimage.posterous.com • This Presentation on SlideShare: http://bit.ly/Y1MW8 • HelloTeapot Source Code: http://github.com/turner/HelloTeapot • HelloTexture Source Code: http://github.com/turner/HelloTexture

I am available for collaboration on iPad, iPodTouch, and iPhone projects. I am also available for coaching and/or presentation of this material to your team or organizaton.

Thank You

http://iphone.keyvisuals.com	1507
http://imho.nu	194
http://www.slideshare.net	95
http://c4c.posterous.com	40
http://www.imho.nu	14
http://translate.googleusercontent.com	12
http://www.iphonepundits.com	4
http://appliant1.rssing.com	3
http://appliant_1.rssing.com	2
http://webcache.googleusercontent.com	2
http://localhost	2
http://openreviews.org	2
http://twitter.com	1
http://74.125.93.132	1
http://trunk.ly	1
http://74.125.47.132	1
http://www.rue89.com	1
http://posterous.com	1

3D Graphics on iPhone

by Douglass Turner on Nov 19, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

18 Embeds 1,883

Statistics

3D Graphics on iPhone — Presentation Transcript