Elastic Image Software LLCiOS OpenGLDouglass TurnerElastic Image Softwareemail: douglass.turner@gmail.commobile: 781 775 3...
Elastic Image Software LLC2A Talk in Three Acts
Elastic Image Software LLC• Introduction to Computer Graphics• Computer Graphics in OpenGL• OpenGL and CocoaTouch Playing ...
Elastic Image Software LLC4Introduction to Computer GraphicsA Trip Down the Graphics Pipeline
Elastic Image Software LLCCopyright Steve Jubinville5Build and Tesselate Model into Facets
Elastic Image Software LLCCopyright Steve Jubinville6Pose Model
Elastic Image Software LLCAim Camera7
Elastic Image Software LLC8Design the Look
Elastic Image Software LLCDesign Appearance. Light Scene.Copyright © 2006 Holger Schömann9
Elastic Image Software LLCCopyright © 2006 Florian WildDesign Appearance. Light Scene.10
Elastic Image Software LLCCopyright © 2006 Jack QiaoDesign Appearance. Light Scene.11
Elastic Image Software LLC12Render Scene
Elastic Image Software LLC13Clip scene to the bounds of the viewing frustrum.
Elastic Image Software LLCRasterize Scene14
Elastic Image Software LLC15for each object in the scene! determine its corresponding pixels! for each corresponding pixel...
Elastic Image Software LLCIn OpenGL, the Look is evaluated at the same time thatdepth comparison is performed.Together the...
Elastic Image Software LLC17Update Model State. This could entail geometrictransformation or perhaps taking a time step in...
Elastic Image Software LLC18Rinse, repeat.
Elastic Image Software LLC19Example.Beautiful Panoramas iPad app. In the App Store: http://bit.ly/QVIDgg
Elastic Image Software LLC20Computer Graphics in OpenGL
Elastic Image Software LLCOpenGL Cocoa21
Elastic Image Software LLCOpenGL is Old School22
Elastic Image Software LLCYou are flipping levers on a large state machine.23
Elastic Image Software LLCOpenGL is brutally spare. Just the essentials.float oldSchool[]glPushMatrix()glPopMatrix()!24
Elastic Image Software LLCKeep in mind OpenGL is a dragster, not a Prius. So, just buckle up ...25
Elastic Image Software LLC... and enjoy the ride, baby!26
Elastic Image Software LLC27Posing models. Positioning light sources.Aiming the camera. All these task requireand understa...
Elastic Image Software LLC28Transformation matrix. Coordinate frameWorld space, Screen space, etc.All are different ways o...
Elastic Image Software LLC29When we move an object we apply atransformation via a transformationmatrix. This transformatio...
Elastic Image Software LLC30The motivation for a coordinate frametransformation is not only motivated by the needto pose a...
Elastic Image Software LLC31We refer to the transformation pipeline: the successiveapplication of a series of transformati...
Elastic Image Software LLCHierarchical modeling is based on the hierarchicalorganization of transformations matrices. Many...
Elastic Image Software LLC33The UIView class supports all the key conceptsof transformations and hierarchy. It is a nice,f...
Elastic Image Software LLC34Hierarchical modeling in Cocoa Touch
Elastic Image Software LLC35struct CGAffineTransform {CGFloat a, b, c, d;CGFloat tx, ty;};struct CATransform3D {CGFloat m1...
Elastic Image Software LLC36UIView properties// Coordinate frame in parent space@property(nonatomic) CGRect frame;// Dimen...
Elastic Image Software LLC37UIView methods// Transform 2D point from/to another view’s coordinate frame- (CGPoint)convertP...
Elastic Image Software LLC38- (BOOL)application:(UIApplication *)applicationdidFinishLaunchingWithOptions:(NSDictionary *)...
Elastic Image Software LLC39Transformation matrices in iOS- (void)setTransform:(CGAffineTransform)newValue {	 // Scale alo...
Elastic Image Software LLCHelloTeapot Demo and Code Walkthroughhttp://github.com/turner/HelloTeapot40
Elastic Image Software LLCIn what follows, we will be referring to the ES1“fixed-function” transformation pipeline provided...
Elastic Image Software LLC42
Elastic Image Software LLC43A camera with a spotlight attached to it orbitsaround a stationary teapot model lit by threest...
Elastic Image Software LLCIn iOS we never explicitly say “render scene”. Rather weinform the system that now would be a go...
Elastic Image Software LLC- (void)startAnimation {animationTimer =[NSTimer scheduledTimerWithTimeInterval:animationInterva...
Elastic Image Software LLC46In OpenGL we manage rendering ourselves- (void)startAnimation {!if (!self.isAnimating) {! !sel...
Elastic Image Software LLC! Position the camera! glMatrixMode(GL_MODELVIEW);!! glLoadIdentity();! glLoadMatrixf(_openGLCam...
Elastic Image Software LLCPosition spotlight at camera location	 glPushMatrix();! glMultMatrixf(_cameraTransform);!!! // A...
Elastic Image Software LLC49glPushMatrix & glPopMatrixThese functions push/pop a transformation matrix on/off thetransform...
Elastic Image Software LLC50glPushMatrix - Create a new matrix and pushit on the stack. Subsequent matrix concatenationis ...
Elastic Image Software LLC51http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the red st...
Elastic Image Software LLC52http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the green ...
Elastic Image Software LLC53http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the blue s...
Elastic Image Software LLC54http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw t...
Elastic Image Software LLC55http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw t...
Elastic Image Software LLC56http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw t...
Elastic Image Software LLC57Notice the absence of GL_CAMERA?You may be surprised to learn there is no concept ofcamera in ...
Elastic Image Software LLCcamera teapotworldwTtwTc58
Elastic Image Software LLCcamera teapotglMatrixMode(GL_MODELVIEW);glLoadMatrixf(_openGLCameraInverseTransform);?cTt59
Elastic Image Software LLCcamera teapotworldwTt(wTc)cTtcTt wTtcTw *=-160
Elastic Image Software LLCcTt wTtcTw *=61
Elastic Image Software LLC- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)u...
Elastic Image Software LLCThe vertices comprising the 3D geometry of the teapotfloat teapot_vertices [] = {0.0663, 0.1178,...
Elastic Image Software LLCThe teapot model is defined as a collection of triangle stripesthat combine to form the teapot s...
Elastic Image Software LLCglEnableClientState(GL_VERTEX_ARRAY);glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices);glEnableCl...
Elastic Image Software LLCDesign the Look66
Elastic Image Software LLC67We now begin leaving behind the fixed-function pipeline of ES1 and focus onES2, GLSL, and visua...
Elastic Image Software LLCThe Look: the lighting, illumination model,and surface shaders that in combinationachieve the de...
Elastic Image Software LLCLights69
Elastic Image Software LLC70Point Light Directional LightSpot Light Area Light
Elastic Image Software LLC71
Elastic Image Software LLCIllumination Model72
Elastic Image Software LLCAmbient DiffuseSpecularDiffuse+Specular73
Elastic Image Software LLCDiffuse(NormalVector, LightVector)Ambient(ConstantReflectance)Specular(LightReflectionVector, EyeV...
Elastic Image Software LLCSurface Shaders75
Elastic Image Software LLCSurface shaders were introduced to the field of computer graphicsby Pixar with its rendering API ...
Elastic Image Software LLCCopyright © Douglass Turner77I created this gallery of shaders using a rendering/shadingsystem I...
Elastic Image Software LLC78Copyright © Douglass Turner
Elastic Image Software LLC ZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf79
Elastic Image Software LLCCopyright: Stephen Molyneaux80
Elastic Image Software LLCHello Shader Demo and Code Walkthroughhttps://github.com/turner/HelloShader81
Elastic Image Software LLC82Before we dive into shaders lets get a lay of the land. There are twoflavors of shaders in GLSL...
Elastic Image Software LLC83Vertex and fragment shaders work together in a pipeline fashion. Vertexattributes - color, sur...
Elastic Image Software LLC84ShowST. A shader that visualizes the texturecoordinate parameterization of a surface.
Elastic Image Software LLC85https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh// ShowST.vsha...
Elastic Image Software LLC86attribute vec2 myVertexST - Vertex attributevarying vec2 v_st - Varies across the surfaceunifo...
Elastic Image Software LLC87https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fshVertex attribu...
Elastic Image Software LLC88https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh// ShowST.fshv...
Elastic Image Software LLC89http://github.com/turner/HelloiPadGLSL/blob/master/Shaders/ShowST.fsh// ShowST.fshvarying vec2...
Elastic Image Software LLC90TexturePairShader. A shader that mixes thecolors of a pair of textures using the tweakedalpha ...
Elastic Image Software LLC91TexturePairShader - vertex shaderhttps://github.com/turner/HelloShader/blob/master/HelloShader...
Elastic Image Software LLC92TexturePairShader - vertex shader// TEITexturePairShader.vshattribute mediump	 vec2	 myVertexS...
Elastic Image Software LLC93TexturePairShader - fragment shader// TEITexturePairShader.fshvarying	 mediump vec2 v_st;unifo...
Elastic Image Software LLC94attribute vec2 myVertexST - Vertex attributevarying vec2 v_st - Varies across the surfaceunifo...
Elastic Image Software LLC95An application communicates with its shaders in the follow ways:• attributes - geometry, color...
Elastic Image Software LLC96Lets look at how we wire our iOS app togetherwith our shaders. We will use a texture shadingex...
Elastic Image Software LLC97First make a texture object ...TEITexture *t =[[ [TEITexture alloc] initWithImageFile:@"twitte...
Elastic Image Software LLC98Load shaders, Bind shader attribute to application handle.// Vertex shaderNSString *vertShader...
Elastic Image Software LLC99Associate shader uniform with application object instance ...TEITexture *t =(TEITexture *)[sel...
Elastic Image Software LLC100We now have a linkage between our application texture object and a texturesampler in our frag...
Elastic Image Software LLC101
Elastic Image Software LLC102
Elastic Image Software LLC103Lets Talk Textures
Elastic Image Software LLC104Texture mapping is the fundamental tool for creating visual complexity
Elastic Image Software LLC105By establishing a mapping between a surface and a texture we can “attach”the texture to the s...
Elastic Image Software LLC106
Elastic Image Software LLC107Shaders can have it both ways: Compute surfacecolor (or surface roughness, or opacity) algori...
Elastic Image Software LLCOpenGL and iOSPlaying Nice Together108
Elastic Image Software LLC• OpenGL is mostly M• Keep V dumb• Lots of chatter between M and CMVC109
Elastic Image Software LLC110Lets take a look at how iOS and OpenGL interact. We will usHello iPad OpenGL as our guide.htt...
Elastic Image Software LLC111-(id)initializeEAGL {!! CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer;!! eaglLayer.opaqu...
Elastic Image Software LLC112Rendering frame rate is sync-ed to the screen refresh rate.The actual rendering is handled vi...
Elastic Image Software LLC113	 // framebuffer	 glGenFramebuffers(1, &m_framebuffer);	 glBindFramebuffer(GL_FRAMEBUFFER, m_...
Elastic Image Software LLC114- (void) render {! ![EAGLContext setCurrentContext:m_context];glBindFramebuffer(GL_FRAMEBUFFE...
Elastic Image Software LLCThank YouDouglass TurnerElastic Image Softwareemail: douglass.turner@gmail.commobile: 781 775 37...
Elastic Image Software LLC116Copyright © Douglass TurnerElastic Image Software LLC
Upcoming SlideShare
Loading in...5
×

iOS OpenGL

14,166

Published on

Original Presentation: http://bit.ly/Lm8DyH
Tweets: @dugla
Email here: douglass.turner@gmail.com

Published in: Technology, Art & Photos
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
14,166
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
236
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

iOS OpenGL

  1. 1. Elastic Image Software LLCiOS OpenGLDouglass TurnerElastic Image Softwareemail: douglass.turner@gmail.commobile: 781 775 3708
  2. 2. Elastic Image Software LLC2A Talk in Three Acts
  3. 3. Elastic Image Software LLC• Introduction to Computer Graphics• Computer Graphics in OpenGL• OpenGL and CocoaTouch Playing Nice Together3
  4. 4. Elastic Image Software LLC4Introduction to Computer GraphicsA Trip Down the Graphics Pipeline
  5. 5. Elastic Image Software LLCCopyright Steve Jubinville5Build and Tesselate Model into Facets
  6. 6. Elastic Image Software LLCCopyright Steve Jubinville6Pose Model
  7. 7. Elastic Image Software LLCAim Camera7
  8. 8. Elastic Image Software LLC8Design the Look
  9. 9. Elastic Image Software LLCDesign Appearance. Light Scene.Copyright © 2006 Holger Schömann9
  10. 10. Elastic Image Software LLCCopyright © 2006 Florian WildDesign Appearance. Light Scene.10
  11. 11. Elastic Image Software LLCCopyright © 2006 Jack QiaoDesign Appearance. Light Scene.11
  12. 12. Elastic Image Software LLC12Render Scene
  13. 13. Elastic Image Software LLC13Clip scene to the bounds of the viewing frustrum.
  14. 14. Elastic Image Software LLCRasterize Scene14
  15. 15. Elastic Image Software LLC15for each object in the scene! determine its corresponding pixels! for each corresponding pixel! ! calculate depth of object at pixel! ! if depth is closer than any object yet seen! ! ! put this object in the pixel! ! endif! endendThe GPU implements a z-buffer algorithm in hardware
  16. 16. Elastic Image Software LLCIn OpenGL, the Look is evaluated at the same time thatdepth comparison is performed.Together these are the core activities of computer graphics.They are the raison dêtre of the GPU. Compute intensive,Highly optimized and parallelized.16
  17. 17. Elastic Image Software LLC17Update Model State. This could entail geometrictransformation or perhaps taking a time step inthe physic engine.
  18. 18. Elastic Image Software LLC18Rinse, repeat.
  19. 19. Elastic Image Software LLC19Example.Beautiful Panoramas iPad app. In the App Store: http://bit.ly/QVIDgg
  20. 20. Elastic Image Software LLC20Computer Graphics in OpenGL
  21. 21. Elastic Image Software LLCOpenGL Cocoa21
  22. 22. Elastic Image Software LLCOpenGL is Old School22
  23. 23. Elastic Image Software LLCYou are flipping levers on a large state machine.23
  24. 24. Elastic Image Software LLCOpenGL is brutally spare. Just the essentials.float oldSchool[]glPushMatrix()glPopMatrix()!24
  25. 25. Elastic Image Software LLCKeep in mind OpenGL is a dragster, not a Prius. So, just buckle up ...25
  26. 26. Elastic Image Software LLC... and enjoy the ride, baby!26
  27. 27. Elastic Image Software LLC27Posing models. Positioning light sources.Aiming the camera. All these task requireand understanding of transformationmatrices.
  28. 28. Elastic Image Software LLC28Transformation matrix. Coordinate frameWorld space, Screen space, etc.All are different ways of thinking aboutthe same underlying concept.
  29. 29. Elastic Image Software LLC29When we move an object we apply atransformation via a transformationmatrix. This transformation is defined interms of a coordinate frame.A specific location on the transformedobject can be interpreted in terms of anycoordinate frame we choose.
  30. 30. Elastic Image Software LLC30The motivation for a coordinate frametransformation is not only motivated by the needto pose a model or aim a camera as in thephysical world.In computer graphics land it often has to do withthe convenience of performing a particularcalculation in one space rather then another.
  31. 31. Elastic Image Software LLC31We refer to the transformation pipeline: the successiveapplication of a series of transformations as a 3D model wendsits way from application land to GPU land.
  32. 32. Elastic Image Software LLCHierarchical modeling is based on the hierarchicalorganization of transformations matrices. Many objects- trees, robots, cars - form natural hierarchies.32
  33. 33. Elastic Image Software LLC33The UIView class supports all the key conceptsof transformations and hierarchy. It is a nice,friendly way to gain intuition about facility withthis powerful concept.
  34. 34. Elastic Image Software LLC34Hierarchical modeling in Cocoa Touch
  35. 35. Elastic Image Software LLC35struct CGAffineTransform {CGFloat a, b, c, d;CGFloat tx, ty;};struct CATransform3D {CGFloat m11, m12, m13, m14;CGFloat m21, m22, m23, m24;CGFloat m31, m32, m33, m34;CGFloat m41, m42, m43, m44;};Transforms
  36. 36. Elastic Image Software LLC36UIView properties// Coordinate frame in parent space@property(nonatomic) CGRect frame;// Dimensions of view in object space@property(nonatomic) CGRect bounds;// View center in parent space@property(nonatomic) CGPoint center;// View transformation matrix@property(nonatomic) CGAffineTransform transform;// Parent view@property(nonatomic,readonly) UIView *superview;// Child views@property(nonatomic,readonly,copy) NSArray *subviews;
  37. 37. Elastic Image Software LLC37UIView methods// Transform 2D point from/to another view’s coordinate frame- (CGPoint)convertPoint:(CGPoint)point toView:(UIView *)view;- (CGPoint)convertPoint:(CGPoint)point fromView:(UIView *)view;// Transform rectangle from/to another view’s coordinate frame- (CGRect)convertRect:(CGRect)rect toView:(UIView *)view;- (CGRect)convertRect:(CGRect)rect fromView:(UIView *)view;// Manipulate view hierarchy- (void)removeFromSuperview;- (void)insertSubview:(UIView *)view atIndex:(NSInteger)index;- (void)addSubview:(UIView *)view;- (void)insertSubview:(UIView *)view belowSubview:(UIView *)siblingSubview;- (void)insertSubview:(UIView *)view aboveSubview:(UIView *)siblingSubview;
  38. 38. Elastic Image Software LLC38- (BOOL)application:(UIApplication *)applicationdidFinishLaunchingWithOptions:(NSDictionary *)launchOptions {[window addSubview:viewController.view];[window makeKeyAndVisible]; return YES;}View hierarchy in iOS
  39. 39. Elastic Image Software LLC39Transformation matrices in iOS- (void)setTransform:(CGAffineTransform)newValue { // Scale along the y-axis only CGAffineTransform constrainedTransform =CGAffineTransformScale(CGAffineTransformIdentity, 1.0, newValue.a); newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d; [super setTransform:newValue]; }
  40. 40. Elastic Image Software LLCHelloTeapot Demo and Code Walkthroughhttp://github.com/turner/HelloTeapot40
  41. 41. Elastic Image Software LLCIn what follows, we will be referring to the ES1“fixed-function” transformation pipeline providedby the GPU.In ES2 (GLSL) the transformation pipeline mustbe handled entirely in application space.41
  42. 42. Elastic Image Software LLC42
  43. 43. Elastic Image Software LLC43A camera with a spotlight attached to it orbitsaround a stationary teapot model lit by threestationary colored point light sources.
  44. 44. Elastic Image Software LLCIn iOS we never explicitly say “render scene”. Rather weinform the system that now would be a good time toredisplay the scene:[myView setNeedsDisplay] and [myView drawRect]44
  45. 45. Elastic Image Software LLC- (void)startAnimation {animationTimer =[NSTimer scheduledTimerWithTimeInterval:animationIntervaltarget:selfselector:@selector(drawView)userInfo:nilrepeats:YES];}45In OpenGL we manage rendering ourselves
  46. 46. Elastic Image Software LLC46In OpenGL we manage rendering ourselves- (void)startAnimation {!if (!self.isAnimating) {! !self.displayLink =[NSClassFromString(@"CADisplayLink") displayLinkWithTarget:selfselector:@selector(drawView:)];[self.displayLink setFrameInterval:animationFrameInterval];[self.displayLink addToRunLoop:[NSRunLoop currentRunLoop]forMode:NSDefaultRunLoopMode];! ! !self.animating = YES;! !} // if (!self.isAnimating)!}
  47. 47. Elastic Image Software LLC! Position the camera! glMatrixMode(GL_MODELVIEW);!! glLoadIdentity();! glLoadMatrixf(_openGLCameraInverseTransform);47http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
  48. 48. Elastic Image Software LLCPosition spotlight at camera location glPushMatrix();! glMultMatrixf(_cameraTransform);!!! // A white spotlight for a camera headlight!GLfloat spotLightColor[] = { 1.0, 1.0, 1.0, 1.0 };GLfloat spotLightPosition[] = { 0.0, 0.0, 2.0, 1.0 };GLfloat spotLightDirection[] = { 0.0, 0.0, -1.0, 1.0 };GLfloat spotCutOff!! ! = 3.5;!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLightColor);!! glLightfv(GL_LIGHT3, GL_POSITION, spotLightPosition);! glLightfv(GL_LIGHT3, GL_SPOT_DIRECTION, spotLightDirection);! glLightf(GL_LIGHT3, GL_SPOT_CUTOFF, spotCutOff);!! glPopMatrix();!48http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
  49. 49. Elastic Image Software LLC49glPushMatrix & glPopMatrixThese functions push/pop a transformation matrix on/off thetransformation stack.This stack is the key data structure for presenting ahierarchical model to the GPU in fixed-function OpenGL.
  50. 50. Elastic Image Software LLC50glPushMatrix - Create a new matrix and pushit on the stack. Subsequent matrix concatenationis done with stack.top.stack.top is the current coordinate frame.glPopMatrix - Pop the stack. The pre-existingmatrix is now stack.top.We revert to the coordinate frame the existedprior to the push.! glPushMatrix();! glMultMatrixf(_cameraTransform);!! glEnable(GL_LIGHT3);! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);! glPopMatrix();!
  51. 51. Elastic Image Software LLC51http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the red stationary light source! const GLfloat redLightColor[]! ! = { 1.0f, 0.0f, 0.0f, 1.0f };! const GLfloat redLightPosition[]! = { 10.0f, 0.0f, 0.0f, 1.0f };! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, redLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, redLightPosition);
  52. 52. Elastic Image Software LLC52http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the green stationary light source! const GLfloat greenLightColor[]! = { 0.0f, 1.0f, 0.0f, 1.0f };! const GLfloat greenLightPosition[]!= { 0.0f, 10.0f, 0.0f, 1.0f };! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, greenLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, greenLightPosition);
  53. 53. Elastic Image Software LLC53http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition the blue stationary light source! const GLfloat blueLightColor[]!! = { 0.0f, 0.0f, 1.0f, 1.0f };! const GLfloat blueLightPosition[]! = { 0.0f, 0.0f, 10.0f, 1.0f };! glEnable(GL_LIGHT0);! glLightfv(GL_LIGHT0, GL_DIFFUSE, blueLightColor);!! glLightfv(GL_LIGHT0, GL_POSITION, blueLightPosition);
  54. 54. Elastic Image Software LLC54http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw the teapot! glPushMatrix();! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {! ! glDrawElements(GL_TRIANGLE_STRIP, indices[i],GL_UNSIGNED_SHORT, &indices[i+1]);! } // for (num_teapot_indices)! glPopMatrix();
  55. 55. Elastic Image Software LLC55http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw the teapot! glPushMatrix();! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {! ! glDrawElements(GL_TRIANGLE_STRIP, indices[i],GL_UNSIGNED_SHORT, &indices[i+1]);! } // for (num_teapot_indices)! glPopMatrix();
  56. 56. Elastic Image Software LLC56http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.mPosition and draw the teapot! glPushMatrix();! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f);! JLMMatrix3DSetScaling(scale, sx, sy, sz);! JLMMatrix3DMultiply(rotation, scale, concatenation);! glMultMatrixf(concatenation);! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {! ! glDrawElements(GL_TRIANGLE_STRIP, indices[i],GL_UNSIGNED_SHORT, &indices[i+1]);! } // for (num_teapot_indices)! glPopMatrix();
  57. 57. Elastic Image Software LLC57Notice the absence of GL_CAMERA?You may be surprised to learn there is no concept ofcamera in OpenGL. Camera posing is equivalent toinverting the camera transform and applying it to thescene observed by the camera.
  58. 58. Elastic Image Software LLCcamera teapotworldwTtwTc58
  59. 59. Elastic Image Software LLCcamera teapotglMatrixMode(GL_MODELVIEW);glLoadMatrixf(_openGLCameraInverseTransform);?cTt59
  60. 60. Elastic Image Software LLCcamera teapotworldwTt(wTc)cTtcTt wTtcTw *=-160
  61. 61. Elastic Image Software LLCcTt wTtcTw *=61
  62. 62. Elastic Image Software LLC- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up;62http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m
  63. 63. Elastic Image Software LLCThe vertices comprising the 3D geometry of the teapotfloat teapot_vertices [] = {0.0663, 0.1178, 0.0,0.0672, 0.1152, 0.0,0.0639, 0.1178, 0.0178043, ... };The surface normals associated with each vertex// The list of surface normals corresponding to the verticesfloat teapot_normals[] = {-0.987635, -0.156768, 0,-0.902861, -0.429933, 0,-0.953562, -0.156989, -0.257047, ... };63
  64. 64. Elastic Image Software LLCThe teapot model is defined as a collection of triangle stripesthat combine to form the teapot surface.short indicies[] = {// how many vertices in vertex strip26,// vertex strip indices1122, 1243, 1272, 1242, ... ,1283, 1199,...};64
  65. 65. Elastic Image Software LLCglEnableClientState(GL_VERTEX_ARRAY);glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices);glEnableClientState(GL_NORMAL_ARRAY);glNormalPointer(GL_FLOAT, 0, teapot_normals);glEnable(GL_NORMALIZE);!65! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {! ! glDrawElements(GL_TRIANGLE_STRIP, indices[i],GL_UNSIGNED_SHORT, &indices[i+1]);! }
  66. 66. Elastic Image Software LLCDesign the Look66
  67. 67. Elastic Image Software LLC67We now begin leaving behind the fixed-function pipeline of ES1 and focus onES2, GLSL, and visual coolness ingeneral.
  68. 68. Elastic Image Software LLCThe Look: the lighting, illumination model,and surface shaders that in combinationachieve the desired look of a model in ascene.68
  69. 69. Elastic Image Software LLCLights69
  70. 70. Elastic Image Software LLC70Point Light Directional LightSpot Light Area Light
  71. 71. Elastic Image Software LLC71
  72. 72. Elastic Image Software LLCIllumination Model72
  73. 73. Elastic Image Software LLCAmbient DiffuseSpecularDiffuse+Specular73
  74. 74. Elastic Image Software LLCDiffuse(NormalVector, LightVector)Ambient(ConstantReflectance)Specular(LightReflectionVector, EyeVector) Shiny(Specular, Diffuse)74
  75. 75. Elastic Image Software LLCSurface Shaders75
  76. 76. Elastic Image Software LLCSurface shaders were introduced to the field of computer graphicsby Pixar with its rendering API and shading language calledRenderMan.A shader is a small functions evaluated at every location on abeing rendered.GLSL borrows heavily from the RenderMan model.76
  77. 77. Elastic Image Software LLCCopyright © Douglass Turner77I created this gallery of shaders using a rendering/shadingsystem I wrote that was inspired by RenderMan.
  78. 78. Elastic Image Software LLC78Copyright © Douglass Turner
  79. 79. Elastic Image Software LLC ZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf79
  80. 80. Elastic Image Software LLCCopyright: Stephen Molyneaux80
  81. 81. Elastic Image Software LLCHello Shader Demo and Code Walkthroughhttps://github.com/turner/HelloShader81
  82. 82. Elastic Image Software LLC82Before we dive into shaders lets get a lay of the land. There are twoflavors of shaders in GLSL.A vertex shader is evaluated at each vertex.A fragment shader is evaluated at each screen space pixelcorresponding to a sampled on the facet being rasterized.
  83. 83. Elastic Image Software LLC83Vertex and fragment shaders work together in a pipeline fashion. Vertexattributes - color, surface normal, texture coordinate - are evaluated inthe vertex shader then passed on to the fragment shader where thosevalues are interpolated across the surface.
  84. 84. Elastic Image Software LLC84ShowST. A shader that visualizes the texturecoordinate parameterization of a surface.
  85. 85. Elastic Image Software LLC85https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;varying vec2 v_st;void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;}ShowST - vertex shader
  86. 86. Elastic Image Software LLC86attribute vec2 myVertexST - Vertex attributevarying vec2 v_st - Varies across the surfaceuniform sampler2D myTexture_0 - Invariant throughout rendering cycleShader variables come in different flavors
  87. 87. Elastic Image Software LLC87https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fshVertex attribute myVertexST is a texture coordinatepassed to the fragment shader via varying variable v_st.// ShowST.vshattribute highp vec4 myVertexXYZ;attribute highp vec2 myVertexST;// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;varying vec2 v_st;void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;}
  88. 88. Elastic Image Software LLC88https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0; }ShowST - fragment shader
  89. 89. Elastic Image Software LLC89http://github.com/turner/HelloiPadGLSL/blob/master/Shaders/ShowST.fsh// ShowST.fshvarying vec2 v_st;void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0; }The texture coordinate values v_st.s and v_st.tare for the red and green channels of the shader color.
  90. 90. Elastic Image Software LLC90TexturePairShader. A shader that mixes thecolors of a pair of textures using the tweakedalpha channel of one of the pair.Fail Whale over Mandril
  91. 91. Elastic Image Software LLC91TexturePairShader - vertex shaderhttps://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;varying mediump vec2 v_st;void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;}
  92. 92. Elastic Image Software LLC92TexturePairShader - vertex shader// TEITexturePairShader.vshattribute mediump vec2 myVertexST;attribute highp vec4 myVertexXYZ;// P * V * M - Projection spaceuniform mediump mat4 myProjectionViewModelMatrix;varying mediump vec2 v_st;void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST;}https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh
  93. 93. Elastic Image Software LLC93TexturePairShader - fragment shader// TEITexturePairShader.fshvarying mediump vec2 v_st;uniform sampler2D myTexture_0;uniform sampler2D myTexture_1;void main() { vec4 rgba_0 = texture2D(myTexture_0, v_st); vec4 rgba_1 = texture2D(myTexture_1, v_st); // Interpolate between the two textures using the alpha// channel of texture 0 as the interpolant. float interpolate = rgba_0.a; gl_FragColor.r = rgba_0.r + (1.0 - interpolate) * rgba_1.r; gl_FragColor.g = rgba_0.g + (1.0 - interpolate) * rgba_1.g; gl_FragColor.b = rgba_0.b + (1.0 - interpolate) * rgba_1.b; gl_FragColor.a = rgba_0.a + (1.0 - interpolate) * rgba_1.a; }https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh
  94. 94. Elastic Image Software LLC94attribute vec2 myVertexST - Vertex attributevarying vec2 v_st - Varies across the surfaceuniform sampler2D myTexture_0 - Invariant throughout rendering cycleShader variables come in different flavors
  95. 95. Elastic Image Software LLC95An application communicates with its shaders in the follow ways:• attributes - geometry, color, normal, texture coordinate• uniforms - texture sampler, matrix, eye vector, light vector• textures - texture channels - r, g, b, a - are not just for colorattribute vec2 myVertexST - Vertex attributevarying vec2 v_st - Varies across the surfaceuniform sampler2D myTexture_0 - Invariant throughout rendering cycle
  96. 96. Elastic Image Software LLC96Lets look at how we wire our iOS app togetherwith our shaders. We will use a texture shadingexample.https://github.com/turner/HelloShader/blob/master/HelloShader/Classes/Renderer/GLRenderer.m
  97. 97. Elastic Image Software LLC97First make a texture object ...TEITexture *t =[[ [TEITexture alloc] initWithImageFile:@"twitter_fail_whale_red_channnel_knockout"extension:@"png"mipmap:YES ] autorelease]; [glView.renderer.rendererHelper.renderables setObject:t forKey:@"texture_0"];
  98. 98. Elastic Image Software LLC98Load shaders, Bind shader attribute to application handle.// Vertex shaderNSString *vertShaderPathname =[[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"vsh"];[self compileShader:&vertShader type:GL_VERTEX_SHADER file:vertShaderPathname];!// Fragment shaderNSString *fragShaderPathname =[[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"fsh"];[self compileShader:&fragShader type:GL_FRAGMENT_SHADER file:fragShaderPathname];!// Bind attributeglBindAttribLocation(m_program, VertexSTAttributeHandle,! "myVertexST");
  99. 99. Elastic Image Software LLC99Associate shader uniform with application object instance ...TEITexture *t =(TEITexture *)[self.rendererHelper.renderables objectForKey:@"texture_0"];t.location = glGetUniformLocation(m_program, "myTexture_0");glActiveTexture( GL_TEXTURE0 );glBindTexture(GL_TEXTURE_2D, t.name);glUniform1i(t.location, 0);Activate a texture unit, bind it to a texture object, and assign a number to thecorresponding texture sampler used in the fragment shader
  100. 100. Elastic Image Software LLC100We now have a linkage between our application texture object and a texturesampler in our fragment shader. Rock.
  101. 101. Elastic Image Software LLC101
  102. 102. Elastic Image Software LLC102
  103. 103. Elastic Image Software LLC103Lets Talk Textures
  104. 104. Elastic Image Software LLC104Texture mapping is the fundamental tool for creating visual complexity
  105. 105. Elastic Image Software LLC105By establishing a mapping between a surface and a texture we can “attach”the texture to the surface.The interpretation of the a can go far beyond that of a decal to be applied toa surface. Bumps, opacity, displacement, and much more can be designedwith a texture.
  106. 106. Elastic Image Software LLC106
  107. 107. Elastic Image Software LLC107Shaders can have it both ways: Compute surfacecolor (or surface roughness, or opacity) algorithmicallyor look it up in a texture. Or a bit of both.
  108. 108. Elastic Image Software LLCOpenGL and iOSPlaying Nice Together108
  109. 109. Elastic Image Software LLC• OpenGL is mostly M• Keep V dumb• Lots of chatter between M and CMVC109
  110. 110. Elastic Image Software LLC110Lets take a look at how iOS and OpenGL interact. We will usHello iPad OpenGL as our guide.https://github.com/turner/HelloShader
  111. 111. Elastic Image Software LLC111-(id)initializeEAGL {!! CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer;!! eaglLayer.opaque = TRUE;! eaglLayer.drawableProperties =! [NSDictionary dictionaryWithObjectsAndKeys:! [NSNumber numberWithBool:FALSE], kEAGLDrawablePropertyRetainedBacking,! kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat,! nil];!! self.renderer = [[[ES2Renderer alloc] init] autorelease];!! m_animating = FALSE;! animationFrameInterval = 1;! m_displayLink = nil;!! return self;!}The view’s layer is used as the rendering surface
  112. 112. Elastic Image Software LLC112Rendering frame rate is sync-ed to the screen refresh rate.The actual rendering is handled via a selector.- (void)startAnimation { if (!self.isAnimating) { self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)]; [self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; self.animating = YES; } // if (!self.isAnimating) }
  113. 113. Elastic Image Software LLC113 // framebuffer glGenFramebuffers(1, &m_framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer); // rgb buffer glGenRenderbuffers(1, &m_colorbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, m_colorbuffer);[m_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:layer]; glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &m_backingWidth);glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &m_backingHeight); // z-buffer glGenRenderbuffers(1, &m_depthbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_depthbuffer); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, m_backingWidth, m_backingHeight); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, m_depthbuffer);A framebuffer object is created. A color and depth bufferare attached.
  114. 114. Elastic Image Software LLC114- (void) render {! ![EAGLContext setCurrentContext:m_context];glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer);// transform, light, shade, etc.glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer);[m_context presentRenderbuffer:GL_RENDERBUFFER];}The render loop. Draw to the colorbuffer then present tothe display. This is the classic “ping pong” between backbuffer and front buffer.
  115. 115. Elastic Image Software LLCThank YouDouglass TurnerElastic Image Softwareemail: douglass.turner@gmail.commobile: 781 775 3708115
  116. 116. Elastic Image Software LLC116Copyright © Douglass TurnerElastic Image Software LLC
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×