CS 354 Introduction


Published on

January 17, 2012 lecture


Published in: Technology, Art & Photos
1 Comment
  • Sir, Thanks alot for the slides.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CS 354 Introduction

  1. 1. CS 354 Introduction Mark Kilgard University of Texas January 17, 2012
  2. 2. CS 354—Computer Graphics <ul><li>Instructor: Mark Kilgard </li></ul><ul><ul><li>Principal Software Engineer, NVIDIA </li></ul></ul><ul><ul><li>Previously at Silicon Graphics </li></ul></ul><ul><li>Teaching Assistant: Randall Smith </li></ul><ul><ul><li>Ph.D. student with Dr. Fussell </li></ul></ul><ul><ul><li>TA’ed the course last semester </li></ul></ul><ul><li>Burdine Hall (BUR) 116 </li></ul><ul><ul><li>Tuesday/Thursday, 9:30-11:00 a.m. </li></ul></ul>
  3. 3. Expectations <ul><li>For us </li></ul><ul><ul><li>Teach practical graphics expertise </li></ul></ul><ul><ul><li>Well-prepared lectures </li></ul></ul><ul><ul><li>Grades fairly reflect your participation and performance </li></ul></ul><ul><ul><li>Available to you </li></ul></ul><ul><ul><ul><li>Use my office hours </li></ul></ul></ul><ul><ul><ul><li>Other times, use Randy </li></ul></ul></ul><ul><ul><ul><li>If not satisfied, contact me [email_address] </li></ul></ul></ul><ul><li>For you </li></ul><ul><ul><li>Attendance at every class </li></ul></ul><ul><ul><ul><li>Expect regular quizzes at the beginning of class </li></ul></ul></ul><ul><li>Programming assignments </li></ul><ul><ul><li>You need to know C/C++ </li></ul></ul><ul><ul><li>Use office hours if you need help </li></ul></ul><ul><ul><li>No cheating </li></ul></ul>
  4. 4. Grading <ul><li>Testing 60% </li></ul><ul><ul><li>Daily quizzes 10% </li></ul></ul><ul><ul><ul><li>2-3 questions, easy </li></ul></ul></ul><ul><ul><li>Homework 10% </li></ul></ul><ul><ul><ul><li>Occasional, mostly math questions </li></ul></ul></ul><ul><ul><li>Exams 40% </li></ul></ul><ul><ul><ul><li>Mid-term 15% </li></ul></ul></ul><ul><ul><ul><li>Final 25% </li></ul></ul></ul><ul><li>Software projects 40% </li></ul><ul><ul><li>Simple Rendering </li></ul></ul><ul><ul><li>Object loader </li></ul></ul><ul><ul><li>Interaction </li></ul></ul><ul><ul><li>Shading </li></ul></ul><ul><ul><li>Performance analysis </li></ul></ul>
  5. 5. Textbook <ul><li>Interactive Computer Graphics: A Top-Down Approach With Shader-Based OpenGL </li></ul><ul><ul><li>by Edward Angel and Dave Shreiner </li></ul></ul><ul><ul><li>Addison-Wesley, 6th edition </li></ul></ul><ul><li>Regular readings assigned </li></ul><ul><ul><li>Match up with lecture topics </li></ul></ul>
  6. 6. Helpful Resources <ul><li>Learning OpenGL </li></ul><ul><ul><li>OpenGL-oriented books </li></ul></ul><ul><li>Supplemental books </li></ul>OpenGL Programming Guide a.k.a. “the red book” OpenGL SuperBible OpenGL A Primer Eric Lengyel’s Mathematics for 3D Game Programming and Computer Graphics Real-Time Rendering by Eric Haines, Tomas Akenine-Moller, Eric Haines, Naty Hoffman
  7. 7. Computer Graphics <ul><li>Nexus of several disciplines </li></ul>Human Perception Artistic Expression Physics of Light Geometry and Mathematics of Surfaces Computer Science VLSI Hardware Design Display & Input Technology Animation & Simulation
  8. 8. Roles for Computer Graphics [Pixar 2010] Story telling Product design [CATIA] Roles for Computer Graphics
  9. 9. Roles for Computer Graphics Training [Commercial simulators] Gaming [Skyrim]
  10. 10. Roles for Computer Graphics User interfaces [Android 4.0] Navigation [Audi]
  11. 11. Roles for Computer Graphics Printing Digital imaging & video [HP Deskjet] [Canon]
  12. 12. What does computer graphics study? <ul><li>Computer-based simulation of </li></ul><ul><ul><li>Shape </li></ul></ul><ul><ul><li>Appearance </li></ul></ul><ul><ul><li>Motion </li></ul></ul>[Litke et.al. 2001] [Chai & Hodgins, 2005] [Sloan et.al. 2005]
  13. 13. Not covered in the class <ul><li>Digital content creation </li></ul><ul><ul><li>No Photoshop, no Maya or 3D Studio Max </li></ul></ul><ul><ul><li>Computer Science class, not an art class </li></ul></ul><ul><li>C/C++ programming </li></ul><ul><ul><li>I expect you know C or C++ under Linux </li></ul></ul><ul><ul><ul><li>Not just the language </li></ul></ul></ul><ul><ul><ul><li>Need to know debugging and software practices </li></ul></ul></ul><ul><ul><li>Since course’s programming projects assume Linux software development </li></ul></ul><ul><ul><ul><li>Use the Computer Science labs (ESN or Painter) </li></ul></ul></ul>
  14. 14. Visual Thinking <ul><li>Human visual system = highest bandwidth input to your brain </li></ul><ul><ul><li>Very natural to want computers “feeding” this input to your brain </li></ul></ul><ul><ul><li>Because people think visually </li></ul></ul><ul><ul><ul><li>Innate intuition for 3D imagery in particular </li></ul></ul></ul><ul><li>Computer graphics </li></ul><ul><ul><li>Takes an abstract representation of a “scene” within a computer’s memory and converts it to a concrete signal (an image, or animating images) representing a view of that scene </li></ul></ul><ul><ul><li>Computer graphics practitioners have a good practical and theoretical understanding of how to do this </li></ul></ul><ul><ul><ul><li>Amazing progress in the last 40 years </li></ul></ul></ul><ul><li>Your brain </li></ul><ul><ul><li>Takes concrete signals (images, the visual world) and converts those signals (back!) into an abstract representation of a scene </li></ul></ul><ul><ul><li>We have only the most rudimentary notions of how this process works </li></ul></ul><ul><ul><ul><li>Teaching a computer to do this is the field of “computer vision” </li></ul></ul></ul>
  15. 15. Reductionist Approach <ul><li>Can’t simply “take a picture” like a camera </li></ul><ul><ul><li>Instead images are “synthesized” from an abstract model of a scene and its view </li></ul></ul><ul><ul><li>Must build graphics out of little bits of work and data </li></ul></ul><ul><li>Philosophers and scientists break down phenomenon to smallest observable units </li></ul><ul><ul><li>Examples </li></ul></ul><ul><ul><ul><li>Philosophical atoms (Leucippus, ancient) </li></ul></ul></ul><ul><ul><ul><li>Triangles (Plato, ancient) </li></ul></ul></ul><ul><ul><ul><li>Chemical atoms (Dalton, pre-modern) </li></ul></ul></ul><ul><ul><ul><li>Physical atoms (Bohr, modern) </li></ul></ul></ul><ul><ul><ul><li>Sub-atomic quantum particles, waves, strings </li></ul></ul></ul><ul><li>Reductionism in graphics </li></ul><ul><ul><li>Pixels, vertexes, and triangles (Plato redux?) </li></ul></ul><ul><ul><li>Discretization pervades, ideal for digital computers </li></ul></ul>
  16. 16. Your Modern World View: Discrete Information, Particularly Images <ul><li>You believe in digital information </li></ul><ul><ul><li>Impossible to avoid this belief in a modern, computerized world </li></ul></ul><ul><ul><li>Text is digital, video is digital, music is digital, communication is digital, even identity is becoming digital </li></ul></ul><ul><ul><li>You believe exact copies are possible, even common </li></ul></ul><ul><ul><li>Pervasive new belief that’s incognizant to world just 50 years ago </li></ul></ul><ul><li>Discrete images implies pixels </li></ul><ul><ul><li>Descartes’s Cartesian plane computerized </li></ul></ul><ul><ul><li>Image ≈ grid of discrete color </li></ul></ul><ul><ul><li>So images are simply numbers </li></ul></ul><ul><ul><ul><li>So can be processed as numbers! </li></ul></ul></ul>
  17. 17. Shape: Objects to Triangles <ul><li>Digital artists approximate solid object shape with meshes of triangles </li></ul><ul><ul><li>Was Plato right? </li></ul></ul>More triangles makes the mesh an increasingly accurate approximation of the bunny shape
  18. 18. What will you learn <ul><li>Fundamentals of computer graphics </li></ul><ul><ul><li>Transformations and viewing </li></ul></ul><ul><ul><li>Rasterization and ray tracing </li></ul></ul><ul><ul><li>Lighting and shading </li></ul></ul><ul><ul><li>Graphics hardware technology </li></ul></ul><ul><ul><li>Mathematics for computer graphics </li></ul></ul><ul><li>Practical graphics programming </li></ul><ul><ul><li>OpenGL programming </li></ul></ul><ul><ul><li>Shader programming </li></ul></ul><ul><ul><li>Performance analysis </li></ul></ul>
  19. 19. <ul><li>Expect programming projects using OpenGL </li></ul><ul><ul><li>Explicitly not just an “OpenGL” class </li></ul></ul><ul><ul><ul><li>But you will learn and use OpenGL </li></ul></ul></ul><ul><ul><ul><li>When relevant Direct3D is discussed too </li></ul></ul></ul><ul><ul><li>You’ll use GLUT so programs can work on Windows, Mac, and Linux </li></ul></ul><ul><li>Plenty of resources to learn OpenGL </li></ul><ul><ul><li>www.opengl.org </li></ul></ul>
  20. 20. What is OpenGL? <ul><li>Its specification is titled “The OpenGL Graphics System ” </li></ul><ul><ul><li>Not just for 3D graphics; imaging too </li></ul></ul><ul><ul><li>“ GL” standard for “Graphics Library” </li></ul></ul><ul><ul><li>“ Open” means industry standard meant for broad adoption with liberal licensing </li></ul></ul><ul><li>Standardized in 1992 </li></ul><ul><ul><li>By Silicon Graphics </li></ul></ul><ul><ul><li>And others: Compaq, DEC, Intel, IBM, Microsoft </li></ul></ul><ul><ul><li>Originally meant for Unix and Windows workstations </li></ul></ul><ul><li>Now de facto graphics acceleration standard </li></ul><ul><ul><li>Now managed by the Khronos industry consortium </li></ul></ul><ul><ul><li>Available everywhere, from supercomputers to cell phones </li></ul></ul><ul><ul><li>Alternative: Direct3D provides similar functionality with a very different API for Microsoft Windows platforms </li></ul></ul>[Marathon Oil visionarium] [ES2 tablet] [MacBook]
  21. 21. OpenGL as an Evolving Standard EXT SGI SGIS SGIX ARB NV Others Others <ul><li>44% of extensions are “core” or multi-vendor </li></ul><ul><li>Lots of vendors have initiated extensions </li></ul><ul><li>Extending OpenGL is industry-wide collaboration </li></ul>ATI APPLE MESA Source: http://www.opengl.org/registry (Dec 2008)
  22. 22. Forces Driving Improvements in Computer Graphics Human desire for Visual Intuition and Entertainment Embarrassing Parallelism of Graphics Increasing Semiconductor Density Particularly the hardware-amenable, latency tolerant nature of rasterization Particularly interactive video games Computer Graphics Moore’s Law
  23. 23. Key Trend in OpenGL Evolution Fixed-function Programmable Simple Configurability Complex Configurability Shaders! High-level languages
  24. 24. Many Perspectives on OpenGL <ul><li>Programmer’s view: </li></ul><ul><ul><li>Application Programming Interface (API) </li></ul></ul><ul><ul><li>Accelerated access to 3D graphics hardware </li></ul></ul><ul><ul><ul><li>Performance and functionality improves with time </li></ul></ul></ul><ul><li>Graphics Architect’s view: </li></ul><ul><ul><li>Detailed functional graphics pipeline </li></ul></ul><ul><ul><ul><li>Well suited to VLSI hardware design </li></ul></ul></ul><ul><li>Software or Hardware System Designer’s view: </li></ul><ul><ul><li>Stable standard component for building larger system </li></ul></ul><ul><ul><ul><li>Platform neutral so fits into other systems well </li></ul></ul></ul><ul><ul><li>Examples: WebGL, Apple’s compositing desktop </li></ul></ul><ul><li>Student’s view: </li></ul><ul><ul><li>Real-world implementation of graphics concepts </li></ul></ul><ul><ul><ul><li>Good pedantic structure for learning graphics </li></ul></ul></ul><ul><ul><li>Practical skill you can really use </li></ul></ul>
  25. 25. Programmer’s View: OpenGL API Example <ul><li>Let’s draw a triangle </li></ul>glShadeModel ( GL_SMOOTH ); // smooth color interpolation glEnable ( GL_DEPTH_TEST ); // enable hidden surface removal glClear (GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT); glBegin (GL_TRIANGLES); { // every 3 vertexes makes a triangle glColor4ub (255, 0, 0, 255); // RGBA=(1,0,0,100%) glVertex3f (-0.8, 0.8, 0.3); // XYZ=(-8/10,8/10,3/10) glColor4ub (0, 255, 0, 255); // RGBA=(0,1,0,100%) glVertex3f ( 0.8, 0.8, -0.2); // XYZ=(8/10,8/10,-2/10) glColor4ub (0, 0, 255, 255); // RGBA=(0,0,1,100%) glVertex3f ( 0.0, -0.8, -0.2); // XYZ=(0,-8/10,-2/10) } glEnd (); Pro Tip: use curly braces to “bracket” nested OpenGL usage; no semantic meaning, just highlights grouping
  26. 26. Initial Logical Coordinate System <ul><li>Think of drawing into a [-1,+1] 3 cube </li></ul>(-0.8, 0.8) (-0.8, 0.8) (0, -0.8) origin at (0,0)
  27. 27. Visualizing Normalized Device Coordinates <ul><li>What does this simple triangle look like with the [-1,+1] 3 cube’s coordinate system? </li></ul><ul><ul><li>We call this coordinate system “Normalize Device Coordinate” or NDC space </li></ul></ul>Wire frame cube shows boundaries of NDC space From NDC views, you can see triangle isn’t “ flat” in the Z direction Two vertices have Z of -0.2—third has Z of 0.3
  28. 28. Programmer’s View: GLUT API Example <ul><li>Windowing code </li></ul>#include <GL/glut.h> // includes necessary OpenGL headers void display() { // << insert code on prior slide here >> glutSwapBuffers (); } void main(int argc, char **argv) { // request double-buffered color window with depth buffer glutInitDisplayMode ( GLUT_RGBA | GLUT_DOUBLE | GLUT_DEPTH ); glutInit (&argc, argv); glutCreateWindow (“simple triangle”); glutDisplayFunc (display); // function to render window glutMainLoop (); } FYI: GLUT = OpenGL Utility Toolkit
  29. 29. Reductionism for Rendering Objects and Scenes <ul><li>Programmers want to render “objects” </li></ul><ul><ul><li>Say a fire truck or molecule </li></ul></ul><ul><ul><li>Arranged relative to other objects (a scene) & then viewed </li></ul></ul><ul><li>Graphics pipeline approach—used by OpenGL and GPUs </li></ul><ul><ul><li>Break objects into geometry batches </li></ul></ul><ul><ul><ul><li>Batches may be meshes or “patches” </li></ul></ul></ul><ul><ul><li>Batches reduce to polygonal primitives </li></ul></ul><ul><ul><ul><li>Typically triangles </li></ul></ul></ul><ul><ul><ul><li>But also lines, points, bitmaps, or images </li></ul></ul></ul><ul><ul><li>Geometric primitives are specified by vertices </li></ul></ul><ul><ul><ul><li>So vertices are assembled into primitives </li></ul></ul></ul><ul><ul><li>Primitives are rasterized into fragments </li></ul></ul><ul><ul><li>Fragments are shaded </li></ul></ul><ul><ul><li>Raster operations take shaded fragments and update the framebuffer </li></ul></ul>
  30. 30. Programmer’s View: Also Programming Shaders inside GPU <ul><li>Multiple programmable domains within the GPU </li></ul><ul><li>Can be programmed in high-level language </li></ul><ul><ul><li>OpenGL Shading Language (GLSL) </li></ul></ul>Geometry Program 3D Application or Game OpenGL API GPU Front End Vertex Assembly Vertex Shader Clipping, Setup, and Rasterization Fragment Shader Texture Fetch Raster Operations Framebuffer Access Memory Interface CPU – GPU Boundary OpenGL 3.3 Attribute Fetch Primitive Assembly Parameter Buffer Read programmable fixed-function Legend
  31. 31. Example Simple GLSL Shaders <ul><li>Vertex Shader </li></ul><ul><ul><li>Operates on each vertex of geometric primitives </li></ul></ul><ul><ul><li>Passes through per-vertex color </li></ul></ul><ul><ul><li>Transforms the vertex to match fixed-function processing </li></ul></ul><ul><li>Fragment Shader </li></ul><ul><ul><li>Operates on each fragment (think pixel) </li></ul></ul><ul><ul><li>Outputs the fragment’s interpolated color to the framebuffer </li></ul></ul>void main(void) { gl_FrontColor = gl_Color ; gl_Position = ftransform (); } void main(void) { gl_FragColor = gl_Color ; } Shaders are way more interesting than these minimal examples
  32. 32. Examples of Complex Shaders
  33. 33. Building Up Shaders   ) + ( ( ) = Diffuse Gloss Specular Decal Result
  34. 34. OpenGL’s Design Philosophy <ul><li>High-performance </li></ul><ul><ul><li>Assumes hardware acceleration </li></ul></ul><ul><li>Defined by a specification </li></ul><ul><ul><li>Rather than a de-facto implementation </li></ul></ul><ul><li>Rendering state machine </li></ul><ul><ul><li>Procedural </li></ul></ul><ul><ul><li>Not a window system, not a scene graph </li></ul></ul><ul><li>No initial sub-setting </li></ul><ul><li>Extensible </li></ul><ul><li>Data type rich </li></ul><ul><li>Cross-platform </li></ul><ul><ul><li>Window system-independent core </li></ul></ul><ul><ul><li>X Window System, Microsoft Windows, OS/2, OS X, etc. </li></ul></ul><ul><li>Multi-language bindings </li></ul><ul><ul><li>C, FORTRAN, etc. </li></ul></ul><ul><ul><li>Not merely an API, rather a system </li></ul></ul>Later had OpenGL ES subset for embedded devices
  35. 35. OpenGL state machine Complicated from inception
  36. 36. Higher-level View of OpenGL <ul><li>From OpenGL 3.0 specification, unchanged since 1.0 </li></ul>
  37. 37. Evolved OpenGL Data Flow vertex shading rasterization & fragment shading texture raster operations framebuffer pixel unpack pixel pack vertex puller client memory pixel transfer glReadPixels / glCopyPixels / glCopyTex{Sub}Image glDrawPixels glBitmap glCopyPixels glTex{Sub}Image glCopyTex{Sub}Image glDrawElements glDrawArrays selection / feedback / transform feedback glVertex* glColor* glTexCoord* etc. blending depth testing stencil testing accumulation storage operations
  38. 38. Buffer-centric Evolution <ul><li>Data moves onto GPU, away from CPU </li></ul><ul><ul><li>Apps on CPUs just too slow at moving data otherwise </li></ul></ul>Vertex Array Buffer Object (VaBO) Transform Feedback Buffer (XBO) Parameter Buffer Object (PaBO) Pixel Unpack Buffer (PuBO) Pixel Pack Buffer (PpBO) Uniform Buffer Object (UBO) Texture Buffer Object (TexBO) Vertex Puller Vertex Shading Geometry Shading Fragment Shading Texturing Array Element Buffer Object (VeBO) Pixel Pipeline vertex data texel data pixel data parameter data glBegin, glDrawElements, etc. glDrawPixels, glTexImage2D, etc. glReadPixels, etc. Framebuffer
  39. 39. Extensibility <ul><li>OpenGL is evolved via extensions </li></ul><ul><ul><li>Legalistic—expressed as “amending” OpenGL’s detailed specification </li></ul></ul><ul><ul><li>Coordinated through Khronos </li></ul></ul><ul><ul><ul><li>But vendors don’t need any blessing or permission to make extensions </li></ul></ul></ul><ul><li>Allows priorities to be shaped quickly </li></ul><ul><ul><li>Example : Direct3Dism extensions have greatly reduced functional incompatibilities keeping Direct3D apps migrate to OpenGL </li></ul></ul><ul><ul><li>All the latest GPU features available in OpenGL </li></ul></ul><ul><ul><ul><li>Example: Tessellation shaders </li></ul></ul></ul>
  40. 40. Software and Hardware Architectures Incorporating 3D Pipelines <ul><li>X Window System </li></ul><ul><li>GeForce 6 Architecture </li></ul>framebuffer updates vertex processing rasterization shading X server Graphics Kernel driver GPU Hardware OpenGL driver Application OpenGL library GLUT X client libraries Operating system kernel (Linux)
  41. 41. Hardware Platforms depending on the OpenGL Architecture <ul><li>Workstation PCs </li></ul><ul><li>Consumer PCs </li></ul><ul><li>High-end Visual Computing Solution (VCS) </li></ul><ul><li>Embedded Applications </li></ul><ul><li>Handheld Devices </li></ul><ul><li>Game Consoles </li></ul>Conventional PC OpenGL Products Unconventional non-PC OpenGL platforms
  42. 42. Student’s View of OpenGL <ul><li>You can learn OpenGL gradually </li></ul><ul><ul><li>Lots of its can be ignored for now </li></ul></ul><ul><ul><li>The “classic” API is particularly nice </li></ul></ul><ul><ul><ul><li>“Deprecation” has ruined the pedagogical niceness of OpenGL; ignore deprecation </li></ul></ul></ul><ul><li>Plenty of documentation and sample code </li></ul><ul><li>Makes concrete the abstract graphics pipeline for rasterization </li></ul>
  43. 43. Next Lecture <ul><li>3D Viewing </li></ul><ul><ul><li>How are 3D scenes processed so they can be turned into an image on the screen? </li></ul></ul><ul><ul><li>Expect a short quiz on today’s lecture </li></ul></ul><ul><ul><ul><li>Easy questions </li></ul></ul></ul><ul><li>Assignments </li></ul><ul><ul><li>Reading </li></ul></ul><ul><ul><ul><li>Chapter 1, pages 1-38 </li></ul></ul></ul><ul><ul><li>Makes sure your CS Unix account is active </li></ul></ul><ul><ul><ul><li>Homework next time will be compiling a simple OpenGL example </li></ul></ul></ul>