By VISY - ""In the country of the blind, the one-eyed man is king""


Published on

"In the country of the blind, the one-eyed man is king" by Visa-Valtteri "visy" Pimia

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

By VISY - ""In the country of the blind, the one-eyed man is king""

  1. 1. "In the country of the blind, the one-eyed man is king"<br />or how fumbling my way through pixel-based effects <br />made learning WebGL a lot easier<br />Visa-Valtteri “visy” Pimiä<br />
  2. 2. My history with pixel-based visuals<br />Started doing simple effects with QBASIC<br />Worked my way to Turbo Pascal and VGA programming<br />Got sidetracked to NES and Atari 2600 development<br />Learned OpenGL mostly professionally, then started using GPU-accelerated graphics<br />
  3. 3. But why talk about something whenI can just speedcode here :-)<br />Processing makes it possible<br />
  4. 4. Coding pixel-based effects<br />Access the screen through a framebuffer, an array of pixel color values, palette index values or bitplane toggles<br />Set pixel color values according to position, time, function value etc.<br />
  5. 5. Proprietary “engines”<br />I started making small games and demo-style little programs<br />Everything was always coded for one single production, with no mind put into reusability of the tools<br />This got old pretty quickly and I found myself toying with the idea of a visual editor for demos<br />
  6. 6. First: S.P.L.I.F<br />TrilobitScriPtableLua Interactive MultiplatForm<br />Tackles a multitude of problems I had with development:<br />Inherently multiplatform (OSX, Win32, Linux) because all the GL calls are coded with Lua.<br />All visual code is reloadable without relaunching or recompiling the program<br />The demo player is seekable and supports setting sync points <br />“The demo is the editor”<br />Always open source<br />
  7. 7. Syncing<br />Timeline file millisecond syncing of demo parts<br />FFT beat sync for effects<br />Seeking for testing<br />
  8. 8. S.P.L.I.F has flaws<br />Fixed OpenGL version, upgrading would require recompiling the Lua libraries for all the three platforms<br />No shader support<br />Terrible hacks for texture handling<br />The OpenGL / Lua bound version I used was found to be rather incomplete<br />
  9. 9. But it worked well enough<br />Counter Intelligence Program by Bilotrip<br />
  10. 10. Need for an upgrade<br />After I heard about WebGL I got the idea of doing something like S.P.L.I.F in the browser with easily reloadable and tweakable visuals<br />WebGL has shaders, so the natural choice was to enable the editing of shader parameters (in addition for JavaScript variables) realtime<br />
  11. 11. Technology behind DarumaGL<br />WebGL canvas for the output<br />Pure JavaScript + HTML5 + CSS3 for the UI<br />Textures loaded from PNG files<br />HTML5 audio from OGG file<br />Timeline.js used for the keyframe editor<br />
  12. 12. Demonstration of the DarumaGL editor<br /><br />
  13. 13. …and it has flaws as well<br />No real timeline dumping (cut & paste from export windows)<br />Shader code still requires reloading of the page<br />HTML5 audio support is still rather flaky cross browsers, and require some time to mature<br />No cross-browser FFT support for sound analysis<br />WebGL is still not available for mobile browsers or Internet Explorer <br />
  14. 14. But…<br />WebGL _is_ a W3 standard, and pretty much all major browser except Internet Explorer have support for it in development roadmaps<br />No plugins required anymore for GPU accelerated content and complex video/audio data processing and playback<br />No third party components required<br />
  15. 15. Lessons learned from pixel effects<br />Pixel shaders are pretty similar to framebuffers in many ways<br />Can set single resolution independent fragments instead of toggling actual pixels on the screen<br />Many fragments are calculated concurrently on the GPU, so the shader program is run in parallel.<br />
  16. 16. Some things of note<br />I tend to avoid calculating complex geometries and like to play around with fragments on a single quad<br />Complex blending and processing passes possible with the GPU -> enables effects that are just simply impossible to do on a framebuffer with a fast FPS<br />
  17. 17. Demonstration<br />MKULTRA by Bilotrip<br /><br />
  18. 18. Thank you for listening!<br />