Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
GRAPHICS 
MATT & HEIKO – DEVELOPER EXPERIENCE ENGINEERS 
ON TWITTER AS @MTHUNGERFORD & @HBEHRENS
THE DISPLAY 
144x168px @ 175 DPI 
black & white only 
25+ FPS
CONTENT 
A Pixel’s Journey 
Dithering 
Accessing the Frame Buffer
A PIXEL’S 
JOURNEY
LAYER STACK 
Layers are added to the 
layer_stack from back to front 
starting from the window layer. 
! 
Layers are then ...
THE COMPOSITOR 
Example of a non-fullscreen application (144x152): 
Application 
frame buffer 
Compositor 
frame buffer 
F...
GRAPHICS APIS 
static void custom_update_proc(Layer *layer, GContext *ctx); 
Layer-equivalents working directly on GContex...
Q&A 
source: https://www.youtube.com/watch?v=Ej2ELEK-C3Q Where To?
GRAPHICS APIS 
static void custom_update_proc(Layer *layer, GContext *ctx); 
Layer-equivalents working directly on GContex...
Drawings advanced raster graphics 
graphics_capture_frame_buffer, graphics_release_frame_buffer 
Doing dithering at runtim...
DITHERING
Dither is an intentionally applied form of noise used to 
randomize quantization error, preventing large-scale 
patterns s...
256 shades of gray 
16 SHADES OF GRAY 
16 shades of gray + nearest color 
16 shades of gray + random noise 
16 shades of g...
original ordered dithering Floyd-Steinberg 
dithering 
nearest color
FLOYD-STEINBERG DITHERING 
for each y from top to bottom 
for each x from left to right 
oldpixel := pixel[x][y] 
newpixel...
source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ original from PSX game “Chrono Cross...
source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ nearest neighbor to 12 colors
source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ Floyd-Steinberg to 12 colors
OFFLINE 
DITHERING
BETTER BITMAPS (GIMP/PHOTOSHOP) 
Resize to desired size (<144x168) using sinc filter 
Convert image to grayscale 
Modify b...
Better Bitmaps (GIMP/Photoshop) 
Open Original Image in the GIMP/Photoshop
Better Bitmaps (GIMP/Photoshop) 
Resize: Image -> Scale Image
Better Bitmaps (GIMP/Photoshop) 
Resize to desired size (<144x168) using sinc filter
Better Bitmaps (GIMP/Photoshop) 
Grayscale: Image -> Mode -> Grayscale
Better Bitmaps (GIMP/Photoshop) 
Brightness-Contrast: Colors -> Brightness-Contrast
Better Bitmaps (GIMP/Photoshop) 
Slide contrast to right to remove colors and balance with brightness
Better Bitmaps (GIMP/Photoshop) 
Posterize: Colors -> Posterize
Better Bitmaps (GIMP/Photoshop) 
Posterize: Select 3 colors (2 for black & white only). 3rd color is Gray.
Better Bitmaps (GIMP/Photoshop) 
Convert to 1-bit (b&w): Image -> Mode -> Indexed
Better Bitmaps (GIMP/Photoshop) 
Select b&w, choose dithering Positioned (Ordered) for dithered gray
Better Bitmaps (GIMP/Photoshop) 
Final Product: Robot with dithered gray_50
BETTER BITMAPS (IMAGEMAGICK) 
convert orig.png 
-adaptive-resize '144x168>' 
-fill '#FFFFFF00' -opaque none 
-type Graysca...
DITHERED ANIMATIONS (APNG) 
ImageMagick scripting 
! 
Ordered dithering (gray_50) 
! 
(a)PNG compression for 200 frames 
!...
RUNTIME 
DITHERING
50% GRAY
PRE-DITHERED 
FLOYD 
STEINBERG 
ORDERED 
DITHERING
PRE-DITHERED 
AT RUNTIME
PRE-DITHERED 
AT RUNTIME
PRE-DITHERED 
AT RUNTIME
PRE-DITHERED 
AT RUNTIME
PRE-DITHERED 
AT RUNTIME
50% GRAY
PRE-DITHERED 
FLOYD 
STEINBERG 
ORDERED 
DITHERING
source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ reduction to 18 colors
IMPLEMENT 
DITHERING 
AT RUNTIME
ORDERED DITHERING (BAYER MATRIX) 
threshold_map_4x4 = 
foreach y 
foreach x 
oldpixel := pixel[x][y] + threshold_map_4x4[x...
ORDERED DITHERING BLACK & WHITE 
foreach y 
foreach x 
newpixel := pixel[x][y] > threshold_map_8x8[x mod 8][y mod 8] ? whi...
DOING LIVE DITHERING 
gray_top 
gray_bottom 
static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); 
! 
static ...
AVOIDING FLOATS frac = 0.0 
gray_top * (1 - 0.0) + 0.0 * gray_bottom 
frac = 0.25 
gray_top * (1 - 0.25) + 0.25 * gray_bot...
AVOIDING FLOATS 
float frac = (float)y / h; // 0.0 … 1.0 
row_gray = gray_top * (1 - frac) + frac * gray_bottom 
// 1 * h ...
DOING LIVE DITHERING 
gray_top 
gray_bottom 
static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); 
! 
static ...
DOING LIVE DITHERING 
gray_top 
gray_bottom 
static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); 
! 
static ...
source: https://www.youtube.com/watch?v=V5O8gj0SQZ0
DOING FAST(!) LIVE DITHERING 
GBitmap 
* graphics_capture_frame_buffer(GContext* ctx) 
Gives you a GBitmap to represent th...
GBITMAP RECAP 
0 
4 
8 
12 
16 
29px 
4 bytes per row 
5px 
unused 
unused 
unused 
unused 
unused 
Fragment of 8 pixels i...
DOING FAST(!) LIVE DITHERING 
static void update_proc_frame_buffer(Layer *layer, GContext *ctx) { 
GBitmap *fb = graphics_...
RENDER 8 PIXELS AT ONCE 
static uint8_t (int16_t y, uint8_t gray) { 
return color_for_gray(0, y, gray) << 7 | 
color_for_g...
DOING FAST(!) LIVE DITHERING 
static void update_proc_frame_buffer(Layer *layer, GContext *ctx) { 
GBitmap *fb = graphics_...
source: https://www.youtube.com/watch?v=V5O8gj0SQZ0
BOING 
BALL
Q&A 
source: https://www.youtube.com/watch?v=Z2QMG9UwXI0
Advanced Techniques: Graphics | Pebble Developer Retreat 2014
Advanced Techniques: Graphics | Pebble Developer Retreat 2014
Advanced Techniques: Graphics | Pebble Developer Retreat 2014
Upcoming SlideShare
Loading in …5
×

Advanced Techniques: Graphics | Pebble Developer Retreat 2014

2,218 views

Published on

You can find the video recording here: https://www.youtube.com/watch?v=lYoHh19RNy4

Heiko Behrens and Matthew Hungerford present advanced programming techniques for Pebble. This presentation focused on graphics techniques including run-time dithering, offline dithering, pixel manipulations, and frame-buffer drawing.

This talk featured the amiga boing ball dithering demo.

Day 1 - Video 3B

Published in: Technology
  • Be the first to comment

Advanced Techniques: Graphics | Pebble Developer Retreat 2014

  1. 1. GRAPHICS MATT & HEIKO – DEVELOPER EXPERIENCE ENGINEERS ON TWITTER AS @MTHUNGERFORD & @HBEHRENS
  2. 2. THE DISPLAY 144x168px @ 175 DPI black & white only 25+ FPS
  3. 3. CONTENT A Pixel’s Journey Dithering Accessing the Frame Buffer
  4. 4. A PIXEL’S JOURNEY
  5. 5. LAYER STACK Layers are added to the layer_stack from back to front starting from the window layer. ! Layers are then rendered from back to front into the application frame buffer. Window
  6. 6. THE COMPOSITOR Example of a non-fullscreen application (144x152): Application frame buffer Compositor frame buffer Final display
  7. 7. GRAPHICS APIS static void custom_update_proc(Layer *layer, GContext *ctx); Layer-equivalents working directly on GContext graphics_draw_text, graphics_draw_bitmap_in_rect Drawings vector graphics gpath_draw_filled, gpath_draw_outline Drawings raster graphics graphics_draw_pixel, graphics_draw_line graphics_draw_circle, graphics_fill_circle, graphics_draw_rect, graphics_draw_round_rect, graphics_fill_rect Animate graphics app_timer_register, app_timer_reschedule, app_timer_cancel Animation, PropertyAnimation
  8. 8. Q&A source: https://www.youtube.com/watch?v=Ej2ELEK-C3Q Where To?
  9. 9. GRAPHICS APIS static void custom_update_proc(Layer *layer, GContext *ctx); Layer-equivalents working directly on GContext graphics_draw_text, graphics_draw_bitmap_in_rect Drawings vector graphics gpath_draw_filled, gpath_draw_outline Drawings raster graphics graphics_draw_pixel, graphics_draw_line graphics_draw_circle, graphics_fill_circle, graphics_draw_rect, graphics_draw_round_rect, graphics_fill_rect Animate graphics app_timer_register, app_timer_reschedule, app_timer_cancel Animation, PropertyAnimation
  10. 10. Drawings advanced raster graphics graphics_capture_frame_buffer, graphics_release_frame_buffer Doing dithering at runtime
  11. 11. DITHERING
  12. 12. Dither is an intentionally applied form of noise used to randomize quantization error, preventing large-scale patterns such as color banding in images. http://en.wikipedia.org/wiki/Dither 256 shades of gray 16 shades of gray + nearest color 16 shades of gray + random noise 16 shades of gray + ordered dithering
  13. 13. 256 shades of gray 16 SHADES OF GRAY 16 shades of gray + nearest color 16 shades of gray + random noise 16 shades of gray + ordered dithering BLACK AND WHITE black & white + nearest color black & white + noise black & white + ordered dithering
  14. 14. original ordered dithering Floyd-Steinberg dithering nearest color
  15. 15. FLOYD-STEINBERG DITHERING for each y from top to bottom for each x from left to right oldpixel := pixel[x][y] newpixel := find_closest_palette_color(oldpixel) pixel[x][y] := newpixel quant_error := oldpixel - newpixel pixel[x+1][y ] := pixel[x+1][y ] + quant_error * 7/16 pixel[x-1][y+1] := pixel[x-1][y+1] + quant_error * 3/16 pixel[x ][y+1] := pixel[x ][y+1] + quant_error * 5/16 pixel[x+1][y+1] := pixel[x+1][y+1] + quant_error * 1/16 ORDERED DITHERING (BAYER MATRIX) threshold_map_4x4 = foreach y foreach x oldpixel := pixel[x][y] + threshold_map_4x4[x mod 4][y mod 4] newpixel := find_closest_palette_color(oldpixel) pixel[x][y] := newpixel
  16. 16. source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ original from PSX game “Chrono Cross”
  17. 17. source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ nearest neighbor to 12 colors
  18. 18. source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ Floyd-Steinberg to 12 colors
  19. 19. OFFLINE DITHERING
  20. 20. BETTER BITMAPS (GIMP/PHOTOSHOP) Resize to desired size (<144x168) using sinc filter Convert image to grayscale Modify brightness/contrast to sharpen image Posterize to 3 colors (if dithering to gray_50) or 2 colors for black and white. Undo-Redo brightness/contrast & posterize till desired effect Convert image to black and white with ordered (positional) dithering
  21. 21. Better Bitmaps (GIMP/Photoshop) Open Original Image in the GIMP/Photoshop
  22. 22. Better Bitmaps (GIMP/Photoshop) Resize: Image -> Scale Image
  23. 23. Better Bitmaps (GIMP/Photoshop) Resize to desired size (<144x168) using sinc filter
  24. 24. Better Bitmaps (GIMP/Photoshop) Grayscale: Image -> Mode -> Grayscale
  25. 25. Better Bitmaps (GIMP/Photoshop) Brightness-Contrast: Colors -> Brightness-Contrast
  26. 26. Better Bitmaps (GIMP/Photoshop) Slide contrast to right to remove colors and balance with brightness
  27. 27. Better Bitmaps (GIMP/Photoshop) Posterize: Colors -> Posterize
  28. 28. Better Bitmaps (GIMP/Photoshop) Posterize: Select 3 colors (2 for black & white only). 3rd color is Gray.
  29. 29. Better Bitmaps (GIMP/Photoshop) Convert to 1-bit (b&w): Image -> Mode -> Indexed
  30. 30. Better Bitmaps (GIMP/Photoshop) Select b&w, choose dithering Positioned (Ordered) for dithered gray
  31. 31. Better Bitmaps (GIMP/Photoshop) Final Product: Robot with dithered gray_50
  32. 32. BETTER BITMAPS (IMAGEMAGICK) convert orig.png -adaptive-resize '144x168>' -fill '#FFFFFF00' -opaque none -type Grayscale -colorspace Gray -black-threshold 30% -white-threshold 70% -ordered-dither 2x1 -colors 2 -depth 1 -define png:compression-level=9 -define png:compression-strategy=0 -define png:exclude-chunk=all new.png
  33. 33. DITHERED ANIMATIONS (APNG) ImageMagick scripting ! Ordered dithering (gray_50) ! (a)PNG compression for 200 frames ! Resource loading and drawing at high FPS (10 FPS in this example)
  34. 34. RUNTIME DITHERING
  35. 35. 50% GRAY
  36. 36. PRE-DITHERED FLOYD STEINBERG ORDERED DITHERING
  37. 37. PRE-DITHERED AT RUNTIME
  38. 38. PRE-DITHERED AT RUNTIME
  39. 39. PRE-DITHERED AT RUNTIME
  40. 40. PRE-DITHERED AT RUNTIME
  41. 41. PRE-DITHERED AT RUNTIME
  42. 42. 50% GRAY
  43. 43. PRE-DITHERED FLOYD STEINBERG ORDERED DITHERING
  44. 44. source: Joel Yliluoma’s dithering page - http://bisqwit.iki.fi/story/howto/dither/jy/ reduction to 18 colors
  45. 45. IMPLEMENT DITHERING AT RUNTIME
  46. 46. ORDERED DITHERING (BAYER MATRIX) threshold_map_4x4 = foreach y foreach x oldpixel := pixel[x][y] + threshold_map_4x4[x mod 4][y mod 4] newpixel := find_closest_palette_color(oldpixel) pixel[x][y] := newpixel ORDERED DITHERING BLACK & WHITE foreach y foreach x newpixel := pixel[x][y] > threshold_map_8x8[x mod 8][y mod 8] ? white : black; pixel[x][y] := newpixel
  47. 47. ORDERED DITHERING BLACK & WHITE foreach y foreach x newpixel := pixel[x][y] > threshold_map_8x8[x mod 8][y mod 8] ? white : black; pixel[x][y] := newpixel IMPLEMENTED IN C // Bayer matrix for ordered dithering static const uint8_t ditherMatrix[8][8] = { { 0*4, 32*4, 8*4, 40*4, 2*4, 34*4, 10*4, 42*4}, {48*4, 16*4, 56*4, 24*4, 50*4, 18*4, 58*4, 26*4}, {12*4, 44*4, 4*4, 36*4, 14*4, 46*4, 6*4, 38*4}, {60*4, 28*4, 52*4, 20*4, 62*4, 30*4, 54*4, 22*4}, { 3*4, 35*4, 11*4, 43*4, 1*4, 33*4, 9*4, 41*4}, {51*4, 19*4, 59*4, 27*4, 49*4, 17*4, 57*4, 25*4}, {15*4, 47*4, 7*4, 39*4, 13*4, 45*4, 5*4, 37*4}, {63*4, 31*4, 55*4, 23*4, 61*4, 29*4, 53*4, 21*4} }; ! static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray) { return gray > ditherMatrix[y % 8][x % 8] ? GColorWhite : GColorBlack; }
  48. 48. DOING LIVE DITHERING gray_top gray_bottom static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); ! static void update_proc_draw_pixel(Layer *layer, GContext *ctx) { ! ! ! } const int16_t h = window_size.h; for (int16_t y = 0; y < h; y++) { ! ! ! ! } uint8_t row_gray = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h);
  49. 49. AVOIDING FLOATS frac = 0.0 gray_top * (1 - 0.0) + 0.0 * gray_bottom frac = 0.25 gray_top * (1 - 0.25) + 0.25 * gray_bottom frac = 0.75 gray_top * (1 - 0.75) + 0.75 * gray_bottom frac = 1.0 gray_top * (1 - 1) + 1.0 * gray_bottom float frac = (float)y / h; // 0.0 … 1.0 row_gray = gray_top * (1 - frac) + frac * gray_bottom
  50. 50. AVOIDING FLOATS float frac = (float)y / h; // 0.0 … 1.0 row_gray = gray_top * (1 - frac) + frac * gray_bottom // 1 * h = h // frac * h = y uint8_t row_gray = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h);
  51. 51. DOING LIVE DITHERING gray_top gray_bottom static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); ! static void update_proc_draw_pixel(Layer *layer, GContext *ctx) { ! ! ! } const int16_t h = window_size.h; for (int16_t y = 0; y < h; y++) { ! ! ! ! } uint8_t row_gray = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h);
  52. 52. DOING LIVE DITHERING gray_top gray_bottom static GColor color_for_gray(int16_t x, int16_t y, uint8_t gray); ! static void update_proc_draw_pixel(Layer *layer, GContext *ctx) { const int16_t h = window_size.h; for (int16_t y = 0; y < h; y++) { uint8_t row_gray = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h); for (int16_t x = 0; x < window_size.w; x++) { graphics_context_set_stroke_color(ctx, color_for_gray(x, y, row_gray)); graphics_draw_pixel(ctx, GPoint(x, y)); } ! ! ! ! ! ! } ! }
  53. 53. source: https://www.youtube.com/watch?v=V5O8gj0SQZ0
  54. 54. DOING FAST(!) LIVE DITHERING GBitmap * graphics_capture_frame_buffer(GContext* ctx) Gives you a GBitmap to represent the current frame buffer bool graphics_release_frame_buffer(GContext* ctx, GBitmap* buffer) …and locks out any other graphic function until you release it
  55. 55. GBITMAP RECAP 0 4 8 12 16 29px 4 bytes per row 5px unused unused unused unused unused Fragment of 8 pixels in a row, stored in 8 bits or 1 byte (with value 0x5C). 0 1 0 1 1 1 0 0 Fragment of 8 pixels in a row, stored in 8 bits or 1 byte (with value 0x5C). 0 1 0 1 1 1 0 0 GBitmap typedef struct { void *addr; uint16_t row_size_bytes; // ... GRect bounds; } GBitmap; uint8_t *byte_offset = (uint8_t*)bmp->addr ; byte_offset += x / 8; byte_offset *= y * bmp->row_size_bytes;
  56. 56. DOING FAST(!) LIVE DITHERING static void update_proc_frame_buffer(Layer *layer, GContext *ctx) { GBitmap *fb = graphics_capture_frame_buffer(ctx); ! uint8_t *row = (uint8_t *)fb->addr; ! row += fb->bounds.origin.y * fb->row_size_bytes; ! ! ! ! ! ! ! ! ! ! ! ! ! ! const int16_t h = fb->bounds.size.h; for (int16_t y = fb->bounds.origin.y; y < h; y++) { uint8_t row_gray_value = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h); ! ! ! ! uint8_t row_gray_dither_pattern = d i t h e r _ p a t t e r n _ 8 (y, row_gray_value); row += fb->row_size_bytes; } ! ! ! graphics_release_frame_buffer(ctx, fb); }
  57. 57. RENDER 8 PIXELS AT ONCE static uint8_t (int16_t y, uint8_t gray) { return color_for_gray(0, y, gray) << 7 | color_for_gray(1, y, gray) << 6 | color_for_gray(2, y, gray) << 5 | color_for_gray(3, y, gray) << 4 | color_for_gray(4, y, gray) << 3 | color_for_gray(5, y, gray) << 2 | color_for_gray(6, y, gray) << 1 | color_for_gray(7, y, gray) << 0; } dither_pattern_8
  58. 58. DOING FAST(!) LIVE DITHERING static void update_proc_frame_buffer(Layer *layer, GContext *ctx) { GBitmap *fb = graphics_capture_frame_buffer(ctx); ! uint8_t *row = (uint8_t *)fb->addr; ! row += fb->bounds.origin.y * fb->row_size_bytes; ! ! ! ! ! ! ! ! ! ! ! ! ! ! const int16_t h = fb->bounds.size.h; for (int16_t y = fb->bounds.origin.y; y < h; y++) { uint8_t row_gray_value = (uint8_t)((gray_top * (h - y) + y * gray_bottom) / h); ! ! ! ! uint8_t row_gray_dither_pattern = d i t h e r _ p a t t e r n _ 8 (y, row_gray_value); memset(row, row_gray_dither_pattern, fb->row_size_bytes); row += fb->row_size_bytes; } ! ! ! graphics_release_frame_buffer(ctx, fb); }
  59. 59. source: https://www.youtube.com/watch?v=V5O8gj0SQZ0
  60. 60. BOING BALL
  61. 61. Q&A source: https://www.youtube.com/watch?v=Z2QMG9UwXI0

×