Considering Tiling for a
better User Experience

Kenneth Rohde Christiansen
WebKit Code Camp, Wiesbaden, December 2009
SOMETHING THAT I HAVE BEEN WORKING ON
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I?
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER
1+ YEAR WORKING WITH WEBKIT
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER
1+ YEAR WORKING WITH WEBKIT 3+ YE...
SOMETHING THAT I HAVE BEEN WORKING ON

WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER
1+ YEAR WORKING WITH WEBKIT 3+ YE...
A TILED
BACKING STORE
What are the problems we are facing today?
The background for considering tiling
What are the problems we are facing today?
The background for considering tiling




            WE PAINT THINGS WE’VE
   ...
What are the problems we are facing today?
The background for considering tiling




                                     ...
What are the problems we are facing today?
The background for considering tiling




                                     ...
What are the problems we are facing today?
The background for considering tiling




                                     ...
What are the problems we are facing today?
The background for considering tiling




                                     ...
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
                         Cache what you ...
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
                         Cache what you ...
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
                         Cache what you ...
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
                         Cache what you ...
How to accomplish this
What is tiling anyway?




  Cache and join paint
  events
                         Cache what you ...
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
        Implemented only for QGr...
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
        Implemented only for QGr...
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
        Implemented only for QGr...
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
        Implemented only for QGr...
My experiment
My basic algorithm



    As an experiment, implement it on the WebKit side
        Implemented only for QGr...
My experiment
My basic algorithm



    Basic algorithm
My experiment
My basic algorithm



    Basic algorithm

        If not in cache, paint the dirty area as a tile, enlarge ...
My experiment
My basic algorithm



    Basic algorithm

        If not in cache, paint the dirty area as a tile, enlarge ...
My experiment
My basic algorithm



    Basic algorithm

        If not in cache, paint the dirty area as a tile, enlarge ...
My experiment
My basic algorithm



    Basic algorithm

        If not in cache, paint the dirty area as a tile, enlarge ...
My experiment
My basic algorithm



    Basic algorithm

        If not in cache, paint the dirty area as a tile, enlarge ...
My experiment
My basic algorithm



    Basic algorithm
My experiment
My basic algorithm



    Basic algorithm

        On update and scroll:
        Update all tiles, blit what...
My experiment
My basic algorithm



    Basic algorithm

        On update and scroll:
        Update all tiles, blit what...
My experiment
My basic algorithm



    Problems
My experiment
My basic algorithm



    Problems

        Scrollbars
        Paint the scrollbars separately, make sure up...
My experiment
My basic algorithm



    Problems

        Scrollbars
        Paint the scrollbars separately, make sure up...
My experiment
My basic algorithm



    Problems
My experiment
My basic algorithm



    Problems

        Cache growth
        Storing the whole page in the cache would b...
My experiment
My basic algorithm



    Problems

        Cache growth
        Storing the whole page in the cache would b...
My experiment
My basic algorithm



    Problems

        Cache growth
        Storing the whole page in the cache would b...
So what is the preliminary results?
Did it really pay off?



      Benjamin wrote a simple scrolling test app, and the
  ...
So what is the preliminary results?
Did it really pay off?



      Benjamin wrote a simple scrolling test app, and the
  ...
So what is the preliminary results?
Did it really pay off?



      Benjamin wrote a simple scrolling test app, and the
  ...
So what is the preliminary results?
Did it really pay off?



      Benjamin wrote a simple scrolling test app, and the
  ...
Where are se’ numbers?
Back up your claims dude!




     Simple painting


               QWebView



      QGraphicsWebV...
Where are se’ numbers?
Back up your claims dude!




     Slashdot page


               QWebView



      QGraphicsWebVie...
Where are se’ numbers?
Back up your claims dude!




     Amazon book page


               QWebView



      QGraphicsWeb...
Where are se’ numbers?
Back up your claims dude!




     Wikipedia Qt page


               QWebView



      QGraphicsWe...
Can’t you do better than that?
Ideas for improvement




    WE CAN!
Can’t you do better than that?
Ideas for improvement




    WE CAN!

       More realistic test suite!
Can’t you do better than that?
Ideas for improvement




    WE CAN!

       More realistic test suite!
       My data str...
Can’t you do better than that?
Ideas for improvement




    WE CAN!

       More realistic test suite!
       My data str...
Can’t you do better than that?
Ideas for improvement




    WE CAN!

       More realistic test suite!
       My data str...
Can’t you do better than that?
Ideas for improvement




    WE CAN!

       More realistic test suite!
       My data str...
Thanks for listening

KENNETH ROHDE CHRISTIANSEN
ext-kenneth.christiansen@nokia.com
kenneth.christiansen@openbossa.org
Upcoming SlideShare
Loading in …5
×

Considering Tiling for a better User Interface

7,821 views

Published on

Published in: Technology, Art & Photos
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,821
On SlideShare
0
From Embeds
0
Number of Embeds
2,437
Actions
Shares
0
Downloads
87
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Considering Tiling for a better User Interface

  1. Considering Tiling for a better User Experience Kenneth Rohde Christiansen WebKit Code Camp, Wiesbaden, December 2009
  2. SOMETHING THAT I HAVE BEEN WORKING ON
  3. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I?
  4. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN
  5. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER
  6. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT
  7. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT 3+ YEARS WORKING WITH APP & FRAMEWORK DEVELOPMENT AT
  8. SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT 3+ YEARS WORKING WITH APP & FRAMEWORK DEVELOPMENT AT NOKIA TECHNOLOGY INSTITUTE, INdT, RECIFE, BRAZIL
  9. A TILED BACKING STORE
  10. What are the problems we are facing today? The background for considering tiling
  11. What are the problems we are facing today? The background for considering tiling WE PAINT THINGS WE’VE ALREADY PAINTED WHEN SCROLLING BACK
  12. What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK
  13. What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK ...CALLING INTO WEBCORE HAS SOME OVERHEAD
  14. What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK FOR INSTANCE, CONSTRUCTING GRAPHICS CONTEXT IS QUITE EXPENSIVE ...CALLING INTO WEBCORE HAS SOME OVERHEAD
  15. What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK FOR INSTANCE, CONSTRUCTING GRAPHICS CONTEXT IS QUITE EXPENSIVE ...CALLING INTO WEBCORE HAS SOME OVERHEAD CLEVER TILING CAN SOLVE THESE ISSUES
  16. How to accomplish this What is tiling anyway? Cache and join paint events
  17. How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles
  18. How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll
  19. How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately
  20. How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately Avoid too many small tiles, due to the cost of constructing GraphicsContexts
  21. How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately Avoid too many small tiles, due to the cost of constructing GraphicsContexts This can be hardware accelerated!
  22. My experiment My basic algorithm As an experiment, implement it on the WebKit side
  23. My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView
  24. My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping
  25. My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport
  26. My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport Why not just make viewport == contents size?
  27. My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport Why not just make viewport == contents size? Because we use WebCore for drawing our scrollbars and that makes a whole lot of sense ... theming ... etc
  28. My experiment My basic algorithm Basic algorithm
  29. My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)
  30. My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen
  31. My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area.
  32. My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area. On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rect is uses as the new area. Remember, we try to avoid calling into WebCore unnecessarily
  33. My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area. On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rect is uses as the new area. Remember, we try to avoid calling into WebCore unnecessarily Furthermore, if the dirty area == covered area, remove it from cache
  34. My experiment My basic algorithm Basic algorithm
  35. My experiment My basic algorithm Basic algorithm On update and scroll: Update all tiles, blit what is in the cache, create tiles for what is not
  36. My experiment My basic algorithm Basic algorithm On update and scroll: Update all tiles, blit what is in the cache, create tiles for what is not That is more of less the basic algorithm, but there are some problems.
  37. My experiment My basic algorithm Problems
  38. My experiment My basic algorithm Problems Scrollbars Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation!
  39. My experiment My basic algorithm Problems Scrollbars Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation! Enlarging tiles When enlarging we much make sure that we don’t cover an area already in the cache, so I need to know what area is cached.
  40. My experiment My basic algorithm Problems
  41. My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size.
  42. My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size. The solution is to give each tile an age and increase the age when not used, and reset it when being blit.
  43. My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size. The solution is to give each tile an age and increase the age when not used, and reset it when being blit. Before adding a new tile, we reserve the needed space for it, removing the oldest tiles.
  44. So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising.
  45. So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS RESONABLE, DUE TO THE XCOPYREGION
  46. So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS THE TILING IS FASTER! RESONABLE, DUE TO THE XCOPYREGION
  47. So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS THE TILING IS FASTER! RESONABLE, DUE TO THE XCOPYREGION But we had some surprises as well. NON-TILED QGRAPHICSWEBVIEW IS EXTREMELY SLOW
  48. Where are se’ numbers? Back up your claims dude! Simple painting QWebView QGraphicsWebView Tiled QGraphicsWebView 0 2,5 5,0 7,5 10,0 Milliseconds
  49. Where are se’ numbers? Back up your claims dude! Slashdot page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 25 50 75 100 Milliseconds
  50. Where are se’ numbers? Back up your claims dude! Amazon book page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 22,5 45,0 67,5 90,0 Milliseconds
  51. Where are se’ numbers? Back up your claims dude! Wikipedia Qt page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 37,5 75,0 112,5 150,0 Milliseconds
  52. Can’t you do better than that? Ideas for improvement WE CAN!
  53. Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite!
  54. Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor profiled
  55. Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor profiled Should be lower in the stack using WebCore constructs
  56. Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor profiled Should be lower in the stack using WebCore constructs Paint in another thread, not block WebCore
  57. Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor profiled Should be lower in the stack using WebCore constructs Paint in another thread, not block WebCore That is why we are here ;-) Now let’s get on to the work!
  58. Thanks for listening KENNETH ROHDE CHRISTIANSEN ext-kenneth.christiansen@nokia.com kenneth.christiansen@openbossa.org

×