Considering Tiling for a better User Interface

Considering Tiling for a better User Experience Kenneth Rohde Christiansen WebKit Code Camp, Wiesbaden, December 2009

SOMETHING THAT I HAVE BEEN WORKING ON

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I?

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT 3+ YEARS WORKING WITH APP & FRAMEWORK DEVELOPMENT AT

SOMETHING THAT I HAVE BEEN WORKING ON WHO AM I? KENNETH R. CHRISTIANSEN WEBKIT REVIEWER 1+ YEAR WORKING WITH WEBKIT 3+ YEARS WORKING WITH APP & FRAMEWORK DEVELOPMENT AT NOKIA TECHNOLOGY INSTITUTE, INdT, RECIFE, BRAZIL

A TILED BACKING STORE

What are the problems we are facing today? The background for considering tiling

What are the problems we are facing today? The background for considering tiling WE PAINT THINGS WE’VE ALREADY PAINTED WHEN SCROLLING BACK

What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK

What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK ...CALLING INTO WEBCORE HAS SOME OVERHEAD

What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK FOR INSTANCE, CONSTRUCTING GRAPHICS CONTEXT IS QUITE EXPENSIVE ...CALLING INTO WEBCORE HAS SOME OVERHEAD

What are the problems we are facing today? The background for considering tiling WE CALL INTO WEBCORE FOR EACH WE PAINT THINGS WE’VE PAINT... ALREADY PAINTED WHEN SCROLLING BACK FOR INSTANCE, CONSTRUCTING GRAPHICS CONTEXT IS QUITE EXPENSIVE ...CALLING INTO WEBCORE HAS SOME OVERHEAD CLEVER TILING CAN SOLVE THESE ISSUES

How to accomplish this What is tiling anyway? Cache and join paint events

How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles

How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll

How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately

How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately Avoid too many small tiles, due to the cost of constructing GraphicsContexts

How to accomplish this What is tiling anyway? Cache and join paint events Cache what you paint in image tiles Blit the existing tiles on scroll Don’t paint non-visible dirty areas immediately Avoid too many small tiles, due to the cost of constructing GraphicsContexts This can be hardware accelerated!

My experiment My basic algorithm As an experiment, implement it on the WebKit side

My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView

My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping

My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport

My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport Why not just make viewport == contents size?

My experiment My basic algorithm As an experiment, implement it on the WebKit side Implemented only for QGraphicsWebView Some changes needed elsewhere: 1) Render methods in abs. coordinates, without clipping 2) Make ScrollView / FrameView send update events outside of the viewport Why not just make viewport == contents size? Because we use WebCore for drawing our scrollbars and that makes a whole lot of sense ... theming ... etc

My experiment My basic algorithm Basic algorithm

My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction)

My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen

My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area.

My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area. On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rect is uses as the new area. Remember, we try to avoid calling into WebCore unnecessarily

My experiment My basic algorithm Basic algorithm If not in cache, paint the dirty area as a tile, enlarge it slightly (64 pixels in each direction) Put in cache, blit to screen Each tile stores it’s covered area as well as a dirty area. On update, we update the dirty area of the intersected tiles. If it has such an area already, the bounding rect is uses as the new area. Remember, we try to avoid calling into WebCore unnecessarily Furthermore, if the dirty area == covered area, remove it from cache

My experiment My basic algorithm Basic algorithm

My experiment My basic algorithm Basic algorithm On update and scroll: Update all tiles, blit what is in the cache, create tiles for what is not

My experiment My basic algorithm Basic algorithm On update and scroll: Update all tiles, blit what is in the cache, create tiles for what is not That is more of less the basic algorithm, but there are some problems.

My experiment My basic algorithm Problems

My experiment My basic algorithm Problems Scrollbars Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation!

My experiment My basic algorithm Problems Scrollbars Paint the scrollbars separately, make sure updates to them do not invalidate any tiles. These are not tiled in my implementation! Enlarging tiles When enlarging we much make sure that we don’t cover an area already in the cache, so I need to know what area is cached.

My experiment My basic algorithm Problems

My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size.

My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size. The solution is to give each tile an age and increase the age when not used, and reset it when being blit.

My experiment My basic algorithm Problems Cache growth Storing the whole page in the cache would be expensive, memory-wise, so the cache has a max size. The solution is to give each tile an age and increase the age when not used, and reset it when being blit. Before adding a new tile, we reserve the needed space for it, removing the oldest tiles.

So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising.

So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS RESONABLE, DUE TO THE XCOPYREGION

So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS THE TILING IS FASTER! RESONABLE, DUE TO THE XCOPYREGION

So what is the preliminary results? Did it really pay off? Benjamin wrote a simple scrolling test app, and the results are quite promising. QWEBVIEW PERFORMS THE TILING IS FASTER! RESONABLE, DUE TO THE XCOPYREGION But we had some surprises as well. NON-TILED QGRAPHICSWEBVIEW IS EXTREMELY SLOW

Where are se’ numbers? Back up your claims dude! Simple painting QWebView QGraphicsWebView Tiled QGraphicsWebView 0 2,5 5,0 7,5 10,0 Milliseconds

Where are se’ numbers? Back up your claims dude! Slashdot page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 25 50 75 100 Milliseconds

Where are se’ numbers? Back up your claims dude! Amazon book page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 22,5 45,0 67,5 90,0 Milliseconds

Where are se’ numbers? Back up your claims dude! Wikipedia Qt page QWebView QGraphicsWebView Tiled QGraphicsWebView 0 37,5 75,0 112,5 150,0 Milliseconds

Can’t you do better than that? Ideas for improvement WE CAN!

Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite!

Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor proﬁled

Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor proﬁled Should be lower in the stack using WebCore constructs

Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor proﬁled Should be lower in the stack using WebCore constructs Paint in another thread, not block WebCore

Can’t you do better than that? Ideas for improvement WE CAN! More realistic test suite! My data structures are not that good, nor proﬁled Should be lower in the stack using WebCore constructs Paint in another thread, not block WebCore That is why we are here ;-) Now let’s get on to the work!

Thanks for listening KENNETH ROHDE CHRISTIANSEN ext-kenneth.christiansen@nokia.com kenneth.christiansen@openbossa.org

http://codeposts.blogspot.com	1226
http://maemo.org	95
http://codeposts.blogspot.fr	85
http://codeposts.blogspot.in	57
http://codeposts.blogspot.de	28
http://codeposts.blogspot.it	22
http://codeposts.blogspot.co.uk	17
http://codeposts.blogspot.com.br	16
http://codeposts.blogspot.tw	13
http://codeposts.blogspot.ca	12
http://codeposts.blogspot.sg	11
http://codeposts.blogspot.fi	11
http://www.slideshare.net	10
http://qtlabs.openbossa.org	8
http://codeposts.blogspot.jp	8
http://codeposts.blogspot.dk	7
http://codeposts.blogspot.co.at	7
http://codeposts.blogspot.kr	7
http://codeposts.blogspot.ru	6
http://codeposts.blogspot.com.au	6
http://www.linkedin.com	5
http://codeposts.blogspot.ro	5
http://codeposts.blogspot.gr	5
http://codeposts.blogspot.nl	5
http://webcache.googleusercontent.com	5
http://codeposts.blogspot.sk	4
http://codeposts.blogspot.hu	4
http://codeposts.blogspot.com.es	3
http://codeposts.blogspot.be	3
http://translate.googleusercontent.com	3
http://codeposts.blogspot.se	2
http://dcdemo.digitalchalk.com	2
http://www.lmodules.com	2
http://xss.yandex.net	2
http://codeposts.blogspot.pt	1
http://codeposts.blogspot.ch	1
http://codeposts.blogspot.hk	1
http://codeposts.blogspot.cz	1
http://codeposts.blogspot.no	1
http://tecnojosejavier.blogspot.com	1
http://river2.rien.tv	1
http://www.codeposts.blogspot.jp	1
http://codeposts.blogspot.co.il	1

Considering Tiling for a better User Interface

by Kenneth Rohde Christiansen

Accessibility

Categories

Upload Details

Usage Rights

43 Embeds 1,711

Statistics