Android Chromium Rendering Pipeline

1. Android Chromium Rendering Pipeline 이형욱 (hyungwook.lee@navercorp.com) 시스템스컴퓨팅G / NAVER LABS 2015-04

2. 2 0. Contents Chapter 1: 브라우저 렌더링에서의 Critical Path란 무엇인가? ① Summary of browser’s main flows ② HTML Parser ③ JavaScriptEngine ④ Render Tree, Layout ⑤ Rendering Pipeline Runs on Main Thread ⑥ New Rendering Model ⑦ GPU Accelerated Compositing Chapter 2: Chromium은 렌더링 성능 향상을 위하여… ① Chrome Vs Chromium ② Why Multi Process Architecture? ③ GPU Process ④ Multi Threads Models in Renderer Process ⑤ Chromium Scheduler: Right things right time Chapter 3: Chromium-powered Android WebView ① What is Android WebView? ② Android WebView Version History ③ Chromium Powered WebView Structure ④ Chrome Vs Android Rendering model ⑤ Chrome Vs Chromium WebView ⑥ Chromium WebView Rendering Pipeline ⑦ The Future of WebView?

3. 3 Chapter 1, 브라우저 렌더링에 서의 Critical Path란 무엇인가?

4. 4 <html> <head> <title> NAVER </title> </head> <body> <div> <h1> Hello </h1> <p> World </p> </div> </body> </html> 1. Summary of browser main flows. [Source: How browsers work]

5. 5 HEAD TITLE NAVER BODY DIV H1 P HTML Hello World <html> <head> <title> NAVER </title> </head> <body> <div> <h1> Hello </h1> <p> World </p> </div> </body> </html> 2. HTML Parser

6. 6 3. Java Script Engine Source Code Parser Syntax Tree Bytecode Generator Bytecode JIT Compiler Machine Code Execution Loader Parser DOM Java Script HTML DOM API Evaluate

7. 7 HTML CSS DOM Tree Style Rules Renderer TreeLookup 4. Render Tree Creation

8. 8 HEAD TITLE NAVER BODY DIV H1 P HTML Hello World Block Block Block Block Block Inline Inline 5. Render Tree

9. 9 6. Layout HEAD TITLE NAVER BODY DIV H1 P HTML Hello World Block Block Block Block Block Inline Inline (0, 0, 1024, 768) (0, 0, 1024, 768) (0, 0, 1024, 55) (10, 20, 1024, 37) (10, 80, 1024, 18)

10. 10 7. Rendering Block (html) Block (body) Block (div) Block (h1) Block (p) Inline (hello) Inline (world) (0, 0, 1024, 768) (0, 0, 1024, 768) (0, 0, 1024, 55) (10, 20, 1024, 37) (10, 80, 1024, 18) 1024 768<div>

11. 11 8. Rendering Pipeline Runs on Main Thread [Source: Blink Rendering Pipeline] [Source: Chrome Graphics - BlinkOn 1]

12. 12 9. New Rendering Model: Layers (1/4) [Source: WebKit.org]

13. 13 9. New Rendering Model: Layers (2/4) 1. Avoid unnecessary repainting - If yellow and red have their own layers, then nothing needs "repainting" while this example animates. 2. Makes some features more efficient or practical - Including: Scrolling, 3D CSS(translate3d, …), opacity, filters, WebGL, etc. [Source: Compositing in Blink and WebKit]

14. 14 9. New Rendering Model: Layers (3/4) 1. It's the root object for the page. <html> 2. CSS position properties (relative, absolute) 3. Has overflow, an alpha mask or reflection 4. CSS filter 5. CSS 3D Transform , Animations 6. <canvas>, <video> 7. will-change (Chrome 36, Firefox 36, Opera 24)

15. 15 Cost of Multi Layers: Memory, Computing <Layer를 너무 많이 사용할 경우 Compositing 시간이 오래 걸림> 9. New Rendering Model: Layers (4/4)

16. 16 10. GPU Accelerated Compositing 1. 각 Layer 별 bitmap을 생성 (Render Layer traversing) 2. Text와 Image는 CPU에서 Rasterization 3. Textures upload into Video memory 4. Compositor에서 Layer 순서에 맞게 Texture를 drawing하여 화면에 Update

17. 17 Chapter 2, Chromium은 렌더 링 성능 향상을 위하여 어떤 기 술들을 사용하고 있는가?

18. 18 Google Chrome Chromium Logo Colorful Blue Crash reporting Yes, if you turn it on None User metrics Yes, if you turn it on No Video and audio tags AAC, MP3, Vorbis and Theora Vorbis and Theora by default Adobe Flash custom (non-free) plugin included in release supports NPAPI plugins, including the one from Adobe PDF support custom (non-free) plugin included in release downloads and displays with system PDF viewer Code Tested by Chrome developers May be modified by distributions Sandbox Always on Depending on the distribution Quality Assurance New releases are tested before sending to users Depending on the distribution 1. Chrome Vs Chromium

19. 19 2. Why Multi Process Architecture? 기존 브라우저의 문제점 • 현재(크롬 제작 당시)의 브라우저는 과거의 OS와 비슷한 구조 • 과거의 OS는 단일 사용자, 협조적 멀티 태스킹 시스템 • 이와 같은 시스템에서, 어떤 어플리케이션의 문제는 OS 전체의 안정성에 영향을 미침 • 단일 프로세스 모델의 브라우저에서는 한 페이지의 문제가 브라우저 전체에 문제를 일으킴 문제해결을 위한 접근 • 절대적으로 안정적인 렌더링 엔진을 만드는 것은 거의 불가능함 • 최근의 OS는 각 어플리케이션을 고립시킴으로써 특정 어플리케이션의 문제가 시스템 전체의 안정 성에 영향을 미치지 않도록 하며, 사용자 각자의 데이터를 보호하고 있음

20. 20 3. Chromium Rendering Pipeline [Source: Chrome Graphics - BlinkOn 1]

21. 21 • GPU Process의 구조적 특징: Security, Robustness, Uniformity, Parallelism • Client(Render Process): GL Command를 생성, Ring Buffer에 Insert -> 일반적인 페이지의 경우 Texture Upload용으로 사용, HTML5 Canvas / WebGL은 직접 GL Command를 사용 • Server(GPU Process): Command를 fetch, execution. • GL command는 GL ES 2.0 API와 거의 유사한 Async command 4. GPU Process [Source: GPU Accelerated Compositing in Chrome]

22. 22 5. Multi Threads: Main Thread Rendering [Source: Chrome Graphics - BlinkOn 1]

23. 23 5. Multi Threads: Threaded Compositor [Source: Chrome Graphics - BlinkOn 1]

24. 24 5. Multi Threads: Threaded Rasterization [Source: Chrome Graphics - BlinkOn 1]

25. 25 6. Skia’s SkPicture Canvas A modern 2D graphics library SkPicture, SkPicturePlayBack: Records and replay draw operations. [http://www.dorothybrowser.com/parallelizing-canvas-rendering/]

26. 26 7. Chromium Rendering Pipeline in detail 1. Record 2. Raster 2.5 Upload 3. Composite update record Thread Proxy update recordPicture Layer updatePicture LayerImpl Raster LayerTreeHost update recordupdate recordGraphics Layer Blink LayerTreeHostImpl Proxy Main Thread Compositor Thread Raster Threads Call into WebKit which calls Skia SkPicture records the paint for deferred raster Decode images In parallel Raster into shared memory Send to GPU process using custom extension GPU process does glTexImage2d asynchronously Send draw cmds To GPU Process over command buffer glBindTexture glDrawElements Main Thread Raster Thread Compositor Thread Tree Sync

27. 27 8. Chromium Scheduler: Right things right time 1. Record 3. Raster 4. Upload 5. Composite 2. Tree Sync 6. SwapBuffer Scheduler Begin Frame Commit Pending Tree Draw & Swap Buffer Update Visible Tiles <StateMachine in Compositor Threads> Activate Pending Tree VSync

28. 28 Renderer Process Cache Thread File Thread DB Thread IO Thread I/O Thread Browser Process Main Thread Renderer Process IPC RenderView RenderWidget Blink RenderView RenderWidget RenderWidget Blink Blink RenderView WebContents RenderViewHost Manager RenderViewHost RenderWidgetHost RenderViewHost RenderWidgetHost RenderViewHost RenderWidgetHost SiteInstance RenderProcessHost SiteInstance RenderProcessHost Main ThreadMain Thread WebContents RenderViewHost Manager RenderMainThread (Single Process Mode) 1 2 3 4 5 6 Compositor GPU Thread (Android) Process Launcher Thread Raster I/O Thread Compositor Raster 9. Chromium Architecture in detail

29. 29 Chapter 3, Android Chrome과 Chromium-powered WebView 의 구조적 차이는 무엇인가?

30. 30 1. What is Android WebView? Web content in a box: - Powers browsers such as AOSP Browser - Displays almost all mobile banner ads - Can be used to create portable HTML-based apps - Often intermixed imperceptibly with native Views <- Android View <- Android WebView

31. 31 2. Android WebView Version History Android <= J: custom WebKit-based “classic” WebView Android K: Chromium 30 / 33-based WebView Android L: Unbundled Evergreen WebView http://com.odroid.com/sigong/nf_file_board/nfile_board_view.php?keyword=&tag= ODROID-U3&bid=214 http://www.ibtimes.co.uk/moto-g-finally-tastes-android-5-0-lollipop-via-cyanogenmod-12-unofficial- build-1475895

32. 32 3. Chromium Powered WebView Structure Blink(WebKit) Content (Multi-process-impl) Content API Blink Chromium Android WebView Framework Chrome Browser Framework

33. 33 4. Chrome Vs Android Rendering Model Make main thread free Make sure contents on time

34. 34 5. Chrome Vs Chromium WebView Chrome Multi-process  UI thread  GPU thread  Texture upload thread  Per renderer process: o Blink thread o CC impl thread o Raster thread Chromium Scheduler SurfaceView Chromium WebView Single-process  Combined UI + renderer CC thread  Android RenderThread (+in-proces s GPU thread)  Canvas/WebGL GPU thread  Blink thread  Raster thread Android Rendering Scheduer Draw functor (Private API) [Source: Android WebView rendering BlinkOn 3]

35. 35 6. Accelerated Android Rendering [Source: http://www.slideshare.net/deview/1d6review-of-android-l-developer-preview] [MainThread]

36. 36 7. Chromium WebView Rendering Pipeline in K DrawGLFunction() Synchronous Compositor ViewRootImpl performDraw() View Draw(Canvas) View onDraw(Canvas) HardwareCanvas draw*(…,Paint) WebView::onDraw() Set draw functor via DrawGLFunctor::requestDrawGL() (canvas.callDrawGLFunction(mNativeDrawGLFunctor)) 3. Composite Chromium WebView 1. Record 2. Raster Main Blink Thread Raster Threads Main Thread

37. 37 8. Chromium WebView Rendering Pipeline in L WebView::onDraw() 1. Set draw functor 2. Request Record & Raster Synchronous Compositor View onDraw(Canvas) HardwareCanvas draw*(…,Paint) Chromium WebView 1. Record 2. Raster Main Raster Threads ViewRootImpl performDraw() Render Blink Thread DrawGLFunction() Synchronous Compositor 3. Composite Render Thread

38. 38 9. Android Chromium WebView Architecture Blink Thread Cache Thread File Thread DB Thread IO Thread Main Thread IPC WebContents RenderViewHost Manager RenderViewHost RenderWidgetHost RenderViewHost RenderWidgetHost RenderViewHost RenderWidgetHost SiteInstance RenderProcessHost SiteInstance RenderProcessHost Chromium WebView’s Process RenderView RenderWidget Blink Process Launcher Thread Raster Thread WebView

39. 39 10. The Future of WebView? “Under active debate at the moment” (2014/11) 1. Opposing use cases 1) WebView as a browser-in-a-box, Vs 2) WebView as one tool in a hybrid app toolset 2. Opposing update priorities 1) App developers want the latest features and performance improvements 2) They also want stability to the point of bug-for-bug compatibility [Source: Android WebView rendering BlinkOn 3]

40. 40 Appendix #1: References 1. http://www.w3.org/TR/CSS2/ 2. http://www.w3.org/TR/DOM-Level-2-Core/core.html 3. http://www.html5rocks.com/en/tutorials/internals/howbrowserswork/ 4. http://deview.kr/2013/detail.nhn?topicSeq=5 5. http://www.slideshare.net/joone/hardware-acceleration-in-webkit 6. http://dev.chromium.org/developers/design-documents/multi-process-architecture 7. https://docs.google.com/a/chromium.org/presentation/d/1pYAGn2AYJ7neFDlDZ9DmL HpwMIskzMUXjFXYR7yfUko/edit 8. http://dev.chromium.org/developers/design-documents/inter-process-communication 9. http://dev.chromium.org/developers/design-documents/multi-process-resource- loading 10. http://dev.chromium.org/developers/design-documents/gpu-accelerated- compositing-in-chrome 11. http://dev.chromium.org/developers/design-documents/compositor-thread- architecture 12. http://dev.chromium.org/developers/design-documents/impl-side-painting 13. http://blog.csdn.net/jaylinzhou/article/details/18313505

41. 41

Android Chromium Rendering Pipeline

You are reading a preview.

Check these out next

Android Chromium Rendering Pipeline

More Related Content

Slideshows for you (20)

Viewers also liked (20)

Similar to Android Chromium Rendering Pipeline (20)

Recently uploaded (20)