Introduction to Skia by Ryan Chou @20141008Ryan Chou
This introduces the fundamental knowledge about Skia which is open-source project used in Android. In this, it contains the history of skia, and the roll of skia in Android.
Android graphic system (SurfaceFlinger) : Design Pattern's perspectiveBin Chen
SurfaceFlinger is a vital system service in Android system, responsible for the composting all the application and system layer and displaying them. In this slide,we looked in detail how surfaceFlinger was designed from Design Pattern's perspective.
Slides from Android Builder's Summit 2014 in San Jose, CA
In this talk I describe the internal workings of the Android graphics stack from the Application layer down through the stack to pixels on the screen. It is a fairly complex journey, taking in two different 2D rendering engines, applications calling OpenGL ES directory, passing buffers on to the system compositor, SurfaceFlinger, and then down to the display controller or frame buffer.
Introduction to Skia by Ryan Chou @20141008Ryan Chou
This introduces the fundamental knowledge about Skia which is open-source project used in Android. In this, it contains the history of skia, and the roll of skia in Android.
Android graphic system (SurfaceFlinger) : Design Pattern's perspectiveBin Chen
SurfaceFlinger is a vital system service in Android system, responsible for the composting all the application and system layer and displaying them. In this slide,we looked in detail how surfaceFlinger was designed from Design Pattern's perspective.
Slides from Android Builder's Summit 2014 in San Jose, CA
In this talk I describe the internal workings of the Android graphics stack from the Application layer down through the stack to pixels on the screen. It is a fairly complex journey, taking in two different 2D rendering engines, applications calling OpenGL ES directory, passing buffers on to the system compositor, SurfaceFlinger, and then down to the display controller or frame buffer.
This presentation covers the working model about Process, Thread, system call, Memory operations, Binder IPC, and interactions with Android frameworks.
Binder is what differentiates Android from Linux, it is most important internal building block of Android, it is a subject every Android programmer should be familiar with
Introduction to Flutter - truly crossplatform, amazingly fastBartosz Kosarzycki
Intro: Flutter meaning rapid variation of electronic signal recently became Dart's framework name for mobile development. This presentation is a short introduction into a cross-platform solution covering iOS/Android. During this 45 minute period you'll learn what is flutter, where it came from and what it's for.
SIGGRAPH Asia 2012: GPU-accelerated Path RenderingMark Kilgard
Presented at SIGGRAPH Asia 2012 in Singapore on Friday, 30 November 14:15 - 16:00 during the "Points and Vectors" session.
Find the paper at http://developer.nvidia.com/game/gpu-accelerated-path-rendering or on Slideshare.
For thirty years, resolution-independent 2D standards (e.g. PostScript, SVG) have relied largely on CPU-based algorithms for the filling and stroking of paths. Learn about our approach to accelerate path rendering with our GPU-based "Stencil, then Cover" programming interface. We've built and productized our OpenGL-based system.
SIGGRAPH 2012: GPU-Accelerated 2D and Web RenderingMark Kilgard
Video replay: http://nvidia.fullviewmedia.com/siggraph2012/ondemand/SS106.html
Location: West Hall Meeting Room 503, Los Angeles Convention Center
Date: Wednesday, August 8, 2012
Time: 2:40 PM – 3:40 PM
The future of GPU-based visual computing integrates the web, resolution-independent 2D graphics, and 3D to maximize interactivity and quality while minimizing consumed power. See what NVIDIA is doing today to accelerate resolution-independent 2D graphics for web content. This presentation explains NVIDIA's unique "stencil, then cover" approach to accelerating path rendering with OpenGL and demonstrates the wide variety of web content that can be accelerated with this approach.
More information: http://developer.nvidia.com/nv-path-rendering
This presentation covers the working model about Process, Thread, system call, Memory operations, Binder IPC, and interactions with Android frameworks.
Binder is what differentiates Android from Linux, it is most important internal building block of Android, it is a subject every Android programmer should be familiar with
Introduction to Flutter - truly crossplatform, amazingly fastBartosz Kosarzycki
Intro: Flutter meaning rapid variation of electronic signal recently became Dart's framework name for mobile development. This presentation is a short introduction into a cross-platform solution covering iOS/Android. During this 45 minute period you'll learn what is flutter, where it came from and what it's for.
SIGGRAPH Asia 2012: GPU-accelerated Path RenderingMark Kilgard
Presented at SIGGRAPH Asia 2012 in Singapore on Friday, 30 November 14:15 - 16:00 during the "Points and Vectors" session.
Find the paper at http://developer.nvidia.com/game/gpu-accelerated-path-rendering or on Slideshare.
For thirty years, resolution-independent 2D standards (e.g. PostScript, SVG) have relied largely on CPU-based algorithms for the filling and stroking of paths. Learn about our approach to accelerate path rendering with our GPU-based "Stencil, then Cover" programming interface. We've built and productized our OpenGL-based system.
SIGGRAPH 2012: GPU-Accelerated 2D and Web RenderingMark Kilgard
Video replay: http://nvidia.fullviewmedia.com/siggraph2012/ondemand/SS106.html
Location: West Hall Meeting Room 503, Los Angeles Convention Center
Date: Wednesday, August 8, 2012
Time: 2:40 PM – 3:40 PM
The future of GPU-based visual computing integrates the web, resolution-independent 2D graphics, and 3D to maximize interactivity and quality while minimizing consumed power. See what NVIDIA is doing today to accelerate resolution-independent 2D graphics for web content. This presentation explains NVIDIA's unique "stencil, then cover" approach to accelerating path rendering with OpenGL and demonstrates the wide variety of web content that can be accelerated with this approach.
More information: http://developer.nvidia.com/nv-path-rendering
Presented at the GPU Technology Conference 2012 in San Jose, California.
Tuesday, May 15, 2012.
Standards such as Scalable Vector Graphics (SVG), PostScript, TrueType outline fonts, and immersive web content such as Flash depend on a resolution-independent 2D rendering paradigm that GPUs have not traditionally accelerated. This tutorial explains a new opportunity to greatly accelerate vector graphics, path rendering, and immersive web standards using the GPU. By attending, you will learn how to write OpenGL applications that accelerate the full range of path rendering functionality. Not only will you learn how to render sophisticated 2D graphics with OpenGL, you will learn to mix such resolution-independent 2D rendering with 3D rendering and do so at dynamic, real-time rates.
We have developed technology that allows a wide range of graphical interfaces to be streamed efficiently over wide area networks. This is an enabling technology that enables remote graphics akin to the way MPEG compression enables video streaming.
The Power of WebGL - Hackeando sua GPU com JavaScriptRaphael Amorim
Vamos aprender a hackear placas de vídeo com Javascript usando WebGL (Web Graphics Library) e a usar a mesma tecnologia para criar realidade virtual usando MozVR. WebGL é uma API em JavaScript, disponível a partir do novo elemento canvas da HTML5, que oferece suporte para renderização de gráficos 2D e gráficos 3D.
Understanding Hardware Acceleration on Mobile BrowsersAriya Hidayat
GPU acceleration on mobile browsers, if it is leveraged correctly, can lead to a smooth and fluid applications, thus improving the user experience. There has been a lot of mentions and best practices of hardware acceleration these days, although so far it has been pretty general and hasn’t provided much technical direction apart from simple magical advice such as “use translate3d”. This talk sheds some more light on browser interactions with the GPU and explain what happens behind the scenes, covering the topic of acceleration of primitive drawing, the use of tiled backing store, and composited layer. Knowing the actual machinery behind hardware acceleration, you will be in the position to plan your strategy to improve the performance of your web application.
Computer Graphics and its applications, Elements of a Graphics, Graphics Systems: Video Display Devices, Raster Scan Systems, Random Scan Systems, Input devices.
: Introduction of Rendering, Raytracing, Antialiasing, Fractals
Talk given at The Rich Web Experience 2008. Check out blog for more demos, and sample code.
I hate images. Not pictures or icons, mind you, but user interface graphics. I think that small gradient PNGs that web developers set to repeat are the spacer gifs of today. Images are hard to change, and slower to download.
- Vector- and Raster-based Graphics
-- Idea behind Vector- and Raster-based Graphics
-- Crispness
-- Overview of Raster-based Drawing APIs
- Platform independent Graphics and GUIs in the Web Browser
-- Bare HTML Pages
-- Plugins and Problems
-- From rich Content to HTML 5
- Drawing with HTML 5 Canvas
-- Continuous, Event driven and free Drawing
-- Basic Drawing "How does Drawing work with JavaScript?"
-- Interaction with Controls
- Vector- and Raster-based Graphics
-- Idea behind Vector- and Raster-based Graphics
-- Crispness
-- Overview of Raster-based Drawing APIs
- Platform independent Graphics and GUIs in the Web Browser
-- Bare HTML Pages
-- Plugins and Problems
-- From rich Content to HTML 5
- Drawing with HTML 5 Canvas
-- Continuous, Event driven and free Drawing
-- Basic Drawing "How does Drawing work with JavaScript?"
-- Interaction with Controls
Cna yuo raed tihs?
Ever wondered how?
Imagine, every pixel, every color, every detail harmoniously coming together to present this very message. If you're intrigued by how this digital canvas works, then prepare to be intrigued even further.
Are you the aspiring game developer, craving a solid foundation in GUI intricacies?
Or perhaps, a budding designer yearning to unravel the screen's layout with a more discerning eye?
Or maybe you're the tech enthusiast, eager to decode the magic behind every pixel.
Well, you're in for a treat!
GDSC KIIT proudly presents our latest Huddle Tech Talk that promises to quench your curiosity. Allow us to introduce:
"Graphics Pipelining: How Your Computer Creates Amazing Pictures, a Peek Behind the Screen"
Event Details:
Date: August 27, 2023
Time: 10:30 am - 12:00 pm
Venue: C-WL-101, Campus 14, KIIT University
If you're interested in decoding how modern displays work and unraveling the way all the layers come together to present this screen to you, then this event is for you. Don't miss out on understanding the very medium through which you're engaging with this message.
So, mark your calendar, set your sights on the screen, and prepare to be captivated in a way that only GDSC KIIT can deliver.
Introduce Brainf*ck, another Turing complete programming language. Then, try to implement the following from scratch: Interpreter, Compiler [x86_64 and ARM], and JIT Compiler.
The promise of the IoT won’t be fulfilled until integrated
software platforms are available that allow software
developers to develop these devices efficiently and in
the most cost-effective manner possible.
This presentation introduces F9 microkernel, new open source
implementation built from scratch, which deploys
modern kernel techniques dedicated to deeply
embedded devices.
Build a full-functioned virtual machine from scratch, when Brainfuck is used. Basic concepts about interpreter, optimizations techniques, language specialization, and platform specific tweaks.
This presentation covers the general concepts about real-time systems, how Linux kernel works for preemption, the latency in Linux, rt-preempt, and Xenomai, the real-time extension as the dual kernel approach.
* Know the reasons why various operating systems exist and how they are functioned for dedicated purposes
* Understand the basic concepts while building system software from scratch
• How can we benefit from cheap ARM boards and the related open source tools?
- Raspberry Pi & STM32F4-Discovery
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
6. Exporting Graphics
• can be exported from any of the levels of the graphics stack
– Application, Graphic Toolkit, Graphic Rendering, Bitmapped Device
7. Exporting Graphics - Application
• Normal way Linux/Android/Iphone runs apps.
– The application itself is exported and run locally.
8. Exporting Graphics - Toolkit
• Technically very complex. Android has 15 different toolkit API variants.
• Every application can extend the toolkit with custom widgets
(subclasses of android.view.View).
Exporting Graphics - Rendering
• Exports graphics at the rendering level.
• In Android there are a number of rendering interfaces that can be used:
– skia graphics
– OpenGL ES 1.1 or OpenGL ES 2.0
– Android.view.View
9. 2D Graphics
• The display presents us the contents of something called the
framebuffer.
• The framebuffer is an area in (V)RAM
• For each pixel on screen there‘s a corresponding memory cell in the
framebuffer
• Pixels are addressed with 2D coordinates.
10. 2D Graphics
• To change what is displayed → change the colors of pixels in (V)RAM.
– Pixel colors are encoded as RGB or RGBA
• To draw shapes → need to figure out which framebuffer pixels we have
to set.
– Images (bitmaps) are not special either
– Pixels of the bitmap get stored in a memory area, just like we store
framebuffer pixels.
• To draw a bitmap to the framebuffer → copy the pixels. (Blitting)
• We can perform the same operations on bitmaps as we perform on the
framebuffer, e.g. draw shapes or other bitmaps.
11. Blitting: copy (parts of) one bitmap to another
Alpha Compositing: blitting + alpha blending
• Alpha value of a pixel
governs transparency
• Instead of overwritting a
destination pixel we mix its
color with the source pixel.
Source: Android Game Development 101, BadlogicGames
Source: Android Game Development 101, BadlogicGames
13. Rendering Level: skia
• The rendering level is the graphics layer that
actually “colors” the pixels in the bitmap.
• skia is a compact open source graphics library
written in C++.
• Currently used in Google Chrome, Chrome OS,
and Android.
Skia is Greek for “shadow”
Skia is Greek for “shadow”
14. Rendering Level: skia
• skia is a complete 2D graphic library for drawing
Text, Geometries, and Images. Features include:
– 3x3 matrices w/ perspective
– Antialiasing, transparency, filters
– Shaders, xfermodes, maskfilters, patheffects
15. Rendering Level: skia
• Each skia call has two components:
– the primitive being drawing
(SkRect, SkPath, etc.)
– color/style attributes (SkPaint)
• Usage example:
canvas.save();
canvas.rotate(45);
canvas.drawRect(rect, paint);
canvas.drawText(“abc”, 3, x, y, paint);
canvas.restore();
17. CPU vs. GPU Limited at Rendering
Tasks over Time
100% 100%
90% 90%
80% 80%
70% 70%
60% 60%
50% GPU 50% GPU
CPU CPU
40% 40%
30% 30%
20% 20%
10% 10%
0% 0%
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Pipelined 3D Interactive Rendering Path Rendering
Goal of NV_path_rendering is to make path rendering a GPU-limited task
Render all interactive pixels, whether 3D or 2D or web content with the GPU
Source: GPU-Accelerated 2D and Web Rendering, Mark Kilgard, NVIDIA
Source: GPU-Accelerated 2D and Web Rendering, Mark Kilgard, NVIDIA
20. skia + gpu
• Problems
– calls glDrawSomethings too many times
– changes gl states too many times
– switches FBO too many times
– vector graphics APIs and shadows are really slow
• Increases dramatically CPU overhead
Draw Call At Once Ideal GL skia
Bitmap Sprite Good Good
Convex Path Good Poor
Concave Path Good Poor
Bitmap Sprite + Path Good Poor
Path + Different Shadow Good Poor
Text + Different Draw Call Good Poor
23. Myths and Facts
• Myth: Android 1.x is slow because of no
hardware accelerations
– NOT TRUE! window compositing utilizes
hardware accelerations.
– But it is quite constrained
• There are 4 Window: Status Bar, Wallpaper,
Launcher, and Menu in the right screenshot.
– Hardware composites animations of Activity
transition, the fading in/out of Menu.
• However, the content of Window (Canvas) is
being accelerated by hardware since Android 3.x
24. View & TextureView
• View
– represents the basic building block for UI
– occupies a rectangular area on the screen and is
responsible for drawing and event handling.
• SurfaceView
– provides a dedicated drawing surface embedded inside of
a view hierarchy.
• TextureView
– Since Android 4.0
– Only activated when hardware acceleration is enabled !
– has the same property of SurfaceView, but you can create
GL surface and perform GL rendering above them.
25. from EGL to SurfaceFlinger
hardware
hardware android software
android software
OpenGL|ES
OpenGL|ES OpenGL|ES renderer
OpenGL|ES renderer
29. skia, again
Drawing basic primitives include rectangles, rounded rectangles, ovals, circles,
arcs, paths, lines, text, bitmaps and sprites. Paths allow for the creation of more
advanced shapes.
Canvas encapsulates all of the state about
drawing into a device (bitmap).
While Canvas holds the state of the drawing device, the state (style) of the
object being drawn is held by Paint, which is provided as a parameter to
each of the draw() methods. Paint holds attributes such as color, typeface,
textSize, strokeWidth, shader (e.g. gradients, patterns), etc.
31. Skia backends
• Render in software
– create a native window and then
– wrap a pointer to its buffer as an SkBitmap
– Initialize an SkCanvas with the bitmap
• Render in hardware acceleration
– create a GLES2 window or framebuffer and
– create the appropriate GrContext, SkGpuDevice, and
SkGpuCanvas
33. Hardware-accelerated 2D Rendering
• Since Android 3.x, more complex than before!
• Major idea: transform the implementation of 2D Graphics
APIs into OpenGL|ES requests
• Texture, Shader, GLContext, pipeline, …
• Major parts for hardware-accelerated 2D Rendering
– Primitive Drawing: Shape, Text,Image
– Layer/Surface Compositing
34. Control hardware accelerations
• Application level
<application android:hardwareAccelerated="true">
– Default value
• False in Android 3.x, True in Android 4.x
• Activity
• Window
WindowManager.LayoutParams.FLAG_HARDWARE_ACCELERATED
• View
– setLayerType(View.LAYER_TYPE_SOFTWARE,null)
View.setLayerType(int type,Paint p)
Layers = Off-screen Buffers or Caches
Layers = Off-screen Buffers or Caches
35. View Layers since Android 3.x
Source:Accelerated Android Rendering, Google I/O 2011
Source:Accelerated Android Rendering, Google I/O 2011
36. How Views are Drawn [Android 3.x]
HardwareCanvas → SkPaint → GLRenderer
no SkGpuCanvas/SkGpuDevice?!
Why can't skia use its OpenGL
backend directly?
37. To answer the previous question, we have to
learn Display List first
• A display list (or display file) is a series of graphics
commands that define an output image. The image is
created (rendered) by executing the commands.
• A display list can represent both two- and three-dimensional
scenes.
• Systems that make use of a display list to store the scene
are called retained mode systems as opposed to immediate
mode systems.
http://en.wikipedia.org/wiki/Display_list
http://developer.android.com/guide/topics/graphics/hardware-accel.html
38. Display List [Android 3.x]
• A display list records a series of graphics related operation and can replay
them later. Display lists are usually built by recording operations on a
android.graphics.Canvas.
• Replaying the operations from a display list avoids executing views drawing
code on every frame, and is thus much more efficient.
39. Display List [Android 4.1]
Source:For Butter or Worse, Google I/O 2012
Source:For Butter or Worse, Google I/O 2012
42. WebKit Rendering – RenderObject
• Each node in the DOM tree that produces visual
output has a corresponding RenderObject.
• RenderObjects are stored in a parallel tree
structure, called the Render Tree.
• RenderObject knows how to present (paint) the
contents of the Node on a display surface.
• It does so by issuing the necessary draw calls to
the GraphicsContext associated with the page
renderer.
– GraphicsContext is ultimately responsible for writing the
pixels on the bitmap that gets displayed to the screen.
43. WebKit Rendering – RenderTree
RenderObjects are stored in a parallel tree structure, called
Render Tree.
44. WebKit Rendering – RenderLayers
• Each RenderObject is associated with a
RenderLayer either directly or indirectly via an
ancestor RenderObject.
• RenderObjects that share the same coordinate
space (e.g. are affected by the same CSS
transform) typically belong to the same
RenderLayer.
• RenderLayers exist so that the elements of the
page are composited in the correct order to
properly display overlapping content,
semitransparent elements, etc.
45. RenderLayers
• In general a RenderObject warrants the creation of
a RenderLayer if
– is the root object for the page
– has explicit CSS position properties (relative,
absolute or a ransform)
– is transparent
– has overflow, an alpha mask or reflection
– Corresponds to <canvas> element that has a 3D
(WebGL) context Corresponds to a <video>
element
46. WebKit Rendering
• RenderLayer hierarchy is traversed recursively starting from
the root and the bulk of the work is done in
RenderLayer::paintLayer().
• WebView is the web page encapsulated in a UI component.
• Web page update → the redraw of WebView
– Adjust layers structure according to the latest content
and then render/record the updated
– Render the updated content
• Various approaches of Rendering Architecture
– Use texture or vector (backing store) as the internal
representation
– multithreaded, multiple processes.
47. Accelerated Compositing
• Idea: to optimize for cases where an element would
be painted to the screen multiple times without its
content changing.
– For example, a menu sliding into the screen, or a
static toolbar on top of a video.
• It does so by creating a scene graph, a tree of
objects (graphics layers), which have properties
attached to them - transformation matrix, opacity,
position, effects etc., and also a notification when
the layer's content needs to be re-rendered.
48.
49. • When accelerated compositing is enabled, some (but not
all) of the RenderLayer's get their own backing surface
(compositing layer) into which they paint instead of drawing
directly into the common bitmap for the page.
• Compositor is responsible for applying the necessary
transformations (as specified by the layer's CSS transform
properties) to each layer before compositing it.
50. • Since painting of the layers is decoupled from compositing,
invalidating one of these layers only results in repainting the
contents of that layer alone and recompositing.
51. Rendering Flow
• Layers Sync
→ done by WebCore itself
• Layers Compositing
→ done by WebKit port (like Android)
• Android 4.x supports Accelerated Compositing and
Hardware Accelerations
– decided by the property of given Canvas
52. [ skia + gpu; Chrome browser ]
JavaScript
(Canvas) OpenGL ES CPU GPU
Computing
Draw Call Draw Call Context
Computing
Draw Call Draw Call Context
Computing
Draw Call Draw Call Context
eck
ttlen
Computing Flush
Draw Call Draw Call
Bo Context
Computing
Draw Call Draw Call Context
• When WebView is redrawn, UI thread performans compositing
– First, TiledTexture in Root Layer of current Page ViewPort
– Then, TiledTexture in other BackingLayers
• The generation of TiledTexture can utilize both CPU and GPU.
– Android 4.0 still uses CPU.
53. Flow of Generating Tiled Texture
• Using CPU
– Take one global SkBitmap and reset (size equals to one
Tile)
– Draw SkPicture → global SkBitmap
– Memory copy from SkBitmap to Graphics Buffer of Tile
• Using GPU
– All real rendering occurs in TextureGenerator thread
– Draw the pre-gernated textures
Page → vector backing store, layer → texture
54. Techniques to make it better
JavaScript
(Canvas) GL request OpenGL ES CPU GPU
Optimizer
Draw Call
Draw Call
Draw Call
Draw Computing
Draw Call Call Context Flush
Draw Call
• improve object lifetime management
• Use GPU specific Backing store implementation
• prefetch optimization for DOM Tree Traversal
• improve texture sharing mechanisms
• Eliminate the loading of U thread
55. SurfaceFlinger
• Android‘s window compositor
– Each window is also a layer.
– The layers are sorted by Z-order. The Z-order is just the layer
type as specified in PhoneWindowManager.java.
• When adding a layer with a Z-order that is already used
by some other layer in SurfaceFlinger‘s list of layers it is
put on top of the layers with the same Z-order.
• Even many SurfaceFlinger rendering operations are
inherently flat (2D), it uses OpenGL ES 1.1 for rendering
– May be memory limited on devices with small displays
– Copybit acceleration may be desirable for UI on some devices
→ deprecated since Android 2.3
– It is known to improve UX with custom copybit module
56. (low-level) overhead
• Composition overhead
– Android extensions such as “EGLImage from Android native
buffer”, can employ copybit (2D) backend to further offload GPU
→ use non-linear textures for 3D applications to improve memory
access locality
• Native ↔ Java communication overhead
– Native code for key operations
– Can be observed by TraceView tool
• Cache management overhead
– range-based L1 and L2 cache functions (clean, invalidate, flush)
– Normally uncached graphics memory is sufficient for gaming use
cases
– Cached buffers result in higher performance for CPU rendering
in compositing systems
57. Performance Tips
• Display List is crucial to user experience, but it has
to be scheduled properly, otherwise CPU loading
gets high unexpectedly.
• Always verify and probe graphics system using
Strict Mode
• When hardware acceleration is enable, prevent the
following operations from being modified/created
frequently:
– Bitmap, Shape, Paint, Path
58. Reference
• Skia & FreeType: Android 2D Graphics Essentials,
Kyungmin Lee, LG Electronics
• How about some Android graphics true facts? Dianne
Hackborn (2011)
• Android 4.0 Graphics and Animations, Romain Guy &
Chet Haase (2011)
• Learning about Android Graphics Subsystem, Bhanu
Chetlapalli (2012)
• Service 與 Android 系統設計,宋寶華