SlideShare a Scribd company logo
1 of 105
Download to read offline
Embedded Software Lab. @ SKKU
105
1
Tizen v2.3 Graphics & UI
Frameworks
Embedded Software Lab. @ SKKU
105
2
• EFL(Enlightenment Foundation Library)
– EFL Features
– Ecore
– Evas
– Edje
– Elementary
– Cserve2
• Window Systems
– X Window System
– Wayland
• DALi 3D Engine
Contents
Embedded Software Lab. @ SKKU
105
3
• UI Frameworks
– Modules to implement interactive GUI
– EFL(Elementary/Ecore), OSP, GTK, Qt
– Input Service Framework(ISF), Voice Framework
• Graphics Libraries
– Modules to render graphic objects
– Evas, Cairo, OpenGL ES, EGL
• Window Systems
– Modules to manage windows and frame buffers
– X Window System, Wayland(Tizen v3.0)
Tizen v2.3 Graphics & UI Frameworks
Embedded Software Lab. @ SKKU
105
4Tizen v2.3 Graphics & UI Frameworks
• Frameworks for UI of core
applications & web applications
Deprecated
(Bada App FW)
Embedded Software Lab. @ SKKU
105
5
EFL
(Enlightenment Foundation Library)
Embedded Software Lab. @ SKKU
105
6
• Started from a desktop window manager
– Default window manager of Bodhi Linux
Enlightenment
Embedded Software Lab. @ SKKU
105
7
• A set of libraries supporting Enlightenment
– Enlightenment 0.17(E17)
• Components are divided by functions
– Event loop: Ecore
– Rendering engine: Evas
– UI: Edje, Elementary, …
– UI builder tools: Enventor, Eflete, ELM Theme Editor
• EFL in Present
– Core components of Tizen Graphics/UI framework
– 26% of code is E17, the other is EFL
Enlightenment Foundation Library
Embedded Software Lab. @ SKKU
105
8
EFL in Tizen v2.3
Legend
Applications EFL & fontsWindow System
H/W
Linux Kernel Frame Buffer Input Devices
H/W
fontconfigfreetype2 OpenGL ES/EGL
Eet
Evas
Ecore
Eina
Edje
Elementary
Core Applications
X Window System
Input
Service
Framework
&
Voice
Framework
kernel / Frame Buffer/ Input Devices
Graphics
ISF& Voice FW
Cairo
Embedded Software Lab. @ SKKU
105
9
• Immediate Mode Graphics System
– Application directly re-paints its portion that is invalidated
– Application re-issues all drawing commands to describe the
entire scene each time a new frame is required
– GTK+ 3, GDI, GDI+
Immediate vs. Retained Mode (1/2)
Application
Window System
GTK+/GDK-X11
1. Invalidate
2. Expose
Graphics Card
Frame Buffer for Screen
3. Drawing
Embedded Software Lab. @ SKKU
105
10
• Retained Mode Graphics System
– The graphics system is responsible for rendering the
drawing objects given by application.
– Application passes drawing objects to graphics system.
– Evas, GTK+ 4(Clutter), WPF
Immediate vs. Retained Mode (2/2)
Application
Evas objects
Evas Rendering Engine
(Graphics system)
Expose
Graphics Card
Framebuffer for screen
X Window System
Embedded Software Lab. @ SKKU
105
11
• High performance
– Retained mode graphics system
– 60fps+ on general smartphones
• Fit for Low-end Hardware
– Small memory, no swap, no GPUs
– OpenGL is not worth for all cases
• Texture keeps speed, but loses memory
• Support for HW rendering
– OpenGL ES
– SIMD vector(MMX, Neon)
Why is EFL used on Tizen? (1/3)
Desktop
Environment
Memory
Used
(MiB)
Enlightenment
0.18.8
83.8
LXDE 0.5.5 87.0
XFCE 4.10.2 110.0
LXQt 0.7.0 113.0
MATE 1.8.1 123.0
Cinnamon 2.2.13 176.3
GNOME3 3.12.2 245.3
KDE 4.13.1 302.6
Unity 7.2.0.14 312.5
Embedded Software Lab. @ SKKU
105
12
• Various back-end engine
– X11 (~Tizen 2.2.1), Wayland (Tizen 3.0~)
– Direct Frame Buffer, DRM, Memory buffers, PS3 Native,
Windows (GDI & DirectDraw), …
• UI Scalability
– Various resolutions (WVGA ~ XQXGA) with the same
layout and resources
Why is EFL used on Tizen? (2/3)
0.6 0.8 1.0Scale factor:
Embedded Software Lab. @ SKKU
105
13
• GUI, Logic separation
• Themes
– A preset package containing
graphical appearance details
• Bindings
– Basically C-based
– EFL is also binded to Javascript,
C++, Python, Ruby, Perl …
– Eo: Generic object model
Why is EFL used on Tizen? (3/3)
Embedded Software Lab. @ SKKU
105
14
• Entry barrier
– Unfamiliar development style
– Lack of documentations, developers
• No object-oriented implementation
– C-based
– Eo project is ongoing
• Incomplete C++ support
– Eolian based E++ project ongoing
– Eolian is EO object parser and C code generator
• No thread safe
EFL Shortcomings
Embedded Software Lab. @ SKKU
105
15
Tizen v2.3 Rendering Path
Textures
OpenGL ES
win
X window server
Frame Buffer
Video
win
EFL win
3D App
OpenGL ES/EGL
Video App.
EFL App.
EFL
EVAS
Composite
Window
Manager
OpenGL ES/EGLOpenGL ES/EGL
MM FWEFL
Rendering pipeline path; rendering data flow
Legend
3D App. Multimedia App.EFL App. Configurable Enable/disable
Texture from Pixmap
OpenGL ES/EGL
Process Texture Mem.
Embedded Software Lab. @ SKKU
105
16Tizen v2.3 Rendering Path on SW Backend
Elementary
Ecore
xlib
Frame
buffer
LCD
controller
60Hz
Update
window areas
X Server
X
shared
Mem.
Edje
Evas
Evas software backendEcore X
Embedded Software Lab. @ SKKU
105
17Tizen v2.3 Rendering Path on GLES Backend
EGL for X
xlib
Frame
buffer
/dev/fb1
LCD
controller
60Hz
Update
window areaX Server
frontback
Double buffer
OpenGL ES H/W
OpenGL ES
Ecore
Elementary
Edje
Evas
Evas OpenGL ES backend Ecore X
Embedded Software Lab. @ SKKU
105
18
• Ecore (E + Core)
– Event loop library with convenience modules
• Evas (E + canVAS)
– Canvas & rendering library
• Edje
– Complex graphical design & layout library
• Elementary
– Widget set
EFL Components: Core
Embedded Software Lab. @ SKKU
105
19
• Eo
– Generic object model for
binding other languages
• Eina
– Library for data types
• Eet
– Data codec and storage
• Efreet
– Freedesktop.org
standards support
EFL Components: Others
• Edbus
– Wrapper of Dbus
• Emotion
– Video/Audio Playback
library
• Eio
– Asynchronous I/O
• Eeze
– Udev hardware detection
• Ethumb
– Thumbnailer & cacher
Embedded Software Lab. @ SKKU
105
20
The core loop of EFL applications
EFL: Ecore
Embedded Software Lab. @ SKKU
105
21
• The core loop of EFL application, event &
marshaling library
– Provides glue to other subsystems
– Supports various display system
• X11, FB, Wayland, Win32, WinCE, etc.
• Features:
– Networking & IPC
– Threading & inter-thread communication
– Evas input feeding and output driving
– More...
Ecore
Embedded Software Lab. @ SKKU
105
22
• elm_main()
– Main function of EFL application
– After initialization, enter into event loop.
Simple EFL Application
Start Ecore main loop!
Elementary UI configuration
(Elementary Window)
On exiting window, it also exits Ecore event loop
Finalization process like freeing resource
Embedded Software Lab. @ SKKU
105
23
• ecore_main_loop_begin()
– The wrapper of Ecore main loop
– Ecore main loop and Glib main loop can be chosen.
Ecore Main Loop (1/2)
Ecore main loop
Glib main loop
Embedded Software Lab. @ SKKU
105
24
• _ecore_main_loop_iterate_internal()
– The implementation of Ecore main loop
– It starts all the timer handler, event handler, rendering
in an application.
Ecore Main Loop (2/2)
Embedded Software Lab. @ SKKU
105
25
• Key down event
Ecore: Usage of Ecore Events
Ecore_Event_Handler *handler;
handler = ecore_event_handler_add(ECORE_EVENT_KEY_DOWN, func, data);
Eina_Bool func(void *data, int type, void *event_info)
{
if (…) return ECORE_CALLBACK_DONE;
return ECORE_CALLBACK_PASS_ON;
}
Ecore_Timer *timer;
double time;
timer = ecore_timer_add(time, func, data);
Eina_Bool func(void *data)
{
if (…) return ECORE_CALLBACK_RENEW;
return ECORE_CALLBACK_CANCEL;
}
• Timer event
Embedded Software Lab. @ SKKU
105
26
Canvas & Rendering Library
EFL: Evas
Embedded Software Lab. @ SKKU
105
27
• Canvas & Rendering Library
• Features
– Retained mode rendering
• Controls rendering behavior in the unit of primitive objects and
smart objects.
– Scene graph
• Tracks the status of objects → minimizes rendering behavior.
– Multiple output paths
• Pure software/OpenGL ES
– Rendering optimization
• 3D H/W Acceleration, SIMD vector operations
– Provides GL Glue Layer
Evas
Embedded Software Lab. @ SKKU
105
28
• Evas uses retained mode rendering
– Application sends Evas objects to Evas.
– Evas does rendering based on Evas objects.
Evas: Retained Mode Rendering
Application
Evas objects
Evas Rendering Engine
(Graphics system)
Expose
Graphics Card
Framebuffer for screen
X Window System
Embedded Software Lab. @ SKKU
105
29
• All Objects Displayed on Canvas
– Controlled by Evas_Object handles.
• Loading fonts and images, rendering glyphs and images,
scaling, blending, 3D transformation etc.
– Evas manages all Evas objects equally.
• Type of Evas Object
– Primitive objects: minimum unit of Evas rendering
• Rectangle, Line, Polygon, Text, Textblock, Textgrid, Image
– Smart objects
• Box, Grid, Table, Customized smart objects(ELM Widgets)
Evas Objects
Embedded Software Lab. @ SKKU
105
30
Evas Primitive Objects
• Rendering units of Evas
Embedded Software Lab. @ SKKU
105
31
• Evas objects composed of primitive objects
– Acts as a group.
– Evas object generic functions also can be used.
Evas Smart Objects
Embedded Software Lab. @ SKKU
105
32
Image Native Surface
Hello World!
Evas_Object_Rect
Evas_Object_Text
Evas_Object_Image
…
GL
Rendering
Engine
SW
Rendering
Engine
X11
Rendering
Engine
X11
Output
Buffer
Output
…
SDL
Output
…
Evas Runtime
OpenGL App
(surface_a
ie. Pixmap or
texture)
Evas_Object_Image
(img_obj_a)
evas_object_image_native_surface_set(img_obj_a, surface_a)
Output
Hello World!
Evas Engine Module
Embedded Software Lab. @ SKKU
105
33
• Scene graph
– A structure that arranges the logical representation of
a graphical scene
– General data structure in GUI applications
• Scene graph in Evas
– Manages Evas objects in graph data structure.
– Tracks all the objects’ status and gets changed area in
the next frame → Minimizes rendering area.
Evas: Scene Graph
Embedded Software Lab. @ SKKU
105
34
• Start Here
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
35
• Next frame is…
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
36
• Calculate actual update region deltas
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
37
• Only draw updated regions
Evas: Minimizing Rendering Region
Embedded Software Lab. @ SKKU
105
38
/* Build by $ gcc evas_rectangle.c -o evas_rectangle `pkg-config --cflags --libs ecore ecore-evas` */
#include <Ecore.h>
#include <Ecore_Evas.h>
int main(int argc, char **argv) {
Ecore_Evas *ee;
Evas *evas;
Evas_Object *bg;
if(ecore_evas_init() <= 0) return 1;
ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);
ecore_evas_show(ee);
evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object.
bg = evas_object_rectangle_add(evas); // Add a rectangle to the given Evas object.
evas_object_color_set(bg, 255, 255, 255, 255); // Set the color of the given Evas object to the given one.
evas_object_move(bg, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport.
evas_object_resize(bg, 200, 200); // Change the size of the given Evas object.
evas_object_show(bg); // Make the given Evas object visible.
ecore_main_loop_begin();
ecore_evas_free(ee);
ecore_evas_shutdown();
return 0;
}
Evas: Rectangle Example
Embedded Software Lab. @ SKKU
105
39
/* Build by $ gcc evas_image.c -o evas_image `pkg-config --cflags --libs ecore ecore-evas` */
#include <Ecore.h>
#include <Ecore_Evas.h>
int main(int argc, char **argv) {
Ecore_Evas *ee;
Evas *evas;
Evas_Object *bg;
if(ecore_evas_init() <= 0) return 1;
ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);
ecore_evas_show(ee);
evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object.
bg = evas_object_image_filled_add(evas); // Add an image to the given Evas object.
evas_object_image_file_set(img, “test.png”, NULL); // Set the image file path of the given Evas object.
evas_object_move(img, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport.
evas_object_resize(img, 200, 200); // Change the size of the given Evas object.
evas_object_show(img); // Make the given Evas object visible.
ecore_main_loop_begin();
ecore_evas_free(ee);
ecore_evas_shutdown();
return 0;
}
Evas: Image Example
Embedded Software Lab. @ SKKU
105
40
Complex graphical design & layout library
EFL: Edje
Embedded Software Lab. @ SKKU
105
41
• Complex graphical design & layout library
• EDC script
– GUI layout description separated from application code
– Can be changed at or during runtime
– Compiled as binary code(edj)
Edje
• Description about:
– Images, colors, positioning,
animations, behaviors, …
Embedded Software Lab. @ SKKU
105
42
• Separation of layout
and logic
– Graphical part: GUI Layout
binary (edj)
– Functionality: Executable
logic binary (C)
• Utility
– edje_cc : compile edc to edj
– edje_decc : de-compile edj
to edc
Edje: Edc & Edj
Embedded Software Lab. @ SKKU
105
43
Edje: Relation with Evas, Ecore
Embedded Software Lab. @ SKKU
105
44
Edje: EDC Script
Embedded Software Lab. @ SKKU
105
45
Edje: EDC Script
Embedded Software Lab. @ SKKU
105
46
Edje: Simple Example
part {
name: “background”;
type: RECT;
mouse_events: 0;
description {
state: “default” 0.0;
color: 255 255 255 255;
}
}
part {
name: “blue_rect”;
type: RECT;
mouse_events: 1;
description {
state: “default” 0.0;
color: 0 0 255 255;
rel1 {
relative: 0.1 0.1;
}
rel2 {
relative: 0.3 0.3;
}
}
}
Embedded Software Lab. @ SKKU
105
47
Edje: Simple Example
part {
name: “red_rect”;
type: RECT;
mouse_events: 0;
description {
state: “default” 0.0;
color: 255 0 0 255;
rel1 {
to: “blue_rect”;
relative: 0 2;
}
rel2 {
to: “blue_rect”;
relative: 1 3;
}
}
description {
state: “down” 0.0;
inherit: “default” 0.0;
rel1.relative: 3 2;
rel2.relative: 4 3;
color: 255 0 0 0;
}
}
Embedded Software Lab. @ SKKU
105
48
Edje: Simple Example
program {
name: “blue_rect.clicked”;
signal: “mouse,clicked,1”;
source: “blue_rect”;
action: STATE_SET “down” 0.0;
transition: ACCELERATE 0.5;
target: “red_rect”;
after: “red_rect.restore”;
}
program {
name: “red_rect.restore”;
action: STATE_SET “default” 0.0;
target: “red_rect”;
}
Embedded Software Lab. @ SKKU
105
49
A set of widgets
EFL: Elementary
Embedded Software Lab. @ SKKU
105
50
• A Set of Widgets
– 약 80여 개
– Containers: Box, Conformant, Grid, Layout, Panes, Scroller, Table, …
– Non-Containers: Bg, Button, Check, Entry, Label, List, Icon, Win, ...
• Features:
– Fast, Finger friendly, Scalability, Themes
– Widget hierarchy, Dynamic language change
Elementary
< Scaled and resized relative to the screen> < Same view but different themes>
Embedded Software Lab. @ SKKU
105
51
• Various Widgets
Elementary Widgets
clock
colorselector
entry
frame
flipselector
check
diskselector
actionslider
bg
index
map
menu
photo, photocam
panel
toolbar
radio
bubble
slider
ctxpopup
notify
separator
segment control
multibuttonentry
panes
glview
fileselector
Label
thumb
Embedded Software Lab. @ SKKU
105
52
• Naviframe
– Switching multiple
pages with stack
• Toolbar
– List of Items
Elementary Widgets
Embedded Software Lab. @ SKKU
105
53
• Entry
– Text input field
– Copy & paste,
various modes
(password, multi-
line, ...)
• Genlist
– Generic List
– Flexible but
complex to use
Elementary Widgets
Embedded Software Lab. @ SKKU
105
54
• Widgets are built in a hierarchical fashion.
• Widget shares it’s functions(APIs) with
child widget.
Elementary: Widget Hierarchy
Embedded Software Lab. @ SKKU
105
55
• UI Scalability
– Relative Positioning
Elementary: Scalability
– Scale Factor
Embedded Software Lab. @ SKKU
105
56
• Theme can be changed in same layout
Elementary: Theme
Embedded Software Lab. @ SKKU
105
57
• Elementary provides utility APIs for multi
language support
Elementary: Dynamic Language Change
Embedded Software Lab. @ SKKU
105
58Elementary: Dynamic Language Change
Embedded Software Lab. @ SKKU
105
59
• Enventor: Dynamic EDC Editor
• Eflete: EFL Edje Theme Editor
• ELM Theme Viewer: Elementary Theme
Viewer
GUI Tools for Customizing Layout
Embedded Software Lab. @ SKKU
105
60
Optimization of EFL
Cserve2
Embedded Software Lab. @ SKKU
105
61
• Duplicated Resource Problem in EFL
– Most images and fonts used in a theme are used by
multiple applications at the same time
– EFL contains a variety of caches and
• Images, fonts, ...
• Objectives of Cserve2:
– Reduce memory consumption by sharing
– Optimize load time by pre-loading data
Cserve2: Motivation
Embedded Software Lab. @ SKKU
105
62
• Sharing image and font data between EFL
application processes
– Started by ProFusion at 2012
• Cache server
– Loads images and fonts
– Shares resources across applications
• EFL applications
– Connect to the server via UNIX socket
– Send requests and wait for answers
– Don't load anything on their own
– Keep local cache pointing to shared data
Cserve2
Embedded Software Lab. @ SKKU
105
63
Cserve2
Embedded Software Lab. @ SKKU
105
64
• Problems on Cserve2
– IPC overheads:
• messages over socket link
• context switches, read, write, ...
– Clients do not know the server's cached data before
sending a request
• Solution
– Expose indexes of shared objects to all clients
• via /dev/shm
– Clients scan the tables and don't wait for the server's
response when the data is already loaded
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
65
• 3 types of objects:
– Data (image data)
• Stored in separate shm files: “img”
– Fixed-size arrays of objects
• Indexes
• Stored in “array” files
– Variable-size pools of objects
• So-called “mempools”
• Indexed by offset + length in a separate array
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
66
• Index tables principles
– Arrays of simple structures of a single type
– Contain:
• ID
• Refcount
• Other fixed-size data (int, bool, ...)
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
67
• Strings
– Two shm files are used: index and mempool
– Mempool contains the strings actual data
– Index table contains:
• String ID
• Refcount
• Offset (in the mempool)
• Length
• So, we can read a string just knowing its
string ID
Cserve2: Shared Indexing
Embedded Software Lab. @ SKKU
105
68
• Cserve2 combines image cache and scale
cache
• Reuse scale cache logic
– Minimizes the number of cached scale images.
– Scaling process is done on server side.
Cserve2: Image Scaling
Embedded Software Lab. @ SKKU
105
69
• Cserve2 image loader crash
– No crash on client side (just no data)
– New slave spawned
• Cserve2 server crash
– enlightenment_start restarts cserve2
– Clients reconnect to new server
• Discard some data
• Keep old image references (no need to load)
• Cserve2 shutdown or restart
– cleanup /dev/shm/
Cserve2: Robustness
Embedded Software Lab. @ SKKU
105
70
• Real Phone Usage
– Workload & Environment
• 8 threads of “elementary_test -to launcher” on 1280x720 screen
– Font size
• ~24x16 (small), ~32x24 (normal), ~40x30 (large)
• In case of normal size, 1 font, 50 characters:
1x32x24x50=37.5KB
– Icon size
• 50x50 (small), 100x100 (large) in 32-bit color
• For one icon, 4Bx50x50=10KB (small), 4Bx100x100=40KB (large)
– Border size
• width 10px, length 720px, 32-bit color, 2x
• 4Bx10x720x2=56KB
Cserve2: Benefits
Embedded Software Lab. @ SKKU
105
71
• Overhead
– When workload runs with Cserved2 turned on: 72176KB
• Size of /dev/shm: 4376KB
– When Cserve2 quits: 66716KB
– Total overhead: 72176KB–66716KB–4376KB=1084KB
• Conclusion:
– Cserve2 saves memory as soon as 1MB of data can be
shared(fonts & images).
• Refer to “Cserve2: Shared cache subsystem
of Evas”.
Cserve2: Overhead
Embedded Software Lab. @ SKKU
105
72
Tizen v2.3 Window Systems:
X Window Systems
Embedded Software Lab. @ SKKU
105
73
• History
– X window system is made by MIT in 1984.
– X11: the current protocol version, established in 1987.
– X.org server: the current reference implementation
developed by the X.Org Foundation
• Designed to be used over network connections
• Features
– Event Driven
– Network Transparent
– Device Independent
– Policy Independent
– Extensible
X Window System
Embedded Software Lab. @ SKKU
105
74
• Client-server Model
– X client is linked with X server
through the X protocol translated
by Xlib.
– X client
• Sends graphical request
• Gets user input.
• Ecore and Evas in Tizen
– X server
• Accepts request for graphical output
• Sends back user input
X Window System: Architecture (1)
Embedded Software Lab. @ SKKU
105
75
X Window System: Architecture (2)
• Client-server Architecture of X System
Embedded Software Lab. @ SKKU
105
76
X Protocol
• Protocol about communication with X
clients and the X server
• Features:
– Object-based
– Asynchronous
• Types of Messages:
– Request (Client → Server)
– Reply
– Event (Server → Client)
– Error
Embedded Software Lab. @ SKKU
105
77
X Server
• Handles Output & Input Processes
• Manages Screens and Input Devices
• Layered Structure
– DIX Layer, DDX Layer, OS Layer, Extension Layer
• Resource
– Everything in X Server controlled through “resources”
– Created on demand
– Used by clients to request operations
– Screens, windows, graphic contexts, pixmaps, regions,
colormaps, fonts, cursors
Embedded Software Lab. @ SKKU
105
78
X Server
• Layered Structure:
– DIX Layer, DDX Layer, OS Layer, Extension Layer
Embedded Software Lab. @ SKKU
105
79
• DIX(Device Independent X) Layer
– Interface between X Clients and DDX/extensions
• DDX(Device Dependent X) Layer
– Deals with graphic and input hardware
• OS Layer
– Client connection management, work scheduling,
memory allocation routines
• Extension Layer
– Enables to get additional functionalities for X server
X Server
Embedded Software Lab. @ SKKU
105
80
DIX Layer
• Interface among DDX/Extensions and X
Clients
– Delivers requests from clients to DDX/extensions
– Sends input events to clients
• Dispatch loop
– DIX Layer manages requests with dispatch loop
• Screens
– One for each physical screen
– Defines functions used to handle operations on
resources
– Main communication channel DIX-DDX
Embedded Software Lab. @ SKKU
105
81
DDX Layer
• A layer dealing with graphic and input HW
• Graphics Output
• Input Processing
– Mouse movement
– Key presses
– Key mappings
Embedded Software Lab. @ SKKU
105
82
DDX Layer: Graphics Output
• Pixel-based
• Screen basic data structure
– Manage all resources
• Output done on drawables
– Pixel data format defined by DDX
– Pixmap format constant across screens
• XYPixmap -> one bitmap for each plane
• Zpixmap -> series of pixel values
– Pixel values interpreted at display time
– Windows, Pixmaps, …
82
Embedded Software Lab. @ SKKU
105
83
DDX Drawables
• Windows
– Visible rectangle on screen
– Border drawn by DDX
• Create, move, resize
– Contents drawn by client
• Server may provide backing store
• Bit gravity (resizing)
– Clipped by parent and sibbling windows
• Pixmaps
– Rectangular array of pixel values
– Reference counted
Embedded Software Lab. @ SKKU
105
84
OS Layer
• Client Monnection Management
• Work Scheduling
– It schedules work to be done for clients
– Ignore/AttendClient()
• Memory Allocation Routines
– Xalloc()/Xrealloc()/Xfree()
Embedded Software Lab. @ SKKU
105
85
Porting X Server
• Code layers
– DIX: Dev.Independent, don’t touch
– OS: OS specific
– DDX: Graphics Device specific
– Extensions: Add features
• Porting Layer: OS + DDX
• Porting process:
– Define functions required by DIX to handle resources
Embedded Software Lab. @ SKKU
105
86
Tizen v3.0 Window Systems:
Wayland
Embedded Software Lab. @ SKKU
105
87
• Display management architecture & protocol
– Window management and composition
– Weston: main open source implementation
– Maintained by Freedesktop.org
– Adopted by Tizen 3.0
• Replacement for X
– No rendering API in Wayland
• Rendering has already been supported by Evas, Cairo, OpenGL ES
– SHM(Shared memory) protocol: support for SW rendering
– Less IPC: single process architecture
Wayland
Embedded Software Lab. @ SKKU
105
88
• Wayland Protocol
– Weston compositor interacts with Wayland
clients(applications) via Wayland protocol.
Wayland Protocol
Embedded Software Lab. @ SKKU
105
89
• EFL and E17(X compositor) are linked to
X.org(X server) via X protocol
Tizen v2.3 Graphic Stack with X.org
Embedded Software Lab. @ SKKU
105
90
• EFL is linked to Weston/E17 compositor
via Wayland protocols
Tizen v3.0 Graphic Stack with Wayland
Embedded Software Lab. @ SKKU
105
91
• Weston provides surface to be rendered to
Evas
• Weston sends input events to Ecore
Wayland with EFL
Embedded Software Lab. @ SKKU
105
92
• EFL controls display and input devices
through several backends.
– Ecore_Evas: Glue to the backends
Backends of EFL
• X Server Backend
– Software_X11, OpenGL_X11
• Wayland Backend
– Wayland_SHM: SW rendering
– Wayland_EGL: HW rendering
– Ecore_Wayland
• Surface Handling and Input
Embedded Software Lab. @ SKKU
105
93
• Performance of Wayland
– 60 FPS(Frames Per Second)
– 16ms for one frame from client to compositor
• Visualization Tool: E-Graph
– A tool to visualize log information and draw FPS curve
– https://gitorious.org/e-graph/e-graph
Wayland Optimization
Embedded Software Lab. @ SKKU
105
94
• Problem
– Rendering when resource is blocked by waiting VSync
• Triple Buffering
– Add one additional buffer for client and compositor
– Before: ~40fps → After: ~48fps
– Time spend on composition: 13ms
Wayland Optimization
Embedded Software Lab. @ SKKU
105
95
• Problem
– Redraw overlapped surface → heavy load during
composition
• Opaque Region
– Overlapped surface, set by Ecore and sent to Weston
– Compositing time: ~13ms → ~5ms
– FPS: 40fps → 60fps
Wayland Optimization
Embedded Software Lab. @ SKKU
105
96
DALi 3D Engine
Embedded Software Lab. @ SKKU
105
97
• Tizen 3D UI
– DALi 3D Engine & UI Toolkit
• DALi is a 3D Engine
– UI is represented as a 3D Scene Graph
– Animations and Transitions are done using 3D Math
(Vector, Quaternion & Matrix)
– Rendering and Visual Effects are done using Open GL
ES Shaders, Vertices and Textures
– OpenGL ES 2 and 3 support
• 2D world is the Z plane 0 in the 3D world
– When using default camera
DALi 3D Engine
Embedded Software Lab. @ SKKU
105
98
• DALi is part of the Tizen
Native Subsystem
– Graphics & UI native module
– Mobile and TV profiles
• Implemented in C++
• DALi (Dynamic
Animation Library)
– 2D and 3D Application UIs
with Realistic Effects &
Animations
– Home Screen, Lock Screen,
Gallery, Music Player …
DALi in Tizen System Architecture
Embedded Software Lab. @ SKKU
105
99
• Evas
– EFL drawing canvas
– SW/GL rendering
backends
• DALi
– OpenGL based Toolkit
– Differential UX/
heavy contents
• CoreGL
– OpenGL wrapper
– Performance
optimization
DALi and Graphics Framework
Embedded Software Lab. @ SKKU
105
100
• Core Library
– Event handling, Scene
Graph, Rendering,
Resource management
• Adaptor
– Threading model
– Integration with the main
loop
• Platform abstraction
– Resource loading and
decoding with multiple
threads
• Toolkit
– Reusable UI controls,
– Effects and Scripting
support
DALi Architecture
Embedded Software Lab. @ SKKU
105
101
• Scene graph based UI is a tree of
Nodes
– Each Node can have 0-N Children
– Each Node inherits its parent
Transformation
– $ Position, Rotation, Scale
• Allows easy layout and animation
management
– Each Node’s Transformation is relative to
a reference point in the parent’s space
• Anchor point in the Nodes own coordinate
space
• Parent origin in the Parents coordinate space
– Child does not have to be inside its
parent area
DALi: 3D Scene Graph
Embedded Software Lab. @ SKKU
105
102
• DALi uses multithreaded
architecture
– Best performance and scalability
• Event Thread
– The main thread in which application
code and event handling runs
• Update Thread
– Updates the nodes on scene
– Runs animations, constraints and
physics
• Render Thread
– OpenGL drawing, texture and
geometry uploading etc
• Resource Threads
– Loads font, image and model
resources and decodes into bitmaps
etc
Multi-threaded Engine
Embedded Software Lab. @ SKKU
105
103
• Actors & UI Controls
– Stage: root of the world
– Actors: image, text, mesh, …
– UI controls: provide additional layout
and scroll
• Animations
• Shader Effects
– modify the appearance of objects
during rendering
• Image Effects
– Cube transition effect, shadow view,
bubble effect, motion blur effect
DALi Features
Embedded Software Lab. @ SKKU
105
104
• ItemView
– Scrolling container based
on data source provided
by application
• ScrollView
– Scrolling container with
scroll effect support
• 3D Models & Bone
Animation
– Industry standard model
supports(Maya, 3DS, …)
• Physical Integration
– Rigid & soft body physics
effects
DALi Features
Embedded Software Lab. @ SKKU
105
105
• Tizen Developer
Conference 2012
– Creating Fancy UIs with
EDJE
– Overview of Graphics and
Input in Tizen
– Tizen Graphics Core Evas
and Scene Graph
• Tizen Developer
Conference 2014
– The Art of Tizen UI
Theme Technology in
Various Profiles
– Tizen 3D UI DALi 3D
Engine
References
• LinuxCon #
– 2012: E17 and EFL
– 2013: The EFL Toolkit
• EFL Workshop 2012 #
– Tizen Native Display Layer:
Architecture and Usage
• Tizen Technical
Workshop #
• EFL Dev Day 2013 #
– Cserve2: Shared Cache
Subsystem for Evas
– EFL on Wayland

More Related Content

What's hot

SIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLSIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGL
Mark Kilgard
 
Android power management
Android power managementAndroid power management
Android power management
Jerrin George
 

What's hot (20)

Scheduling in Android
Scheduling in AndroidScheduling in Android
Scheduling in Android
 
The Android graphics path, in depth
The Android graphics path, in depthThe Android graphics path, in depth
The Android graphics path, in depth
 
Memory Management in TIZEN - Samsung SW Platform Team
Memory Management in TIZEN - Samsung SW Platform TeamMemory Management in TIZEN - Samsung SW Platform Team
Memory Management in TIZEN - Samsung SW Platform Team
 
Android presentation
Android presentationAndroid presentation
Android presentation
 
Design and Concepts of Android Graphics
Design and Concepts of Android GraphicsDesign and Concepts of Android Graphics
Design and Concepts of Android Graphics
 
SIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGLSIGGRAPH Asia 2008 Modern OpenGL
SIGGRAPH Asia 2008 Modern OpenGL
 
Android AIDL Concept
Android AIDL ConceptAndroid AIDL Concept
Android AIDL Concept
 
Embedded Android : System Development - Part II (Linux device drivers)
Embedded Android : System Development - Part II (Linux device drivers)Embedded Android : System Development - Part II (Linux device drivers)
Embedded Android : System Development - Part II (Linux device drivers)
 
Android power management
Android power managementAndroid power management
Android power management
 
Binder: Android IPC
Binder: Android IPCBinder: Android IPC
Binder: Android IPC
 
Embedded Android : System Development - Part III
Embedded Android : System Development - Part IIIEmbedded Android : System Development - Part III
Embedded Android : System Development - Part III
 
Automotive android
Automotive androidAutomotive android
Automotive android
 
Android media framework overview
Android media framework overviewAndroid media framework overview
Android media framework overview
 
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
"Learning AOSP" - Android Hardware Abstraction Layer (HAL)
 
Android internals 07 - Android graphics (rev_1.1)
Android internals 07 - Android graphics (rev_1.1)Android internals 07 - Android graphics (rev_1.1)
Android internals 07 - Android graphics (rev_1.1)
 
OpenGL L04-Lighting
OpenGL L04-LightingOpenGL L04-Lighting
OpenGL L04-Lighting
 
Developing Automotive Linux
Developing Automotive LinuxDeveloping Automotive Linux
Developing Automotive Linux
 
Embedded Android Workshop
Embedded Android WorkshopEmbedded Android Workshop
Embedded Android Workshop
 
Siggraph2016 - The Devil is in the Details: idTech 666
Siggraph2016 - The Devil is in the Details: idTech 666Siggraph2016 - The Devil is in the Details: idTech 666
Siggraph2016 - The Devil is in the Details: idTech 666
 
Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?Android Treble: Blessing or Trouble?
Android Treble: Blessing or Trouble?
 

Similar to Why is EFL used on Tizen?

Enlightenment Foundation Libraries (Overview)
Enlightenment Foundation Libraries (Overview)Enlightenment Foundation Libraries (Overview)
Enlightenment Foundation Libraries (Overview)
Samsung Open Source Group
 
Minko - Flash Conference #5
Minko - Flash Conference #5Minko - Flash Conference #5
Minko - Flash Conference #5
Minko3D
 
WebGL games with Minko - Next Game Frontier 2014
WebGL games with Minko - Next Game Frontier 2014WebGL games with Minko - Next Game Frontier 2014
WebGL games with Minko - Next Game Frontier 2014
Minko3D
 

Similar to Why is EFL used on Tizen? (20)

Enlightenment Foundation Libraries (Overview)
Enlightenment Foundation Libraries (Overview)Enlightenment Foundation Libraries (Overview)
Enlightenment Foundation Libraries (Overview)
 
OpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIsOpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIs
 
The Art of Tizen UI Theme Technology in Various Profiles (OSCON 2014)
The Art of Tizen UI Theme Technology in Various Profiles (OSCON 2014)The Art of Tizen UI Theme Technology in Various Profiles (OSCON 2014)
The Art of Tizen UI Theme Technology in Various Profiles (OSCON 2014)
 
Whats new in Eclipse Indigo ? (@DemoCamp Grenoble 2011)
Whats new in Eclipse Indigo ? (@DemoCamp Grenoble 2011)Whats new in Eclipse Indigo ? (@DemoCamp Grenoble 2011)
Whats new in Eclipse Indigo ? (@DemoCamp Grenoble 2011)
 
Minko - Flash Conference #5
Minko - Flash Conference #5Minko - Flash Conference #5
Minko - Flash Conference #5
 
EFL (Tizen Native Display Layer – Architecture & Usage) - Carsten Haitzler
EFL (Tizen Native Display Layer – Architecture & Usage)  - Carsten HaitzlerEFL (Tizen Native Display Layer – Architecture & Usage)  - Carsten Haitzler
EFL (Tizen Native Display Layer – Architecture & Usage) - Carsten Haitzler
 
Advanced Graphics Workshop - GFX2011
Advanced Graphics Workshop - GFX2011Advanced Graphics Workshop - GFX2011
Advanced Graphics Workshop - GFX2011
 
Eclipse Testing Day 2010. Xored Q7
Eclipse Testing Day 2010. Xored Q7Eclipse Testing Day 2010. Xored Q7
Eclipse Testing Day 2010. Xored Q7
 
Omnet++
Omnet++Omnet++
Omnet++
 
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
 
Containers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshellContainers, Serverless and Functions in a nutshell
Containers, Serverless and Functions in a nutshell
 
WebGL games with Minko - Next Game Frontier 2014
WebGL games with Minko - Next Game Frontier 2014WebGL games with Minko - Next Game Frontier 2014
WebGL games with Minko - Next Game Frontier 2014
 
Efficient and Advanced Omniscient Debugging for xDSMLs (SLE 2015)
Efficient and Advanced Omniscient Debugging for xDSMLs (SLE 2015)Efficient and Advanced Omniscient Debugging for xDSMLs (SLE 2015)
Efficient and Advanced Omniscient Debugging for xDSMLs (SLE 2015)
 
Unreal Engine 4 Introduction
Unreal Engine 4 IntroductionUnreal Engine 4 Introduction
Unreal Engine 4 Introduction
 
Puppet devops wdec
Puppet devops wdecPuppet devops wdec
Puppet devops wdec
 
Eclipse e4
Eclipse e4Eclipse e4
Eclipse e4
 
Simulating Networks Using Cisco Modeling Labs (TechWiseTV Workshop)
Simulating Networks Using Cisco Modeling Labs (TechWiseTV Workshop)Simulating Networks Using Cisco Modeling Labs (TechWiseTV Workshop)
Simulating Networks Using Cisco Modeling Labs (TechWiseTV Workshop)
 
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
 
Unikernels
UnikernelsUnikernels
Unikernels
 
Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015
 

More from Ryo Jin

More from Ryo Jin (20)

Samsung Z4 User Manual
Samsung Z4 User ManualSamsung Z4 User Manual
Samsung Z4 User Manual
 
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Samsung ARTIK 050 (ARTIK ZERO) Modules Data SheetSamsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
Samsung ARTIK 050 (ARTIK ZERO) Modules Data Sheet
 
Introduction to Watch Face Development with Tizen Studio
Introduction to Watch Face Development with Tizen StudioIntroduction to Watch Face Development with Tizen Studio
Introduction to Watch Face Development with Tizen Studio
 
Panduan Penggunaan Perangkat Wearable Tizen
Panduan Penggunaan Perangkat Wearable TizenPanduan Penggunaan Perangkat Wearable Tizen
Panduan Penggunaan Perangkat Wearable Tizen
 
Cara Menggunakan Smartphone Tizen
Cara Menggunakan Smartphone TizenCara Menggunakan Smartphone Tizen
Cara Menggunakan Smartphone Tizen
 
Gear Fit2 Watchface Design Guide
Gear Fit2 Watchface Design GuideGear Fit2 Watchface Design Guide
Gear Fit2 Watchface Design Guide
 
Samsung Indonesia: Tizen Store
Samsung Indonesia: Tizen StoreSamsung Indonesia: Tizen Store
Samsung Indonesia: Tizen Store
 
Samsung Indonesia: Tizen Wearables
Samsung Indonesia: Tizen WearablesSamsung Indonesia: Tizen Wearables
Samsung Indonesia: Tizen Wearables
 
Samsung Indonesia: Tizen Web Apps
Samsung Indonesia: Tizen Web AppsSamsung Indonesia: Tizen Web Apps
Samsung Indonesia: Tizen Web Apps
 
Samsung Indonesia: Tizen Native App
Samsung Indonesia: Tizen Native AppSamsung Indonesia: Tizen Native App
Samsung Indonesia: Tizen Native App
 
Samsung Indonesia: Tizen Platform Overview and IoT
Samsung Indonesia: Tizen Platform Overview and IoTSamsung Indonesia: Tizen Platform Overview and IoT
Samsung Indonesia: Tizen Platform Overview and IoT
 
Russian Tizen Project
Russian Tizen ProjectRussian Tizen Project
Russian Tizen Project
 
Samsung SM-R360 Tizen User Manual
Samsung SM-R360 Tizen User ManualSamsung SM-R360 Tizen User Manual
Samsung SM-R360 Tizen User Manual
 
Tizen Micro Profile for IoT device
Tizen Micro Profile for IoT deviceTizen Micro Profile for IoT device
Tizen Micro Profile for IoT device
 
Panduan Dasar Pemrograman Tizen
Panduan Dasar Pemrograman TizenPanduan Dasar Pemrograman Tizen
Panduan Dasar Pemrograman Tizen
 
The Story of Enlightenment, EFL, Tizen and Wayland
The Story of Enlightenment, EFL, Tizen and WaylandThe Story of Enlightenment, EFL, Tizen and Wayland
The Story of Enlightenment, EFL, Tizen and Wayland
 
Tizen PASS
Tizen PASSTizen PASS
Tizen PASS
 
Tizen PASS
Tizen PASSTizen PASS
Tizen PASS
 
Samsung Gear UI Design Guidelines
Samsung Gear UI Design GuidelinesSamsung Gear UI Design Guidelines
Samsung Gear UI Design Guidelines
 
Attack surface analysis of Tizen devices
Attack surface analysis of Tizen devicesAttack surface analysis of Tizen devices
Attack surface analysis of Tizen devices
 

Recently uploaded

%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
masabamasaba
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
 

Recently uploaded (20)

%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto
 
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdfAzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf
 
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public AdministrationWSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
 
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...
 
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next IntegrationWSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
WSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AIWSO2CON 2024 Slides - Unlocking Value with AI
WSO2CON 2024 Slides - Unlocking Value with AI
 
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...
 
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and ApplicationsWSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
WSO2CON 2024 - Architecting AI in the Enterprise: APIs and Applications
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of TransformationWSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
WSO2CON 2024 - Designing Event-Driven Enterprises: Stories of Transformation
 
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public AdministrationWSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
WSO2CON 2024 - How CSI Piemonte Is Apifying the Public Administration
 
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
WSO2CON 2024 - IoT Needs CIAM: The Importance of Centralized IAM in a Growing...
 
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAMWSO2Con2024 - Organization Management: The Revolution in B2B CIAM
WSO2Con2024 - Organization Management: The Revolution in B2B CIAM
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
WSO2Con2024 - Hello Choreo Presentation - Kanchana
WSO2Con2024 - Hello Choreo Presentation - KanchanaWSO2Con2024 - Hello Choreo Presentation - Kanchana
WSO2Con2024 - Hello Choreo Presentation - Kanchana
 
WSO2Con2024 - Low-Code Integration Tooling
WSO2Con2024 - Low-Code Integration ToolingWSO2Con2024 - Low-Code Integration Tooling
WSO2Con2024 - Low-Code Integration Tooling
 
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open SourceWSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
WSO2CON 2024 - Freedom First—Unleashing Developer Potential with Open Source
 
WSO2Con2024 - From Blueprint to Brilliance: WSO2's Guide to API-First Enginee...
WSO2Con2024 - From Blueprint to Brilliance: WSO2's Guide to API-First Enginee...WSO2Con2024 - From Blueprint to Brilliance: WSO2's Guide to API-First Enginee...
WSO2Con2024 - From Blueprint to Brilliance: WSO2's Guide to API-First Enginee...
 

Why is EFL used on Tizen?

  • 1. Embedded Software Lab. @ SKKU 105 1 Tizen v2.3 Graphics & UI Frameworks
  • 2. Embedded Software Lab. @ SKKU 105 2 • EFL(Enlightenment Foundation Library) – EFL Features – Ecore – Evas – Edje – Elementary – Cserve2 • Window Systems – X Window System – Wayland • DALi 3D Engine Contents
  • 3. Embedded Software Lab. @ SKKU 105 3 • UI Frameworks – Modules to implement interactive GUI – EFL(Elementary/Ecore), OSP, GTK, Qt – Input Service Framework(ISF), Voice Framework • Graphics Libraries – Modules to render graphic objects – Evas, Cairo, OpenGL ES, EGL • Window Systems – Modules to manage windows and frame buffers – X Window System, Wayland(Tizen v3.0) Tizen v2.3 Graphics & UI Frameworks
  • 4. Embedded Software Lab. @ SKKU 105 4Tizen v2.3 Graphics & UI Frameworks • Frameworks for UI of core applications & web applications Deprecated (Bada App FW)
  • 5. Embedded Software Lab. @ SKKU 105 5 EFL (Enlightenment Foundation Library)
  • 6. Embedded Software Lab. @ SKKU 105 6 • Started from a desktop window manager – Default window manager of Bodhi Linux Enlightenment
  • 7. Embedded Software Lab. @ SKKU 105 7 • A set of libraries supporting Enlightenment – Enlightenment 0.17(E17) • Components are divided by functions – Event loop: Ecore – Rendering engine: Evas – UI: Edje, Elementary, … – UI builder tools: Enventor, Eflete, ELM Theme Editor • EFL in Present – Core components of Tizen Graphics/UI framework – 26% of code is E17, the other is EFL Enlightenment Foundation Library
  • 8. Embedded Software Lab. @ SKKU 105 8 EFL in Tizen v2.3 Legend Applications EFL & fontsWindow System H/W Linux Kernel Frame Buffer Input Devices H/W fontconfigfreetype2 OpenGL ES/EGL Eet Evas Ecore Eina Edje Elementary Core Applications X Window System Input Service Framework & Voice Framework kernel / Frame Buffer/ Input Devices Graphics ISF& Voice FW Cairo
  • 9. Embedded Software Lab. @ SKKU 105 9 • Immediate Mode Graphics System – Application directly re-paints its portion that is invalidated – Application re-issues all drawing commands to describe the entire scene each time a new frame is required – GTK+ 3, GDI, GDI+ Immediate vs. Retained Mode (1/2) Application Window System GTK+/GDK-X11 1. Invalidate 2. Expose Graphics Card Frame Buffer for Screen 3. Drawing
  • 10. Embedded Software Lab. @ SKKU 105 10 • Retained Mode Graphics System – The graphics system is responsible for rendering the drawing objects given by application. – Application passes drawing objects to graphics system. – Evas, GTK+ 4(Clutter), WPF Immediate vs. Retained Mode (2/2) Application Evas objects Evas Rendering Engine (Graphics system) Expose Graphics Card Framebuffer for screen X Window System
  • 11. Embedded Software Lab. @ SKKU 105 11 • High performance – Retained mode graphics system – 60fps+ on general smartphones • Fit for Low-end Hardware – Small memory, no swap, no GPUs – OpenGL is not worth for all cases • Texture keeps speed, but loses memory • Support for HW rendering – OpenGL ES – SIMD vector(MMX, Neon) Why is EFL used on Tizen? (1/3) Desktop Environment Memory Used (MiB) Enlightenment 0.18.8 83.8 LXDE 0.5.5 87.0 XFCE 4.10.2 110.0 LXQt 0.7.0 113.0 MATE 1.8.1 123.0 Cinnamon 2.2.13 176.3 GNOME3 3.12.2 245.3 KDE 4.13.1 302.6 Unity 7.2.0.14 312.5
  • 12. Embedded Software Lab. @ SKKU 105 12 • Various back-end engine – X11 (~Tizen 2.2.1), Wayland (Tizen 3.0~) – Direct Frame Buffer, DRM, Memory buffers, PS3 Native, Windows (GDI & DirectDraw), … • UI Scalability – Various resolutions (WVGA ~ XQXGA) with the same layout and resources Why is EFL used on Tizen? (2/3) 0.6 0.8 1.0Scale factor:
  • 13. Embedded Software Lab. @ SKKU 105 13 • GUI, Logic separation • Themes – A preset package containing graphical appearance details • Bindings – Basically C-based – EFL is also binded to Javascript, C++, Python, Ruby, Perl … – Eo: Generic object model Why is EFL used on Tizen? (3/3)
  • 14. Embedded Software Lab. @ SKKU 105 14 • Entry barrier – Unfamiliar development style – Lack of documentations, developers • No object-oriented implementation – C-based – Eo project is ongoing • Incomplete C++ support – Eolian based E++ project ongoing – Eolian is EO object parser and C code generator • No thread safe EFL Shortcomings
  • 15. Embedded Software Lab. @ SKKU 105 15 Tizen v2.3 Rendering Path Textures OpenGL ES win X window server Frame Buffer Video win EFL win 3D App OpenGL ES/EGL Video App. EFL App. EFL EVAS Composite Window Manager OpenGL ES/EGLOpenGL ES/EGL MM FWEFL Rendering pipeline path; rendering data flow Legend 3D App. Multimedia App.EFL App. Configurable Enable/disable Texture from Pixmap OpenGL ES/EGL Process Texture Mem.
  • 16. Embedded Software Lab. @ SKKU 105 16Tizen v2.3 Rendering Path on SW Backend Elementary Ecore xlib Frame buffer LCD controller 60Hz Update window areas X Server X shared Mem. Edje Evas Evas software backendEcore X
  • 17. Embedded Software Lab. @ SKKU 105 17Tizen v2.3 Rendering Path on GLES Backend EGL for X xlib Frame buffer /dev/fb1 LCD controller 60Hz Update window areaX Server frontback Double buffer OpenGL ES H/W OpenGL ES Ecore Elementary Edje Evas Evas OpenGL ES backend Ecore X
  • 18. Embedded Software Lab. @ SKKU 105 18 • Ecore (E + Core) – Event loop library with convenience modules • Evas (E + canVAS) – Canvas & rendering library • Edje – Complex graphical design & layout library • Elementary – Widget set EFL Components: Core
  • 19. Embedded Software Lab. @ SKKU 105 19 • Eo – Generic object model for binding other languages • Eina – Library for data types • Eet – Data codec and storage • Efreet – Freedesktop.org standards support EFL Components: Others • Edbus – Wrapper of Dbus • Emotion – Video/Audio Playback library • Eio – Asynchronous I/O • Eeze – Udev hardware detection • Ethumb – Thumbnailer & cacher
  • 20. Embedded Software Lab. @ SKKU 105 20 The core loop of EFL applications EFL: Ecore
  • 21. Embedded Software Lab. @ SKKU 105 21 • The core loop of EFL application, event & marshaling library – Provides glue to other subsystems – Supports various display system • X11, FB, Wayland, Win32, WinCE, etc. • Features: – Networking & IPC – Threading & inter-thread communication – Evas input feeding and output driving – More... Ecore
  • 22. Embedded Software Lab. @ SKKU 105 22 • elm_main() – Main function of EFL application – After initialization, enter into event loop. Simple EFL Application Start Ecore main loop! Elementary UI configuration (Elementary Window) On exiting window, it also exits Ecore event loop Finalization process like freeing resource
  • 23. Embedded Software Lab. @ SKKU 105 23 • ecore_main_loop_begin() – The wrapper of Ecore main loop – Ecore main loop and Glib main loop can be chosen. Ecore Main Loop (1/2) Ecore main loop Glib main loop
  • 24. Embedded Software Lab. @ SKKU 105 24 • _ecore_main_loop_iterate_internal() – The implementation of Ecore main loop – It starts all the timer handler, event handler, rendering in an application. Ecore Main Loop (2/2)
  • 25. Embedded Software Lab. @ SKKU 105 25 • Key down event Ecore: Usage of Ecore Events Ecore_Event_Handler *handler; handler = ecore_event_handler_add(ECORE_EVENT_KEY_DOWN, func, data); Eina_Bool func(void *data, int type, void *event_info) { if (…) return ECORE_CALLBACK_DONE; return ECORE_CALLBACK_PASS_ON; } Ecore_Timer *timer; double time; timer = ecore_timer_add(time, func, data); Eina_Bool func(void *data) { if (…) return ECORE_CALLBACK_RENEW; return ECORE_CALLBACK_CANCEL; } • Timer event
  • 26. Embedded Software Lab. @ SKKU 105 26 Canvas & Rendering Library EFL: Evas
  • 27. Embedded Software Lab. @ SKKU 105 27 • Canvas & Rendering Library • Features – Retained mode rendering • Controls rendering behavior in the unit of primitive objects and smart objects. – Scene graph • Tracks the status of objects → minimizes rendering behavior. – Multiple output paths • Pure software/OpenGL ES – Rendering optimization • 3D H/W Acceleration, SIMD vector operations – Provides GL Glue Layer Evas
  • 28. Embedded Software Lab. @ SKKU 105 28 • Evas uses retained mode rendering – Application sends Evas objects to Evas. – Evas does rendering based on Evas objects. Evas: Retained Mode Rendering Application Evas objects Evas Rendering Engine (Graphics system) Expose Graphics Card Framebuffer for screen X Window System
  • 29. Embedded Software Lab. @ SKKU 105 29 • All Objects Displayed on Canvas – Controlled by Evas_Object handles. • Loading fonts and images, rendering glyphs and images, scaling, blending, 3D transformation etc. – Evas manages all Evas objects equally. • Type of Evas Object – Primitive objects: minimum unit of Evas rendering • Rectangle, Line, Polygon, Text, Textblock, Textgrid, Image – Smart objects • Box, Grid, Table, Customized smart objects(ELM Widgets) Evas Objects
  • 30. Embedded Software Lab. @ SKKU 105 30 Evas Primitive Objects • Rendering units of Evas
  • 31. Embedded Software Lab. @ SKKU 105 31 • Evas objects composed of primitive objects – Acts as a group. – Evas object generic functions also can be used. Evas Smart Objects
  • 32. Embedded Software Lab. @ SKKU 105 32 Image Native Surface Hello World! Evas_Object_Rect Evas_Object_Text Evas_Object_Image … GL Rendering Engine SW Rendering Engine X11 Rendering Engine X11 Output Buffer Output … SDL Output … Evas Runtime OpenGL App (surface_a ie. Pixmap or texture) Evas_Object_Image (img_obj_a) evas_object_image_native_surface_set(img_obj_a, surface_a) Output Hello World! Evas Engine Module
  • 33. Embedded Software Lab. @ SKKU 105 33 • Scene graph – A structure that arranges the logical representation of a graphical scene – General data structure in GUI applications • Scene graph in Evas – Manages Evas objects in graph data structure. – Tracks all the objects’ status and gets changed area in the next frame → Minimizes rendering area. Evas: Scene Graph
  • 34. Embedded Software Lab. @ SKKU 105 34 • Start Here Evas: Minimizing Rendering Region
  • 35. Embedded Software Lab. @ SKKU 105 35 • Next frame is… Evas: Minimizing Rendering Region
  • 36. Embedded Software Lab. @ SKKU 105 36 • Calculate actual update region deltas Evas: Minimizing Rendering Region
  • 37. Embedded Software Lab. @ SKKU 105 37 • Only draw updated regions Evas: Minimizing Rendering Region
  • 38. Embedded Software Lab. @ SKKU 105 38 /* Build by $ gcc evas_rectangle.c -o evas_rectangle `pkg-config --cflags --libs ecore ecore-evas` */ #include <Ecore.h> #include <Ecore_Evas.h> int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee); evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_rectangle_add(evas); // Add a rectangle to the given Evas object. evas_object_color_set(bg, 255, 255, 255, 255); // Set the color of the given Evas object to the given one. evas_object_move(bg, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(bg, 200, 200); // Change the size of the given Evas object. evas_object_show(bg); // Make the given Evas object visible. ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown(); return 0; } Evas: Rectangle Example
  • 39. Embedded Software Lab. @ SKKU 105 39 /* Build by $ gcc evas_image.c -o evas_image `pkg-config --cflags --libs ecore ecore-evas` */ #include <Ecore.h> #include <Ecore_Evas.h> int main(int argc, char **argv) { Ecore_Evas *ee; Evas *evas; Evas_Object *bg; if(ecore_evas_init() <= 0) return 1; ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL); ecore_evas_show(ee); evas = ecore_evas_get(ee); // Get the Ecore_Evas's Evas object. bg = evas_object_image_filled_add(evas); // Add an image to the given Evas object. evas_object_image_file_set(img, “test.png”, NULL); // Set the image file path of the given Evas object. evas_object_move(img, 0, 0); // Move the given Evas object to the given location inside its canvas' viewport. evas_object_resize(img, 200, 200); // Change the size of the given Evas object. evas_object_show(img); // Make the given Evas object visible. ecore_main_loop_begin(); ecore_evas_free(ee); ecore_evas_shutdown(); return 0; } Evas: Image Example
  • 40. Embedded Software Lab. @ SKKU 105 40 Complex graphical design & layout library EFL: Edje
  • 41. Embedded Software Lab. @ SKKU 105 41 • Complex graphical design & layout library • EDC script – GUI layout description separated from application code – Can be changed at or during runtime – Compiled as binary code(edj) Edje • Description about: – Images, colors, positioning, animations, behaviors, …
  • 42. Embedded Software Lab. @ SKKU 105 42 • Separation of layout and logic – Graphical part: GUI Layout binary (edj) – Functionality: Executable logic binary (C) • Utility – edje_cc : compile edc to edj – edje_decc : de-compile edj to edc Edje: Edc & Edj
  • 43. Embedded Software Lab. @ SKKU 105 43 Edje: Relation with Evas, Ecore
  • 44. Embedded Software Lab. @ SKKU 105 44 Edje: EDC Script
  • 45. Embedded Software Lab. @ SKKU 105 45 Edje: EDC Script
  • 46. Embedded Software Lab. @ SKKU 105 46 Edje: Simple Example part { name: “background”; type: RECT; mouse_events: 0; description { state: “default” 0.0; color: 255 255 255 255; } } part { name: “blue_rect”; type: RECT; mouse_events: 1; description { state: “default” 0.0; color: 0 0 255 255; rel1 { relative: 0.1 0.1; } rel2 { relative: 0.3 0.3; } } }
  • 47. Embedded Software Lab. @ SKKU 105 47 Edje: Simple Example part { name: “red_rect”; type: RECT; mouse_events: 0; description { state: “default” 0.0; color: 255 0 0 255; rel1 { to: “blue_rect”; relative: 0 2; } rel2 { to: “blue_rect”; relative: 1 3; } } description { state: “down” 0.0; inherit: “default” 0.0; rel1.relative: 3 2; rel2.relative: 4 3; color: 255 0 0 0; } }
  • 48. Embedded Software Lab. @ SKKU 105 48 Edje: Simple Example program { name: “blue_rect.clicked”; signal: “mouse,clicked,1”; source: “blue_rect”; action: STATE_SET “down” 0.0; transition: ACCELERATE 0.5; target: “red_rect”; after: “red_rect.restore”; } program { name: “red_rect.restore”; action: STATE_SET “default” 0.0; target: “red_rect”; }
  • 49. Embedded Software Lab. @ SKKU 105 49 A set of widgets EFL: Elementary
  • 50. Embedded Software Lab. @ SKKU 105 50 • A Set of Widgets – 약 80여 개 – Containers: Box, Conformant, Grid, Layout, Panes, Scroller, Table, … – Non-Containers: Bg, Button, Check, Entry, Label, List, Icon, Win, ... • Features: – Fast, Finger friendly, Scalability, Themes – Widget hierarchy, Dynamic language change Elementary < Scaled and resized relative to the screen> < Same view but different themes>
  • 51. Embedded Software Lab. @ SKKU 105 51 • Various Widgets Elementary Widgets clock colorselector entry frame flipselector check diskselector actionslider bg index map menu photo, photocam panel toolbar radio bubble slider ctxpopup notify separator segment control multibuttonentry panes glview fileselector Label thumb
  • 52. Embedded Software Lab. @ SKKU 105 52 • Naviframe – Switching multiple pages with stack • Toolbar – List of Items Elementary Widgets
  • 53. Embedded Software Lab. @ SKKU 105 53 • Entry – Text input field – Copy & paste, various modes (password, multi- line, ...) • Genlist – Generic List – Flexible but complex to use Elementary Widgets
  • 54. Embedded Software Lab. @ SKKU 105 54 • Widgets are built in a hierarchical fashion. • Widget shares it’s functions(APIs) with child widget. Elementary: Widget Hierarchy
  • 55. Embedded Software Lab. @ SKKU 105 55 • UI Scalability – Relative Positioning Elementary: Scalability – Scale Factor
  • 56. Embedded Software Lab. @ SKKU 105 56 • Theme can be changed in same layout Elementary: Theme
  • 57. Embedded Software Lab. @ SKKU 105 57 • Elementary provides utility APIs for multi language support Elementary: Dynamic Language Change
  • 58. Embedded Software Lab. @ SKKU 105 58Elementary: Dynamic Language Change
  • 59. Embedded Software Lab. @ SKKU 105 59 • Enventor: Dynamic EDC Editor • Eflete: EFL Edje Theme Editor • ELM Theme Viewer: Elementary Theme Viewer GUI Tools for Customizing Layout
  • 60. Embedded Software Lab. @ SKKU 105 60 Optimization of EFL Cserve2
  • 61. Embedded Software Lab. @ SKKU 105 61 • Duplicated Resource Problem in EFL – Most images and fonts used in a theme are used by multiple applications at the same time – EFL contains a variety of caches and • Images, fonts, ... • Objectives of Cserve2: – Reduce memory consumption by sharing – Optimize load time by pre-loading data Cserve2: Motivation
  • 62. Embedded Software Lab. @ SKKU 105 62 • Sharing image and font data between EFL application processes – Started by ProFusion at 2012 • Cache server – Loads images and fonts – Shares resources across applications • EFL applications – Connect to the server via UNIX socket – Send requests and wait for answers – Don't load anything on their own – Keep local cache pointing to shared data Cserve2
  • 63. Embedded Software Lab. @ SKKU 105 63 Cserve2
  • 64. Embedded Software Lab. @ SKKU 105 64 • Problems on Cserve2 – IPC overheads: • messages over socket link • context switches, read, write, ... – Clients do not know the server's cached data before sending a request • Solution – Expose indexes of shared objects to all clients • via /dev/shm – Clients scan the tables and don't wait for the server's response when the data is already loaded Cserve2: Shared Indexing
  • 65. Embedded Software Lab. @ SKKU 105 65 • 3 types of objects: – Data (image data) • Stored in separate shm files: “img” – Fixed-size arrays of objects • Indexes • Stored in “array” files – Variable-size pools of objects • So-called “mempools” • Indexed by offset + length in a separate array Cserve2: Shared Indexing
  • 66. Embedded Software Lab. @ SKKU 105 66 • Index tables principles – Arrays of simple structures of a single type – Contain: • ID • Refcount • Other fixed-size data (int, bool, ...) Cserve2: Shared Indexing
  • 67. Embedded Software Lab. @ SKKU 105 67 • Strings – Two shm files are used: index and mempool – Mempool contains the strings actual data – Index table contains: • String ID • Refcount • Offset (in the mempool) • Length • So, we can read a string just knowing its string ID Cserve2: Shared Indexing
  • 68. Embedded Software Lab. @ SKKU 105 68 • Cserve2 combines image cache and scale cache • Reuse scale cache logic – Minimizes the number of cached scale images. – Scaling process is done on server side. Cserve2: Image Scaling
  • 69. Embedded Software Lab. @ SKKU 105 69 • Cserve2 image loader crash – No crash on client side (just no data) – New slave spawned • Cserve2 server crash – enlightenment_start restarts cserve2 – Clients reconnect to new server • Discard some data • Keep old image references (no need to load) • Cserve2 shutdown or restart – cleanup /dev/shm/ Cserve2: Robustness
  • 70. Embedded Software Lab. @ SKKU 105 70 • Real Phone Usage – Workload & Environment • 8 threads of “elementary_test -to launcher” on 1280x720 screen – Font size • ~24x16 (small), ~32x24 (normal), ~40x30 (large) • In case of normal size, 1 font, 50 characters: 1x32x24x50=37.5KB – Icon size • 50x50 (small), 100x100 (large) in 32-bit color • For one icon, 4Bx50x50=10KB (small), 4Bx100x100=40KB (large) – Border size • width 10px, length 720px, 32-bit color, 2x • 4Bx10x720x2=56KB Cserve2: Benefits
  • 71. Embedded Software Lab. @ SKKU 105 71 • Overhead – When workload runs with Cserved2 turned on: 72176KB • Size of /dev/shm: 4376KB – When Cserve2 quits: 66716KB – Total overhead: 72176KB–66716KB–4376KB=1084KB • Conclusion: – Cserve2 saves memory as soon as 1MB of data can be shared(fonts & images). • Refer to “Cserve2: Shared cache subsystem of Evas”. Cserve2: Overhead
  • 72. Embedded Software Lab. @ SKKU 105 72 Tizen v2.3 Window Systems: X Window Systems
  • 73. Embedded Software Lab. @ SKKU 105 73 • History – X window system is made by MIT in 1984. – X11: the current protocol version, established in 1987. – X.org server: the current reference implementation developed by the X.Org Foundation • Designed to be used over network connections • Features – Event Driven – Network Transparent – Device Independent – Policy Independent – Extensible X Window System
  • 74. Embedded Software Lab. @ SKKU 105 74 • Client-server Model – X client is linked with X server through the X protocol translated by Xlib. – X client • Sends graphical request • Gets user input. • Ecore and Evas in Tizen – X server • Accepts request for graphical output • Sends back user input X Window System: Architecture (1)
  • 75. Embedded Software Lab. @ SKKU 105 75 X Window System: Architecture (2) • Client-server Architecture of X System
  • 76. Embedded Software Lab. @ SKKU 105 76 X Protocol • Protocol about communication with X clients and the X server • Features: – Object-based – Asynchronous • Types of Messages: – Request (Client → Server) – Reply – Event (Server → Client) – Error
  • 77. Embedded Software Lab. @ SKKU 105 77 X Server • Handles Output & Input Processes • Manages Screens and Input Devices • Layered Structure – DIX Layer, DDX Layer, OS Layer, Extension Layer • Resource – Everything in X Server controlled through “resources” – Created on demand – Used by clients to request operations – Screens, windows, graphic contexts, pixmaps, regions, colormaps, fonts, cursors
  • 78. Embedded Software Lab. @ SKKU 105 78 X Server • Layered Structure: – DIX Layer, DDX Layer, OS Layer, Extension Layer
  • 79. Embedded Software Lab. @ SKKU 105 79 • DIX(Device Independent X) Layer – Interface between X Clients and DDX/extensions • DDX(Device Dependent X) Layer – Deals with graphic and input hardware • OS Layer – Client connection management, work scheduling, memory allocation routines • Extension Layer – Enables to get additional functionalities for X server X Server
  • 80. Embedded Software Lab. @ SKKU 105 80 DIX Layer • Interface among DDX/Extensions and X Clients – Delivers requests from clients to DDX/extensions – Sends input events to clients • Dispatch loop – DIX Layer manages requests with dispatch loop • Screens – One for each physical screen – Defines functions used to handle operations on resources – Main communication channel DIX-DDX
  • 81. Embedded Software Lab. @ SKKU 105 81 DDX Layer • A layer dealing with graphic and input HW • Graphics Output • Input Processing – Mouse movement – Key presses – Key mappings
  • 82. Embedded Software Lab. @ SKKU 105 82 DDX Layer: Graphics Output • Pixel-based • Screen basic data structure – Manage all resources • Output done on drawables – Pixel data format defined by DDX – Pixmap format constant across screens • XYPixmap -> one bitmap for each plane • Zpixmap -> series of pixel values – Pixel values interpreted at display time – Windows, Pixmaps, … 82
  • 83. Embedded Software Lab. @ SKKU 105 83 DDX Drawables • Windows – Visible rectangle on screen – Border drawn by DDX • Create, move, resize – Contents drawn by client • Server may provide backing store • Bit gravity (resizing) – Clipped by parent and sibbling windows • Pixmaps – Rectangular array of pixel values – Reference counted
  • 84. Embedded Software Lab. @ SKKU 105 84 OS Layer • Client Monnection Management • Work Scheduling – It schedules work to be done for clients – Ignore/AttendClient() • Memory Allocation Routines – Xalloc()/Xrealloc()/Xfree()
  • 85. Embedded Software Lab. @ SKKU 105 85 Porting X Server • Code layers – DIX: Dev.Independent, don’t touch – OS: OS specific – DDX: Graphics Device specific – Extensions: Add features • Porting Layer: OS + DDX • Porting process: – Define functions required by DIX to handle resources
  • 86. Embedded Software Lab. @ SKKU 105 86 Tizen v3.0 Window Systems: Wayland
  • 87. Embedded Software Lab. @ SKKU 105 87 • Display management architecture & protocol – Window management and composition – Weston: main open source implementation – Maintained by Freedesktop.org – Adopted by Tizen 3.0 • Replacement for X – No rendering API in Wayland • Rendering has already been supported by Evas, Cairo, OpenGL ES – SHM(Shared memory) protocol: support for SW rendering – Less IPC: single process architecture Wayland
  • 88. Embedded Software Lab. @ SKKU 105 88 • Wayland Protocol – Weston compositor interacts with Wayland clients(applications) via Wayland protocol. Wayland Protocol
  • 89. Embedded Software Lab. @ SKKU 105 89 • EFL and E17(X compositor) are linked to X.org(X server) via X protocol Tizen v2.3 Graphic Stack with X.org
  • 90. Embedded Software Lab. @ SKKU 105 90 • EFL is linked to Weston/E17 compositor via Wayland protocols Tizen v3.0 Graphic Stack with Wayland
  • 91. Embedded Software Lab. @ SKKU 105 91 • Weston provides surface to be rendered to Evas • Weston sends input events to Ecore Wayland with EFL
  • 92. Embedded Software Lab. @ SKKU 105 92 • EFL controls display and input devices through several backends. – Ecore_Evas: Glue to the backends Backends of EFL • X Server Backend – Software_X11, OpenGL_X11 • Wayland Backend – Wayland_SHM: SW rendering – Wayland_EGL: HW rendering – Ecore_Wayland • Surface Handling and Input
  • 93. Embedded Software Lab. @ SKKU 105 93 • Performance of Wayland – 60 FPS(Frames Per Second) – 16ms for one frame from client to compositor • Visualization Tool: E-Graph – A tool to visualize log information and draw FPS curve – https://gitorious.org/e-graph/e-graph Wayland Optimization
  • 94. Embedded Software Lab. @ SKKU 105 94 • Problem – Rendering when resource is blocked by waiting VSync • Triple Buffering – Add one additional buffer for client and compositor – Before: ~40fps → After: ~48fps – Time spend on composition: 13ms Wayland Optimization
  • 95. Embedded Software Lab. @ SKKU 105 95 • Problem – Redraw overlapped surface → heavy load during composition • Opaque Region – Overlapped surface, set by Ecore and sent to Weston – Compositing time: ~13ms → ~5ms – FPS: 40fps → 60fps Wayland Optimization
  • 96. Embedded Software Lab. @ SKKU 105 96 DALi 3D Engine
  • 97. Embedded Software Lab. @ SKKU 105 97 • Tizen 3D UI – DALi 3D Engine & UI Toolkit • DALi is a 3D Engine – UI is represented as a 3D Scene Graph – Animations and Transitions are done using 3D Math (Vector, Quaternion & Matrix) – Rendering and Visual Effects are done using Open GL ES Shaders, Vertices and Textures – OpenGL ES 2 and 3 support • 2D world is the Z plane 0 in the 3D world – When using default camera DALi 3D Engine
  • 98. Embedded Software Lab. @ SKKU 105 98 • DALi is part of the Tizen Native Subsystem – Graphics & UI native module – Mobile and TV profiles • Implemented in C++ • DALi (Dynamic Animation Library) – 2D and 3D Application UIs with Realistic Effects & Animations – Home Screen, Lock Screen, Gallery, Music Player … DALi in Tizen System Architecture
  • 99. Embedded Software Lab. @ SKKU 105 99 • Evas – EFL drawing canvas – SW/GL rendering backends • DALi – OpenGL based Toolkit – Differential UX/ heavy contents • CoreGL – OpenGL wrapper – Performance optimization DALi and Graphics Framework
  • 100. Embedded Software Lab. @ SKKU 105 100 • Core Library – Event handling, Scene Graph, Rendering, Resource management • Adaptor – Threading model – Integration with the main loop • Platform abstraction – Resource loading and decoding with multiple threads • Toolkit – Reusable UI controls, – Effects and Scripting support DALi Architecture
  • 101. Embedded Software Lab. @ SKKU 105 101 • Scene graph based UI is a tree of Nodes – Each Node can have 0-N Children – Each Node inherits its parent Transformation – $ Position, Rotation, Scale • Allows easy layout and animation management – Each Node’s Transformation is relative to a reference point in the parent’s space • Anchor point in the Nodes own coordinate space • Parent origin in the Parents coordinate space – Child does not have to be inside its parent area DALi: 3D Scene Graph
  • 102. Embedded Software Lab. @ SKKU 105 102 • DALi uses multithreaded architecture – Best performance and scalability • Event Thread – The main thread in which application code and event handling runs • Update Thread – Updates the nodes on scene – Runs animations, constraints and physics • Render Thread – OpenGL drawing, texture and geometry uploading etc • Resource Threads – Loads font, image and model resources and decodes into bitmaps etc Multi-threaded Engine
  • 103. Embedded Software Lab. @ SKKU 105 103 • Actors & UI Controls – Stage: root of the world – Actors: image, text, mesh, … – UI controls: provide additional layout and scroll • Animations • Shader Effects – modify the appearance of objects during rendering • Image Effects – Cube transition effect, shadow view, bubble effect, motion blur effect DALi Features
  • 104. Embedded Software Lab. @ SKKU 105 104 • ItemView – Scrolling container based on data source provided by application • ScrollView – Scrolling container with scroll effect support • 3D Models & Bone Animation – Industry standard model supports(Maya, 3DS, …) • Physical Integration – Rigid & soft body physics effects DALi Features
  • 105. Embedded Software Lab. @ SKKU 105 105 • Tizen Developer Conference 2012 – Creating Fancy UIs with EDJE – Overview of Graphics and Input in Tizen – Tizen Graphics Core Evas and Scene Graph • Tizen Developer Conference 2014 – The Art of Tizen UI Theme Technology in Various Profiles – Tizen 3D UI DALi 3D Engine References • LinuxCon # – 2012: E17 and EFL – 2013: The EFL Toolkit • EFL Workshop 2012 # – Tizen Native Display Layer: Architecture and Usage • Tizen Technical Workshop # • EFL Dev Day 2013 # – Cserve2: Shared Cache Subsystem for Evas – EFL on Wayland