0
Display and Kernel
Review and Future
Kernel Recipes 2013
Paris
Laurent Pinchart
laurent.pinchart@ideasonboard.com
Problem Definition
display /
/ graphics /
/ video
Problem - Purpose
display / graphics /
video
format
memory /
deep pipeline
device / CPU
Problem - Source
format
memory / deep pipeline
device / CPU
rotation /
/ scaling /
/ composing
Problem - Processing
rotation
scaling
composing
X11 /
/ Wayland /
/ MIR /
/ SDL /
/ DirectFB /
/ Raw API
Problem - Stack
X11
Wayland
SDL
DirectFB
Raw API
DRM /
/ FBDEV /
/ V4L2
Problem - API
DRM
FBDEV
V4L2
Display – Scanout
Frame Buffer
Display – Composition
CRTC
Plane(s)
Composition
Origins
Origins – FBDEV
FBD
EV,Linux
1.3.94
1996 2012
4C
C
Form
ats
2000
Blanking
Origins – V4L2
V4L,Linux
2.2.0
1999 2002 2006
V4L2,Linux
2.4.0
V4L
deprecated
2008
V4L2
subdev
2011
M
edia
controller
vide...
Origins – DRM/KMS
2000
D
R
M
,Linux
2.2.18
2009
KM
S,TTM
2008
G
EM
Planes,D
M
ABU
F
2011 2012
D
um
b
Buffers
Features
Memory Management
DRM FB V4L2
Dynamic Allocation Yes No Yes
Multiple Buffers Yes panning Yes
Import dmabuf No userptr
Expo...
Mode Setting
DRM FB V4L2
Formats 4CC
RGB
4CC
4CC
Enumeration Planes No Yes
Negotiation No No Yes
Atomicity Yes No No
Transformations
DRM FB V4L2
Overlays Yes No Yes
Rotation Yes No Yes
Scaling Yes No Yes
Cropping/Panning Yes Yes Yes
Documentation
api core drivers
drm
fb
v4l2
DRM API Documentation
The DRM core exports several interfaces to applications,
generally intended to be used through corre...
Documentation
api core drivers
drm
fb
v4l2
Code Size
api core drivers
drm
fb
v4l2
Cumulative Changes - API
0
1000
2000
3000
4000
5000
6000
drm
fb
v4l2
drm
fb
v4l2
Cumulative Changes - Core
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
50000
drm
fb
v4l2
drm
fb
v4l2
Cumulative Changes - Drivers
0
200000
400000
600000
800000
1000000
1200000
1400000
1600000
1800000
drm
fb
v4l2
drm
fb
v4l2
Mailing List Traffic
0
200
400
600
800
1000
1200
1400
1600
1800
drm
fb
v4l2
drm+bug
Device Model – FBDEV
Frame Buffer
(memory)
Display
Device
Device Model – DRM/KMS
Frame Buffer
(memory)
CRTC Encoder Connector
Planes
(memory)
Encoder
Connector
Connector
Device Model – V4L2
Buffers
(memory)
Display
Device
Connector
Device Model – V4L2/MC
V4L2
subdev
0
1
2
V4L2
subdev
0 1
V4L2
queue
0
Device Model – V4L2/MC
Use Cases
Use Cases - FBDEV
Use Cases - FBDEV
(that's it...)
Use Cases - V4L2
Video
Use Cases – DRM/KMS
Everything else
FB vs. DRM - sloccount
fbdev
0
500
1000
1500
2000
2500
2370
FB vs. DRM - sloccount
fbdev drm
0
500
1000
1500
2000
2500
2370
1659
DRM/KMS API
drm
kms
exynos
i810
i915
mga
nouveau
r128
radeon
savage
sis
via
KMS - Device Model
Frame Buffer
(memory)
CRTC Encoder Connector
Plane
(memory)
SoCMemory Off-Chip
Encoder Connector
KMS – Frame Buffer
CRTC
Frame Buffer
GEM
Object(s)
Memory
● width
● height
● format
● pitches
● offsets
Properties
DRM/KMS – GEM Object
CRTC
Frame Buffer
GEM Object
● width
● height
● bpp
● pitch
● size
Properties
Memory
DRM – Handles
Process A
Local
Handle
GEM
Object
Process B
Send FD
SCM_RIGHTS
Global
FD
1 2
Global
FD
4
Local
Handle
3
Active
Area
KMS – Modes (1/2)
sync
back porch
front porch
active area
Active
Area
KMS – Modes (2/2)
sync
back porch
front porch
active area
hdisplay
hsync_start
htotal
hsync_end
KMS – Mode Setting
crtc
fb
Active
Area
x
y
mode.hdisplay
mode.vdisplay
*connectors
num_connectors
mode
struct drm_mode_set...
Source: http://www.flickr.com/photos/buckaroobay/3721809183/
WIP – Display Framework
Common Display
Framework
http://lwn.net/Articles/512363/
Entity
Entity
Link Port
CDF - Entity Model
CDF - Device Tree
hdmi_encoder {
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
hdmi_input: endpoint@0 {
remote...
Display
Controller
Video
Entity
Pipeline Controller
controlcontrol
Entity Entity
CDF - Configuration Model
Display
Controller
Video
Entity
streaming control
streaming control
Entity Entity
CDF - Streaming Control
CDF - Integration
Display
Controller
Driver
Control Bus
API
FBDEV DRM
SPI/I2C Mem Map
Panel Driver
DT
pdata
DSI
• dri-devel@listsfreedesktop.org
• linux-fbdev@vger.kernel.org
• linux-media@vger.kernel.org
• laurent.pinchart@ideasonboa...
? !
merci.
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Kernel Recipes 2013 - Overview display in the Linux kernel
Upcoming SlideShare
Loading in...5
×

Kernel Recipes 2013 - Overview display in the Linux kernel

1,406

Published on

The Linux kernel has adapted in recent years to cope with the arrival of the embedded platforms using more and more advanced graphics capabilities. This presentation will explore the past challenges, but also the present and future, focusing on the current situation, the problems we face and the solutions that are currently proposed and considered.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,406
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
51
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Kernel Recipes 2013 - Overview display in the Linux kernel"

  1. 1. Display and Kernel Review and Future Kernel Recipes 2013 Paris Laurent Pinchart laurent.pinchart@ideasonboard.com
  2. 2. Problem Definition
  3. 3. display /
  4. 4. / graphics /
  5. 5. / video
  6. 6. Problem - Purpose display / graphics / video
  7. 7. format
  8. 8. memory / deep pipeline
  9. 9. device / CPU
  10. 10. Problem - Source format memory / deep pipeline device / CPU
  11. 11. rotation /
  12. 12. / scaling /
  13. 13. / composing
  14. 14. Problem - Processing rotation scaling composing
  15. 15. X11 /
  16. 16. / Wayland /
  17. 17. / MIR /
  18. 18. / SDL /
  19. 19. / DirectFB /
  20. 20. / Raw API
  21. 21. Problem - Stack X11 Wayland SDL DirectFB Raw API
  22. 22. DRM /
  23. 23. / FBDEV /
  24. 24. / V4L2
  25. 25. Problem - API DRM FBDEV V4L2
  26. 26. Display – Scanout Frame Buffer
  27. 27. Display – Composition CRTC Plane(s) Composition
  28. 28. Origins
  29. 29. Origins – FBDEV FBD EV,Linux 1.3.94 1996 2012 4C C Form ats 2000 Blanking
  30. 30. Origins – V4L2 V4L,Linux 2.2.0 1999 2002 2006 V4L2,Linux 2.4.0 V4L deprecated 2008 V4L2 subdev 2011 M edia controller videobuf2 2012 D M ABU F
  31. 31. Origins – DRM/KMS 2000 D R M ,Linux 2.2.18 2009 KM S,TTM 2008 G EM Planes,D M ABU F 2011 2012 D um b Buffers
  32. 32. Features
  33. 33. Memory Management DRM FB V4L2 Dynamic Allocation Yes No Yes Multiple Buffers Yes panning Yes Import dmabuf No userptr Export dmabuf mmap mmap mmap
  34. 34. Mode Setting DRM FB V4L2 Formats 4CC RGB 4CC 4CC Enumeration Planes No Yes Negotiation No No Yes Atomicity Yes No No
  35. 35. Transformations DRM FB V4L2 Overlays Yes No Yes Rotation Yes No Yes Scaling Yes No Yes Cropping/Panning Yes Yes Yes
  36. 36. Documentation api core drivers drm fb v4l2
  37. 37. DRM API Documentation The DRM core exports several interfaces to applications, generally intended to be used through corresponding libdrm wrapper functions. In addition, drivers export device-specific interfaces for use by userspace drivers & device-aware applications through ioctls and sysfs files. External interfaces include: memory mapping, context management, DMA operations, AGP management, vblank control, fence management, memory management, and output management. Cover generic ioctls and sysfs layout here. We only need high- level info, since man pages should cover the rest. “ „
  38. 38. Documentation api core drivers drm fb v4l2
  39. 39. Code Size api core drivers drm fb v4l2
  40. 40. Cumulative Changes - API 0 1000 2000 3000 4000 5000 6000 drm fb v4l2 drm fb v4l2
  41. 41. Cumulative Changes - Core 0 5000 10000 15000 20000 25000 30000 35000 40000 45000 50000 drm fb v4l2 drm fb v4l2
  42. 42. Cumulative Changes - Drivers 0 200000 400000 600000 800000 1000000 1200000 1400000 1600000 1800000 drm fb v4l2 drm fb v4l2
  43. 43. Mailing List Traffic 0 200 400 600 800 1000 1200 1400 1600 1800 drm fb v4l2 drm+bug
  44. 44. Device Model – FBDEV Frame Buffer (memory) Display Device
  45. 45. Device Model – DRM/KMS Frame Buffer (memory) CRTC Encoder Connector Planes (memory) Encoder Connector Connector
  46. 46. Device Model – V4L2 Buffers (memory) Display Device Connector
  47. 47. Device Model – V4L2/MC V4L2 subdev 0 1 2 V4L2 subdev 0 1 V4L2 queue 0
  48. 48. Device Model – V4L2/MC
  49. 49. Use Cases
  50. 50. Use Cases - FBDEV
  51. 51. Use Cases - FBDEV (that's it...)
  52. 52. Use Cases - V4L2 Video
  53. 53. Use Cases – DRM/KMS Everything else
  54. 54. FB vs. DRM - sloccount fbdev 0 500 1000 1500 2000 2500 2370
  55. 55. FB vs. DRM - sloccount fbdev drm 0 500 1000 1500 2000 2500 2370 1659
  56. 56. DRM/KMS API drm kms exynos i810 i915 mga nouveau r128 radeon savage sis via
  57. 57. KMS - Device Model Frame Buffer (memory) CRTC Encoder Connector Plane (memory) SoCMemory Off-Chip Encoder Connector
  58. 58. KMS – Frame Buffer CRTC Frame Buffer GEM Object(s) Memory ● width ● height ● format ● pitches ● offsets Properties
  59. 59. DRM/KMS – GEM Object CRTC Frame Buffer GEM Object ● width ● height ● bpp ● pitch ● size Properties Memory
  60. 60. DRM – Handles Process A Local Handle GEM Object Process B Send FD SCM_RIGHTS Global FD 1 2 Global FD 4 Local Handle 3
  61. 61. Active Area KMS – Modes (1/2) sync back porch front porch active area
  62. 62. Active Area KMS – Modes (2/2) sync back porch front porch active area hdisplay hsync_start htotal hsync_end
  63. 63. KMS – Mode Setting crtc fb Active Area x y mode.hdisplay mode.vdisplay *connectors num_connectors mode struct drm_mode_set { struct drm_framebuffer *fb; struct drm_crtc *crtc; struct drm_display_mode *mode; uint32_t x; uint32_t y; struct drm_connector **connectors; size_t num_connectors; };
  64. 64. Source: http://www.flickr.com/photos/buckaroobay/3721809183/
  65. 65. WIP – Display Framework Common Display Framework http://lwn.net/Articles/512363/
  66. 66. Entity Entity Link Port CDF - Entity Model
  67. 67. CDF - Device Tree hdmi_encoder { ports { #address-cells = <1>; #size-cells = <0>; port@0 { hdmi_input: endpoint@0 { remote = <&display_output>; }; }; port@1 { endpoint@0 { ... }; endpoint@1 { ... }; }; }; };
  68. 68. Display Controller Video Entity Pipeline Controller controlcontrol Entity Entity CDF - Configuration Model
  69. 69. Display Controller Video Entity streaming control streaming control Entity Entity CDF - Streaming Control
  70. 70. CDF - Integration Display Controller Driver Control Bus API FBDEV DRM SPI/I2C Mem Map Panel Driver DT pdata DSI
  71. 71. • dri-devel@listsfreedesktop.org • linux-fbdev@vger.kernel.org • linux-media@vger.kernel.org • laurent.pinchart@ideasonboard.com Contact
  72. 72. ? !
  73. 73. merci.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×