The document provides an overview of the ION memory allocator used in Android graphics to facilitate buffer sharing between hardware devices via DMA and discusses various related components like synchronization primitives and graphics modes. It highlights ION's capabilities, challenges, and the need for deterministic behavior in allocation, as well as differentiates it from other systems like DMA-BUF and CMA. Future plans involve addressing technical assumptions, improving interfaces, and enhancing synchronization methods to better manage graphics memory operations.

1. Android Graphics Upstreaming
Linaro Connect Europe 2013

2. 2
Overview
●
Covering
– ION
– Sync
– KMS/HWComposer
●
Hoping for active discussion

3. 3
Disclosure
I'm not a DMA expert, nor am I all that
familiar with details around graphics
I'm likely to be wrong in more then one
place

4. 4
ION

5. 5
What is the issue ION solves?
●
Provides a way to allocate buffers so that they can be
shared between different hardware devices (via DMA)
to avoid copying
●
Different devices have different constraints
– Physically contiguous memory
– Smaller memory aperture (32bit device accessing LPAE/64bit
memory)
– Different pagetable sizes
●
Provides a method to select type of buffer that satisfies
the constraints
●
While mostly used for graphics, ION is not graphics
specific

6. 6
????
Would contrived cartoon examples help?

7. 7

8. 8
CPU full virtual and physical addressing

9. 9
GPU supports full memory range + scatter/gather

10. 10
Camera is 32bit, and can only do DMA
to physically contiguous memory

11. 11
Crypto engine only supports 32bits,
but does support scatter/gather

12. 12
MMC supports full memory range,
but only contiguous physical memory

13. 13
Virtual allocation

14. 14
Resulting physical allocation

15. 15
kmalloc for physically contiguous allocation

16. 16
CMA allows kernel to make space for
contiguously physical allocations

17. 17
Carveout memory is physically contiguous
memory reserved at boot

18. 18
ION interface
●
Provides way for userland to allocate buffers
from various “pools of memory” (aka: heaps)
– SYSTEM: Virtually contiguous (vmalloc)
– SYSTEM_CONTIG: Small physically contiguous
(kmalloc)
– CARVEOUT: Large reserved physically contiguous
– CHUNK: Carveout + large page tables
– CUSTOM: Whatever hardware vendors want (ick)
– CMA: Sometime in the future?

19. 19
ION Interface (cont)
●
Allows freeing, mapping and passing of
those buffers to other applications and
drivers
– Buffers shared as file descriptors

20. 20
Using our examples
●
CPU + GPU: SYSTEM
●
CPU + MMC: SYSTEM_CONTIG
●
CPU + CAMERA: CARVEOUT
●
CPU + GPU + CAMERA: CARVEOUT
●
CPU + GPU + MMC: SYSTEM_CONTIG
●
Note: ION does not help calculate what the proper
heap is for the given combination of hardware. It just
provides userland an interface to specify a heap that
userland knows satisfies the hardware constraints

21. 21
ION developer priorities
●
Android developers very focused on avoiding “jank”
- frame drops, jerky animations
●
Want very deterministic behavior
– They worry about CMA since it may spend a variable
amount of time to move memory on a large allocation
– Delayed constraint-solving dma-buf allocation ideas are
similarly not considered viable (by Android devs)
●
Want to centralize as much logic as possible in ION
core, so any optimizations can be made once in the
core infrastructure
– Avoid lots of per-driver tweaking

22. 22
Isn't this what dma-buf does?
●
ION pre-dates dma-buf
●
dma-buf provides a subset of what ION does
●
dma-buf is more of a encapsulation structure for
buffers of different types
– Allows buffers to be passed between different drivers and
userland
– Basically a marshaling structure
– Does not specify how the buffers are allocated
●
ION also has its own buffer encapsulation structure
– ION added support to export dmabufs (sort of)

23. 23
Isn't this what CMA does?
●
Again: Sort of.
●
CMA allows for large physically contiguous memory allocations by
migrating memory to make room for the large allocation
●
Pros:
– Avoids wasting memory with carveouts if they aren't in use.
– CMA has pluggable allocators and options that can allow for allocations that
satisfy the constraints needed.
●
Cons:
– CMA is kernel-internal only for now, and doesn't have a interface to allow
userland to allocate buffers or specify constraint options
– Migrating pages to make room can cause non-deterministic delays. Android
developers want deterministic behavior.
●
Patches to support CMA via ION have been submitted by Benjamin
Gaignard (Android developer plan on accepting them).

24. 24
What about TTM, GEM and PRIME?
You are now in the acronym pit of despair!
DRM, DRI, DRI2, EXA, UXA, GEM, TTM, UMA, GTT

25. 25
What about TTM, GEM?
●
TTM: Graphics memory manager for discrete gpus that have
their own video-ram.
– Considered complicated / poorly documented
– Provides fence synchronization facility
●
GEM: More minimal approach to TTM
– Developed by Intel, focused on their hardware
– Limited to UMA devices (ie: integrated graphics)
– No synchronization (fence) primitives
●
Those have to be implemented w/ driver-specific ioctls
– Allows for sharing of buffers between applications by named ids
●
GEM-ified TTM: TTM backend w/ GEM API

26. 26
What about PRIME?
●
PRIME: GEM extended to use file
descriptors for passing object
references/buffers between drivers and
userland
– Uses dmabuf for passing buffers around
– Required for “hybrid graphics” where there
are multiple gpu (discrete and integrated)
working together.

27. 27
Issues with ION
●
Doesn't build on non 32-bit ARM architectures
●
Quite a bit of DMA api misuse
– Lots of ARM specific assumptions about DMA rules that aren't
generically portable
●
Exports kernel pointers to userland (makes compat_ioctl
support difficult)
●
Larger portability issue that applications have to
understand the hardware buffer constraints in order to
select the right heap to use
– On different hardware, different heaps may be available, as
well as different devices with different constraints
– Same userland wouldn't necessarily work on different hardware

28. 28
DMA-API Misuse
●
CPUs and Devices both cache memory
– To keep coherency, we need to flush caches
before initiating DMA
– This requires a direction and a device
●
ION pre sync's data, before knowing which
device its going to. Leaves device value as
NULL. Works for their uses
– Broken for IOMMUs

29. 29
What is our plan with ION?
●
Working w/ Android and ARM developers to address 32bit
ARM assumptions
●
Working with Arnd to try to sort out if we can address the
dma-api misuse, or decide if new dma-apis are needed
●
Try to come up with a way for the interface to expose less
hardware specific detail
– Query devices for an opaque heap-cookie they support, which
could be OR-ed with other cookies to determine which heap to use
for cross device buffers
●
All of this may break current interface compatibility :(
●
I suspect getting ION into staging is as good as it will get
●
Other ideas?

30. 30
Sync

31. 31
What is Sync?
●
Provides synchronization primitives that can be
shared across processes
●
Used mostly to synchronize both drivers and
applications drawing to the screen
●
Like a condition-wait variable, but can be backed by
hardware primitives
– Some gpus support hardware mutexes
●
Provides lots of debugging data for sorting out
synchronization issues
●
In staging directory as of 3.10

32. 32
Sync Interface
●
Timelines and fences
– Applications set fences at specific points on
timeline and wait
struct sw_sync_create_fence_data data;
data.value = fence_count
ioctl(timeline_fd, SW_SYNC_IOC_CREATE_FENCE, &data);
ioctl(data.fence, SYNC_IOC_WAIT, &timeout);
– Controlling thread increments timeline, waking
any processes waiting.
ioctl(timeline_fd, SW_SYNC_IOC_INC, &count);

33. 33
What about Dmabuf-fences?
●
Developed by Maarten Lankhorst, Daniel Vetter and
Rob Clark
●
Creates similar synchronization fences that are tied to
specific dma-buf buffers
●
Provides implicit synchronization
– Android's Sync is explicit synchronization, requiring
developers to add the logic
●
Limited to dma-buf buffers
– Android's Sync driver can be used in more varied contexts

34. 34
Daniel Vetter's take:
“The fundamental difference between android syncpoints and the dma_buf
fences is that syncpoints use explicit userspace synchronization objects which
get passed around as fds. Whereas dma_buf fences are all implicitly attached to
the respective dma_bufs, so userspace can just pass around the buffer object fds
and the kernel ensures that magic happens and everything is synced up properly.
Imo the later approach has two big upsides:
- Implicit sync objects are a _much_ simpler programming model. Think
synchronous file i/o vs. aio. And if the kernel doesn't suck, there's not really a
performance disadvantage, at least for the shared buffer use-case. GL drivers
might still need explicit syncing for their gpu state objects for the last ounce of
performance, but that's not relevant.
- Having fences attached directly to dma_buf objects is the only way to make
dynamic buffers (i.e. eviction from garts/memory) possible. Currently every
graphics driver on android seems to just pin their buffers into main memory so
there's no need for that. And ion also only cares about pinned buffers. But I
expect that this will change.”

35. 35
What about wait/wound-style mutexes?
●
Also developed by Maarten Lankhorst and Daniel Vetter
●
Developed to handle the case where buffers are shared
between devices. Since buffers may not be ordered in the same
way on all devices, there may be the possiblility for ABBA
deadlocks
●
Wait/wound style mutexes provide a global ticket (or context)
which orders acquisitions. If a deadlock occurs, the oldest ticket
holder waits for the mutex, while the younger holders have to
“back off” and drop the locks they hold.
●
Kernel driver interface only, not something userspace can use.
●
I suspect this to be a base for dmabuf-fences
●
Queued to be merged for 3.11

36. 36
What is our plan with Sync?
●
Try to stir discussion between community
and Android developers on explicit vs
implicit synchronization issues
●
Follow along to see if any part of the
implementations can be shared
●
Other ideas?

37. 37
KMS HWComposer

38. 38
What is KMS?
●
Kernel Mode Setting
●
Makes the kernel responsible for graphics
mode (resolution, refresh, orientation)
– Avoids races with userland and hardware
– Can switch modes on OOPs to display message

39. 39
What is HWComposer?
●
Per-platform userspace code that manages
composition acceleration
●
Part of the HAL layer
●
Currently using fb
●
Would be nice to convert HWComposer to
KMS

40. 40
What is our plan with
KMS/HWC/HAL?
●
Android devs likely already working on KMS enabled HAL
– Likely to be optimized specifically for next hardware release
– Not likely to be generic KMS HAL
●
Areas that may need work:
– Sync and vsync notifications with KMS
●
Hopefully this resolves the pageflipping framebuffer issue?
– Gralloc allocates 2x y_res
– Most fb drivers don't support this
●
Other thoughts/ideas?