Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
John Stultz <john.stultz@linaro.org>
LMG Kernel Team
AOSP Devboard Update &
Recent/Future Pain Points
ENGINEERS AND DEVICES
WORKING TOGETHER
Now there are two: https://source.android.com/source/devices
HiKey HiKey960
ENGINEERS AND DEVICES
WORKING TOGETHER
Hardware overview
HiKey
● HiSilicon Kirin 620 SoC
● 64bit ARM A53
● 8 cores
● 1 or ...
ENGINEERS AND DEVICES
WORKING TOGETHER
Remaining out-of-tree patches (4.14-rc)
HiKey
● Minor USB improvements
● ION ABI re...
ENGINEERS AND DEVICES
WORKING TOGETHER
96Boards.org
Vendor neutral hardware standard for developer boards
Allows ecosystem...
ENGINEERS AND DEVICES
WORKING TOGETHER
96Boards.org
Vendor neutral hardware standard for developer boards
Allows ecosystem...
ENGINEERS AND DEVICES
WORKING TOGETHER
Neonkey
Neonkey is certified 96Boards mezzanine: https://source.android.com/source/...
Why this matters
ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4...
ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4...
ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4...
ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4...
ENGINEERS AND DEVICES
WORKING TOGETHER
v3.10 Released
June 2013
2015 Nexus devices
(Oct 2015)
v3.18 Released
Dec 2014
v4.4...
ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel/Device Release Latencies
2 years, 4 months
1 y...
ENGINEERS AND DEVICES
WORKING TOGETHER
SoC Android Trees (circa ~2015)
HiKey 4.9 Kernel Tree (~July)
HiKey 4.9 Kernel Tree (~July)
ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel/Device Release Latencies
2 years, 4 months
1 y...
ENGINEERS AND DEVICES
WORKING TOGETHER
2015 Nexus devices
(Oct 2015)
Kernel Release Latencies
2 years, 4 months
1 year, 10...
HiKey 4.9 Kernel Tree (after HiKey960)
Benefits
ENGINEERS
AND DEVICES
WORKING
TOGETHER
Benefits to Android Devs
● Inexpensive Android development board
● Amount of board-...
ENGINEERS
AND DEVICES
WORKING
TOGETHER
Benefits to Android Devs (cont)
● Development of new Bootloader features
○ UEFI boo...
ENGINEERS AND DEVICES
WORKING TOGETHER
Upstream Testing
Have a AOSP target that can run mainline kernels.
● LKFT efforts
●...
Recent & Future Efforts
ENGINEERS AND DEVICES
WORKING TOGETHER
Recent efforts
Bring-up and validation of android-4.9 kernel tree
HiKey960 migratio...
ENGINEERS AND DEVICES
WORKING TOGETHER
Cadence HiFi DSP SDK for Android
• A development and demonstration platform for Aud...
ENGINEERS AND DEVICES
WORKING TOGETHER
List of Demonstrations
• App processor to HiFi 3 DSP IPC (Inter processor communica...
ENGINEERS AND DEVICES
WORKING TOGETHER
Direction forward
Utilize HiKey as arm64 target for lots of upstream and LTS testin...
ENGINEERS AND DEVICES
WORKING TOGETHER
Generic Build
● Kconfig based
● Supports multiple devices/architectures from single...
Pain Points
ENGINEERS AND DEVICES
WORKING TOGETHER
AOSP pain points
Devboards aren’t really mobile devices (no panel, touch, baseband,...
ENGINEERS AND DEVICES
WORKING TOGETHER
Community pain points
Support multiple older kernels (v3.18, v4.4, v4.9, v4.14)
Dif...
ENGINEERS AND DEVICES
WORKING TOGETHER
Vendor pain points
Treble trouble
● Wifi / Bluetooth / gralloc all needed work
● Bl...
ENGINEERS AND DEVICES
WORKING TOGETHER
Things to watch out for...
ION ABI (4.12+) changes will affect lots of vendors with...
ENGINEERS AND DEVICES
WORKING TOGETHER
Questions/Discussion
Price vs Power preference?
Upstreaming board support takes a f...
Thank You
#SFO17
Questions: John Stultz <john.stultz@linaro.org>
SFO17 keynotes and videos on: connect.linaro.org
For furt...
Backup Slides
ENGINEERS AND DEVICES
WORKING TOGETHER
Hardware Architecture of Cadence HiFi 3 DSP in Hikey960
App Processor
Audio DSP (Hi...
ENGINEERS AND DEVICES
WORKING TOGETHER
DSP Software Architecture of the Hikey960
Kernel
App Processor
HiFi 3 DSP
XAF
Xtens...
Upcoming SlideShare
Loading in …5
×

AOSP Devboard update and recent/future pain points - SFO17-506

1,272 views

Published on

Session ID: SFO17-506
Session Name: AOSP Devboard update and recent/future pain points - SFO17-506
Speaker:
Track: LMG


★ Session Summary ★
Cover an outline of the AOSP Devboard effort with HiKey and now HiKey960, work on upstreaming, generic HAL implementations for Linux common hardware interfaces, graphics efforts, along with issues coming from the transition to Treble, and from upstream kernel work colliding with legacy Android kernel interfaces.

This presentation will be similar to one given earlier at Linux Plumbers but will go a bit deeper on issues and will provide some context to the discussion had at Linux Plumbers.
---------------------------------------------------
★ Resources ★
Event Page: http://connect.linaro.org/resource/sfo17/sfo17-506/
Presentation:
Video:
---------------------------------------------------

★ Event Details ★
Linaro Connect San Francisco 2017 (SFO17)
25-29 September 2017
Hyatt Regency San Francisco Airport

---------------------------------------------------
Keyword:
http://www.linaro.org
http://connect.linaro.org
---------------------------------------------------
Follow us on Social Media
https://www.facebook.com/LinaroOrg
https://twitter.com/linaroorg
https://www.youtube.com/user/linaroorg?sub_confirmation=1
https://www.linkedin.com/company/1026961

Published in: Technology
  • Be the first to comment

AOSP Devboard update and recent/future pain points - SFO17-506

  1. 1. John Stultz <john.stultz@linaro.org> LMG Kernel Team AOSP Devboard Update & Recent/Future Pain Points
  2. 2. ENGINEERS AND DEVICES WORKING TOGETHER Now there are two: https://source.android.com/source/devices HiKey HiKey960
  3. 3. ENGINEERS AND DEVICES WORKING TOGETHER Hardware overview HiKey ● HiSilicon Kirin 620 SoC ● 64bit ARM A53 ● 8 cores ● 1 or 2 GB RAM ● 4 or 8 GB eMMC ● Mali 450 (Utgard) Graphics ● USB2 Type A & USB-OTG ● $100-120 HiKey960 ● HiSilicon Kirin 960 SoC ● 64bit ARM 4xA53 + 4xA73 ● 8 core big.LITTLE design ● 3 GB RAM ● 32 GB UFS Storage ● Mali G71 (Bifrost) Graphics ● PCIe ● USB3 Type A & USB-C (USB2) ● 5GHz WiFi & BT 4.1 ● $240
  4. 4. ENGINEERS AND DEVICES WORKING TOGETHER Remaining out-of-tree patches (4.14-rc) HiKey ● Minor USB improvements ● ION ABI revert HACK ● Mali gpu HiKey960 ● regulators ● ufs ● mailbox ● cpufreq/thermal ● usb ● iommu ● drm driver ● dts ● Mali gpu ● ...
  5. 5. ENGINEERS AND DEVICES WORKING TOGETHER 96Boards.org Vendor neutral hardware standard for developer boards Allows ecosystem of peripherals that can be shared
  6. 6. ENGINEERS AND DEVICES WORKING TOGETHER 96Boards.org Vendor neutral hardware standard for developer boards Allows ecosystem of peripherals that can be shared
  7. 7. ENGINEERS AND DEVICES WORKING TOGETHER Neonkey Neonkey is certified 96Boards mezzanine: https://source.android.com/source/devices#neonkey ● CPU: STM32F11CE ● NOR Flash: 512 KB ● SRAM: 128 KB ● Pressure sensor: BMP280 ● ARM Hall sensor: MRMS501A ● Temp/Humidity sensor: SI7034-A10 ● ALS/Proximity sensor: RPR-0521RS ● LED driver with 15LEDs: LP3943 ● Accel/Gyro+Geomagnetic sensors: ○ BMI160+BMM150 ● 4 GPIO-driver LEDs ● I2C expansion ● GPIO (2 lines) expansion ● JTAG connector ● 96Boards LS Expansion connector
  8. 8. Why this matters
  9. 9. ENGINEERS AND DEVICES WORKING TOGETHER v3.10 Released June 2013 2015 Nexus devices (Oct 2015) v3.18 Released Dec 2014 v4.4 Released Jan 2016 New Phones, Old Kernels 2 years, 4 months v4.9 Released Dec 2016
  10. 10. ENGINEERS AND DEVICES WORKING TOGETHER v3.10 Released June 2013 2015 Nexus devices (Oct 2015) v3.18 Released Dec 2014 v4.4 Released Jan 2016 New Phones, Old Kernels 2 years, 4 months v4.9 Released Dec 2016 2016 Pixel Phone (Oct 2016) 1 year, 10 m onths
  11. 11. ENGINEERS AND DEVICES WORKING TOGETHER v3.10 Released June 2013 2015 Nexus devices (Oct 2015) v3.18 Released Dec 2014 v4.4 Released Jan 2016 New Phones, Old Kernels 2 years, 4 months 1 year, 3 months v4.9 Released Dec 2016 2016 Pixel Phone (Oct 2016) 1 year, 10 m onths Early 2016 Flagship (Mar 2016) 1 year, 4 months Early 2017 Flagship (April 2017)
  12. 12. ENGINEERS AND DEVICES WORKING TOGETHER v3.10 Released June 2013 2015 Nexus devices (Oct 2015) v3.18 Released Dec 2014 v4.4 Released Jan 2016 A Year Ahead 2 years, 4 months 1 year, 3 months v4.9 Released Dec 2016 2016 Pixel Phone (Oct 2016) 1 year, 10 m onths Early 2016 Flagship (Mar 2016) 1 year, 4 months Early 2017 Flagship (April 2017) HiKey on 4.4 (Apr 2016)
  13. 13. ENGINEERS AND DEVICES WORKING TOGETHER v3.10 Released June 2013 2015 Nexus devices (Oct 2015) v3.18 Released Dec 2014 v4.4 Released Jan 2016 A Year Ahead 2 years, 4 months 1 year, 3 months v4.9 Released Dec 2016 2016 Pixel Phone (Oct 2016) 1 year, 10 m onths Early 2016 Flagship (Mar 2016) 1 year, 4 months Early 2017 Flagship (April 2017) HiKey on 4.4 (Apr 2016) HiKey on 4.9 (Feb 2017)
  14. 14. ENGINEERS AND DEVICES WORKING TOGETHER 2015 Nexus devices (Oct 2015) Kernel/Device Release Latencies 2 years, 4 months 1 year, 10 months 2016 Pixel Phone (Oct 2016) Early 2016 Flagship (Mar 2016) 1 year, 3 months Early 2017 Flagship (April 2017) 1 year, 4 months 2014 2015 2016 2017 HiKey on 4.4 (April 2016) 3m 2m v3.10 (Jun 2013) v3.18 (Dec 2014) v4.4 (Jan 2016) v4.9 (Dec 2016) HiKey on 4.9 (Feb 2017)
  15. 15. ENGINEERS AND DEVICES WORKING TOGETHER SoC Android Trees (circa ~2015)
  16. 16. HiKey 4.9 Kernel Tree (~July)
  17. 17. HiKey 4.9 Kernel Tree (~July)
  18. 18. ENGINEERS AND DEVICES WORKING TOGETHER 2015 Nexus devices (Oct 2015) Kernel/Device Release Latencies 2 years, 4 months 1 year, 10 months 2016 Pixel Phone (Oct 2016) Early 2016 Flagship (Mar 2016) 1 year, 3 months Early 2017 Flagship (April 2017) 1 year, 4 months 2014 2015 2016 2017 HiKey on 4.4 (April 2016) 3m 2m v3.10 (Jun 2013) v3.18 (Dec 2014) v4.4 (Jan 2016) v4.9 (Dec 2016) HiKey on 4.9 (Feb 2017)
  19. 19. ENGINEERS AND DEVICES WORKING TOGETHER 2015 Nexus devices (Oct 2015) Kernel Release Latencies 2 years, 4 months 1 year, 10 months 2016 Pixel Phone (Oct 2016) Early 2016 Flagship (Mar 2016) 1 year, 3 months Early 2017 Flagship (April 2017) 1 year, 4 months 2014 2015 2016 2017 HiKey on 4.4 (April 2016) HiKey960 on 4.4 (April 2017) 3m 2m v3.10 (Jun 2013) v3.18 (Dec 2014) v4.4 (Jan 2016) v4.9 (Dec 2016) HiKey on 4.9 (Feb 2017) HiKey960 on 4.9 (Aug 2017) 1 year, 4 months 8 months
  20. 20. HiKey 4.9 Kernel Tree (after HiKey960)
  21. 21. Benefits
  22. 22. ENGINEERS AND DEVICES WORKING TOGETHER Benefits to Android Devs ● Inexpensive Android development board ● Amount of board-specific patches is small ● Helps development of new Kernel features ○ Android Verity ○ Hibernation ○ Sdcardfs ○ SELinux ● Validation of Kernel patches / -stable updates ○ Wide range of supported kernels ○ Example: 3.18, (4.1), 4.4, 4.9 ● Development of Android runtime libraries ○ Wide range of supported kernels
  23. 23. ENGINEERS AND DEVICES WORKING TOGETHER Benefits to Android Devs (cont) ● Development of new Bootloader features ○ UEFI bootloader is provided in source ○ Boot to RAM ● Development and Support of new Hardware modules ○ Display boards ○ Camera modules ○ Sensor modules ( ex: Neonkey sensorhub) ○ Connectivity modules ● Testing ○ Manual ○ Automation
  24. 24. ENGINEERS AND DEVICES WORKING TOGETHER Upstream Testing Have a AOSP target that can run mainline kernels. ● LKFT efforts ● Weekly manual testing every mainline -rc release w/ HiKey ● Manually testing mainline ~daily during merge window ○ Small enough patchset that rebase-bisection isn’t bad Found and reported numerous regressions against upstream (SELinux, drm, binder, ipv6 networking, etc) Also reported a number of issues that AOSP needs to address to work with upstream kernels (New SELinux policies, dma_buf fences, ION destaging ABI break)
  25. 25. Recent & Future Efforts
  26. 26. ENGINEERS AND DEVICES WORKING TOGETHER Recent efforts Bring-up and validation of android-4.9 kernel tree HiKey960 migration to v4.9 Migration to Android O Transition to Generic Linux Bluetooth HAL (hikey & hikey960) Parameterized EAS PowerHAL to make it shared/reusable Gralloc binderization & migration to opensource gralloc on hikey960 Migration to new mali driver, which works w/ dma-buf fence SYNC_FILE interface In Progress: libION/Gralloc support for old and new (4.12+) ION ABIs Hibernation proof-of-concept support on HiKey Cadence HIFI3 DSP enablement
  27. 27. ENGINEERS AND DEVICES WORKING TOGETHER Cadence HiFi DSP SDK for Android • A development and demonstration platform for Audio and Speech Applications • Target Applications – Audio and speech codecs – Voice pre-processing such as acoustic echo cancellation, noise reduction – Multi-microphone processing – Audio post processing including object audio processing and rendering – Immersive audio, 3D audio, AR/VR – Speech recognition – Neural network based processing
  28. 28. ENGINEERS AND DEVICES WORKING TOGETHER List of Demonstrations • App processor to HiFi 3 DSP IPC (Inter processor communication) • PCM Playback to USB Speaker via HiFi 3 DSP • PCM Record from USB Microphone via HiFi 3 DSP • PCM Playback to HDMI Speaker via HiFi 3 DSP • MP3 Decoder on HiFI 3 DSP 09-29-2017 AOSP App processor Code Release AOSP DSP Code Release 11-10-2017 Cadence HiFi DSP SDK Launch TBD Multi-Microphone support TBD Linaro Connect Demo 10-13-2017 Roadmap
  29. 29. ENGINEERS AND DEVICES WORKING TOGETHER Direction forward Utilize HiKey as arm64 target for lots of upstream and LTS testing HiKey960 upstreaming android-4.14 bringup and validation UEFI transition(hikey960), updates & improvements (A/B, ?) drm_hwcomposer Proving ground for upstream EAS Improved treble implementation HiKey960 accelerated video encoding/decoding integration Upstream DT fragments/overlay solution
  30. 30. ENGINEERS AND DEVICES WORKING TOGETHER Generic Build ● Kconfig based ● Supports multiple devices/architectures from single build system ● db410c, db820c, nexus7, qemu-x86_64/aarch64, raspi3... ● Need more community involvement Work on open-graphics stack (mesa3d, gbm_gralloc, drm_hwcomposer, libdrm) Other Work
  31. 31. Pain Points
  32. 32. ENGINEERS AND DEVICES WORKING TOGETHER AOSP pain points Devboards aren’t really mobile devices (no panel, touch, baseband, etc) Devboards aren’t “product” level targets internally, so AOSP changes break them fairly often. Support Effort/Load (bugs/feature requests): ● HiKey: ~240 over 2 years ● HiKey960: ~ 60 over 0.5 years ● ~1 issue every 3 days Mali blobs make us dependent on ARM for continued support of hardware No real space for developing shared/common HALs (cross-project dependencies avoided) Very difficult process to add/update project trees
  33. 33. ENGINEERS AND DEVICES WORKING TOGETHER Community pain points Support multiple older kernels (v3.18, v4.4, v4.9, v4.14) Difficult to develop clean solutions that cross all those points. Example: drm_hwcomposer HWC2 implementation needs changes in v4.10+ Still need something for v3.18, v4.4 kernels Android Sync/dma-buf fence transition, ION ABI changes, etc have been difficult Would be nice if upstream efforts provided better migration paths
  34. 34. ENGINEERS AND DEVICES WORKING TOGETHER Vendor pain points Treble trouble ● Wifi / Bluetooth / gralloc all needed work ● Bluetooth was broken for 3 weeks during the transition ○ In part motivated the generic linux bluetooth HAL work ● Gralloc changes required changes into libGLES_mali.so ○ Required ARM to respin We want to provide a good reference implementation, so feedback on that would be good.
  35. 35. ENGINEERS AND DEVICES WORKING TOGETHER Things to watch out for... ION ABI (4.12+) changes will affect lots of vendors with custom implementations ● We really need to solve this right, but so far low involvement from vendors in community discussions! ● GET INVOLVED! Start aligning with open graphics stack! ● Work with community to resolve issues that keep you on custom implementations
  36. 36. ENGINEERS AND DEVICES WORKING TOGETHER Questions/Discussion Price vs Power preference? Upstreaming board support takes a frustratingly long time! Multiple boards (and supporting multiple kernels) is hard to scale! Still not getting to the point where we’re really pushing some of the advanced Android functionality upstream (offloaded sensor collection, wifi/bt scanning, etc). Standard Kernel interfaces for Android are becoming a smaller issue than standard bootloader interfaces for Android (Reboot reason, dtb handling, fastboot flashing, secure boot, A/B updates, etc). Some work going on in UEFI, but with limited resources.
  37. 37. Thank You #SFO17 Questions: John Stultz <john.stultz@linaro.org> SFO17 keynotes and videos on: connect.linaro.org For further information: www.linaro.org
  38. 38. Backup Slides
  39. 39. ENGINEERS AND DEVICES WORKING TOGETHER Hardware Architecture of Cadence HiFi 3 DSP in Hikey960 App Processor Audio DSP (HiFi 3) Hi3660 SoC LPDDR4 Shared Memory HiFi 3 DSP Core (500 MHz) DRAM IRAM D-Cache I-Cache SoC I/O
  40. 40. ENGINEERS AND DEVICES WORKING TOGETHER DSP Software Architecture of the Hikey960 Kernel App Processor HiFi 3 DSP XAF Xtensa Audio framework Audio HALUSB HAL Host Side XAF Audio Signal Processing component DSP Side XAF Your App SW IPC Inter-processor connect (Shared Memory) DSP Developers API (enables direct control of components) HiFi Driver App Developers API (abstracts DSP with few APIs)

×