Resource: LCA14 Name: LCA14-416: Big Endian - where are we? Date: 06-03-2014 Speaker: Victor Kamensky

1. Thu 6 March, 4:10pm, Victor Kamensky
LCA14-416: Big Endian - Where are
we?

2. • Mandatory read:
Big and Little Endian Inside Out – Ben Herrenschmidt, LPC12
• ARM CPU architecture is endian agnostic - can do either
• ARM Linux ecosystem traditionally is Little Endian
• Surge of interest in ARM Big Endian support
What is Big and Little Endian?

3. • Very big, 100M+ LOC, legacy code, Big Endian only
• Runs on MIPS and PPC CPUs, looking forward to ARM
CPU adoption
• Not because Big Endian is better for networking
applications
Why is Big Endian?

4. • Last few years ARM Linux runs in Little Endian mode
• XScale CPU runs in BE32 mode
• Big Endian BE32 ecosystem for XScale CPUs still exists
in distros like OpenWRT
• Some vendors offered very minimal V7 BE BSP support
ARM Big Endian Linux History

5. • Build reasonable, supported, Linux Big Endian
ecosystem for ARM V7 and V8 CPUs. Support as
“second class citizen” is good enough
• Mixed mode support where Little Endian and Big Endian
code could coexist on the same box
• Tools and methodologies that could help transition Big
Endian Only code to Endian Agnostic
LNG Endian Team was Set to Do

6. • Truly great open source effort
• Effort got help from many companies and individuals
• Few to mention, but not limited:
• Codethink
• ARM
• Linaro
• Thank you!
We Got Help … a Lot!

7. • V7 ARM BE8 gcc support existed for a while
• V8 ARM BE gcc support was verified/fixed by Linaro
toolchain group
• Linaro toolchain releases for gcc 4.8 include Big Endian
variants:
• V7 - armeb
• V8 - aarch64_be
• Some issues may remain for Neon vector intrinsics
ARM V7 & V8 Big Endian Toolchain

8. • Bootloader, u-boot or uefi, remains Little Endian
• All typical SoCs peripheral devices are Little Endian
ARM Big Endian Kernel Context

9. • BE8 ARM V7 core support was driven by Ben Dooks
from Codethink
• LNG Endian team collaborated with Ben: testing, fixes,
review
• BE8 V7 core support series appeared in 3.13
ARM V7 Big Endian Kernel Core Support

10. • Some just worked … with very minor changes
• TC2
• Mvebu
• Highbank
• LNG Endian team use TI Pandaboard and Samsung
Arndale as reference and development platform for Big
Endian work (a lot of regular driver fixes)
• TI Pandaboard changes are in process of being upstreamed
• Samsung Arndale changes no intention to upstream
• LNG Endian team helped, in different degrees, to enable
Big Endian support on
• TI Keystone
• LSI Amarillo
ARM V7 Big Endian Kernel BSP

11. • Supported through Linaro OE
• Changes are in meta-linaro/meta-bigendian
• Very few user-land packages are affected
• Few examples:
• strace
• python libffi
ARM V7 Big Endian User-land

12. • LNG Endian team tested and fixed different kernel
functional areas which could be affected by endianity
change
• Just to mention few:
• BE kexec/kdump - minor fixes
• BE kprobes - upstream pull request posted
• BE uprobes - folded into regular Linaro uprobes work
• BE KVM host support - patches posted for upstream review
ARM V7 Big Endian Kernel Advanced
Features

13. • Done by ARM folks: Matthew Leach, Will Deacon
• Was very nice surprise
• Included in 3.13
• Works/tested on fastmodels and real V8 h/w
ARM V8 Big Endian Kernel Core Support

14. • Big Endian V8 KVM host support patches posted for
upstream review
• We will test other advanced features on V8 as soon as
they are available on V8
ARM V8 Kernel Big Endian Advanced
Features

15. • Supported through Linaro OE
• aarch64_be target support was added to
openembedded-core with help of Linaro release team
• Ran into few minor issues
ARM V8 Big Endian User-land

16. • Java
• Valgrind
• Gcc neon vector intrinsics
Anything else? What did we miss?
Known Big Endian ARM Linux Features
that don’t Work

17. • “Special” Big Endian user-land process on top of Little
Endian Kernel along side with regular Little Endian
processes
• Byte-swaps on system call and ioctl level in kernel
• Cisco conducted some experiments in this area:
• converted around 30 system calls and 20 ioctls
• messy, big, intrusive change
• non-trivial code in futex area
• Byte-swaps before calling Little endian function:
• did not actually try
• but seems even more harder
• i.e how to debug process that may have Big Endian and Little Endian
code coexisting
Mixed Mode: Big and Little Endian

18. • Done by Marc Zyngier
• Looks nice and clean
• See Crossing the endianness bridge - Marc Zyngier KVM Forum '13
• Marc showed Big Endian guest running on top of Little
Endian KVM host
• Require changes in emulator - patches for kvmtool by
Marc are available
• Guest need to treat virtio config as LE; not committed
yet; may need to wait for virtio-2 support
• Once Big Endian KVM support appeared it could run
Little Endian guests (tested on V7 and V8)
• If we have time will make a demo of it
Mixed Mode: through KVM

19. • Cisco runs big project in this area, attempting to convert
100M+ Big Endian only LOC into Endian agnostic code
• Effort is much larger compared to anything LNG Endian
team did
• Approach outline:
• sparse with a lot of changes used to annotated code and detect
endian issues
• sparse tool was changed to do source to source transformation
• python plugin added that allows write external rules how to transform
code
• Considered very risky and big, may take long time
• Cisco most likely will be able to share and present
details about the effort by next Linaro Connect (tentative
agreement reached)
Big Endian to Endian Agnostic Tools

20. • ARM V7 and V8 Big Endian basic ecosystem looks good
• Mixed mode support should be addressed through KVM
virtualization
• Big Endian porting mainly will be based on Cisco internal
project experience
Summary

21. More about Linaro Connect: http://connect.linaro.org
More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
Linaro members: www.linaro.org/members