Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau
•
0 likes•681 views
Abstract I will describe the work I did on the open source LatticeMico32 softcore CPU to add a MMU to its pipeline and then how I added support for this enhanced LM32 CPU in the NetBSD <http: /> 6 kernel. I will quickly explain what a MMU is and how it works in LM32. I will then show an overview of the steps I followed to add support for this new CPU and this new System-on-Chip to the NetBSD kernel. Afterward I will explain some of the choices made for this port, especially those in relation with the MMU handling : the (machine-dependant) virtual memory system (aka pmap). I will demo the boot of the NetBSD kernel on QEMU emulating LM32 CPU and then on the Milkymist One VJ Station. Speaker bio Yann Sionneau is a 26 year-old French embedded software engineer passionate about learning how embedded systems work in general. Yann is part of the M-Labs <http: /> (fka Milkymist) community that is working on developing open source digital designs on FPGAs as well as making it more and more easy to do so by providing a simple but yet powerful framework for System-on-Chip design. Yann contributed the original RTEMS Board Support Package of the Milkymist One video synthesizer, a Memory Management Unit (MMU) for the Open Source soft-core CPU LatticeMico32, and ported NetBSD kernel for the LM32 CPU and the Milkymist System-on-Chip. Yann recently became an EdgeBSD developer and his work on LM32 support is currently upstream in an EdgeBSD branch.
Recommended
More Related Content
Similar to Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau
Similar to Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau (20)
More from eurobsdcon
More from eurobsdcon (20)
Recently uploaded
Recently uploaded (20)
Porting NetBSD to the LatticeMico32 open source CPU by Yann Sionneau
- 1. Por$ng NetBSD on the open source La5ceMico32 CPU Yann Sionneau M-‐Labs @ EuroBSDCon 2014
- 2. About me • Yann Sionneau • Embedded soGware developer • Working for M-‐Labs – hOp://m-‐labs.hk • @yannsionneau on twiOer • Email: ys@m-‐labs.hk
- 3. I’m going to talk about… My por$ng of NetBSD and EdgeBSD on the Milkymist One board
- 4. Agenda • I) The hardware part: the MMU – How La5ceMico32 MMU works • II) The soGware part – My journey por$ng NetBSD to a new CPU
- 8. The Milkymist One uses an FPGA
- 9. What’s an FPGA?? • A chip
- 10. FPGA internals
- 12. La5ceMico32 CPU • 32 bits Harvard Architecture RISC • Big Endian • 6 stages • Fully bypassed • Op$onal configurable I/D caches – Direct mapped or – 2-‐way set associa$ve • Wishbone on-‐chip bus
- 13. Works on … • Everything actually : device agnos$c
- 14. DE0-‐Nano
- 15. Milkymist One
- 16. Papilio Pro
- 17. Mixxeo
- 18. KC705
- 19. La5ceMico32 , Good points • Small • Portable (works with several FPGA vendors) • Fast (~100 MHz on Slowtanpartan 6, 125 MHz on Kintex-‐7, up to 200 MHz without DDR3) • Actually works • GCC/Binu$ls/GDB/Qemu/uCLinux/OpenWRT support • OPEN SOURCE
- 20. La5ceMico32, Bad points • No Memory Management Unit… yet!
- 21. La5ceMico32, Bad points • No Memory Management Unit… yet! Done þ
- 22. Used in… • Closed source commercial ASICs • Open source projects • Can achieve 800 MHz in TSMC 90nm standard cell process
- 26. Main Memory CPU Internal Before PHYSICAL ADDRESS PHYSICAL ADDRESS PA PA
- 27. Main Memory CPU Internal Raising excep$on AGer VIRTUAL ADDRESSES PHYSICAL ADDRESSES
- 28. What’s the MMU’s job? • Translate « virtual addresses » into « physical addresses » • Memory protecGon against unwanted execu$on of code or data write (e.g. soGware bug or security issue) – Memory right access management
- 29. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address
- 30. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address How does the MMU know the VA-‐>PA transla$on ?
- 31. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address Page Table
- 32. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address Page Table Why « PAGE »?
- 33. Why « Page »? • 0x00000004 -‐> 0x10000000 • 0x00000005 -‐> 0x10000001 • 0x00000006 -‐> 0x10000002 Etc…
- 34. Why « Page »? • 0x00000004 -‐> 0x10000000 • 0x00000005 -‐> 0x10000001 • 0x00000006 -‐> 0x10000002 Etc… This is WRONG!!!
- 35. Why « Page »? • 0x00000*** -‐> 0x10000*** • 0x00001*** -‐> 0x10001*** • 0x00002*** -‐> 0x10002*** Etc…
- 36. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address Page Table
- 37. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address Page Table TLB TLB : Transla$on Lookaside Buffer
- 38. Main Memory CPU pipeline VA PA VA : Virtual Address PA : Physical Address Page Table TLB OperaGng System Updates the Gets informa$on from the Updates the
- 39. Features? • Page size – Only 4 kB 32 bits physical address : xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx How many bits of an address indicate the offset within a given page?
- 40. Features? • Page size – Only 4 kB 32 bits physical address : xxxxxxxx xxxxxxxx xxxx xxxx xxxxxxxx Page number [31:12] 20 bits Offset [11:0] 12 bits
- 41. Features? • 2 TLB (Transla$on Lookaside Buffer) – ITLB – DTLB • Each TLB contains 1024 entries – How many bits needed to index the TLB?
- 42. Features? • 2 TLB (Transla$on Lookaside Buffer) – ITLB – DTLB • Each TLB contains 1024 entries – How many bits needed to index the TLB? 10 bits!
- 43. Features? • No hardware page-‐tree walker – i.e. TLB is soGware assisted
- 44. Virtual address Load or store? Instruc$on or Data? Physical address Access granted/denied
- 45. Virtual address Load or store? Instruc$on or Data? Physical address Access granted/denied I don’t know!
- 46. Let’s have a look inside
- 47. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001004
- 48. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page number Offset in the page
- 49. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4
- 50. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001
- 51. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 VPN = 0xA0001 à 1010 0000 0000 0000 0001
- 52. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1 VPN = 0xA0001 à 1010 0000 00 00 0000 0001 TLB index, used to select a TLB line
- 53. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1 VPN = 0xA0001 à 1010 0000 00 00 0000 0001 TLB index, used to select a TLB line
- 54. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1 VPN = 0xA0001 à 1010 0000 00 00 0000 0001 Tag = 0x280 à 1010 0000 00 =
- 55. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1 VPN = 0xA0001 à 1010 0000 00 00 0000 0001 Tag = 0x280 à 1010 0000 00 = ⇒ Physical page number = 0xB0001
- 56. Line index Tag [10] Physical page number [20] Read-‐only [1] Valid [1] 0 0xABC 0xABC00 0 0 1 0x280 0xB0001 1 1 2 0x300 0x00001 0 1 The TLB VA = 0xA0001 004 Page offset = 4 Virtual Page number = 0xA0001 TLB index = 1 VPN = 0xA0001 à 1010 0000 00 00 0000 0001 Tag = 0x280 à 1010 0000 00 = ⇒ Physical page number = 0xB0001 ⇒ Physical Address = 0xB0001004
- 57. Por$ng NetBSD • 1°) NetBSD cross compila$on toolchain – build.sh – Makefiles here and there – Arch-‐specific directories Allows to do: $ ./build.sh -‐U -‐m milkymist tools
- 58. Por$ng NetBSD • 2°) Support for built-‐ins in libkern – NetBSD kernel is • Not linked against libgcc • Linked against libkern – Need to implement basic arithme$c func$ons emiOed by gcc in object code – Implementa$on in sys/lib/libkern/arch/lm32
- 59. Por$ng NetBSD • 3°) Building my first kernel – Create sys/arch/lm32 and sys/arch/milkymist – Populate • sys/arch/<cpu|soc>/include • sys/arch/<cpu|soc>/conf – Stub, stub, stub… Allows to do: $ ./build.sh -‐m milkymist -‐U kernel=GENERIC
- 60. Por$ng NetBSD • 4°) Write basic console driver for early prints struct consdev milkymist_com_cons = { […] milkymist_com_cngetc, /* cn_getc: kernel getchar interface */ milkymist_com_cnputc, /* cn_putc: kernel putchar interface */ […] };
- 61. Por$ng NetBSD • 5°) Implement excep$on handlers • 6°) Call milkymist_startup() C code – Ini$alize console driver • -‐> consinit() -‐> milkymist_uart_cnaOach() • cn_tab = &milkymist_com_cons; – Ini$aliaze virtual memory subsystem • Call MD pmap_bootstrap() – Let the kernel boot • Call NetBSD MI main()
- 62. Por$ng NetBSD • 7°) Implement pmap.9 pmap -‐-‐ machine-‐dependent por/on of the virtual memory system – pmap_bootstrap() – pmap_init, pmap_create, pmap_destroy … – SW managed TLB? -‐> sys/uvm/pmap/ – used in (PowerPC Booke and LM32)
- 63. Por$ng NetBSD • 8°) Implement copyin/copyout • 9°) Implement atomic opera$ons – No atomic instruc$on à RAS (Restartable Atomic Sequence) CAS (Compare And Swap) – Other atomic and locking ops built around this CAS
- 64. RAS CAS int _atomic_cas_32(vola$le uint32_t *val, uint32_t old, uint32_t new); _atomic_cas_32: _atomic_cas_ras_start: lw r4, (r1+0) /* load *val into r4 */ bne r4, r2, 1f /* compare r4 (*val) and old (r2) */ sw (r1+0), r3 _atomic_cas_ras_end: 1: mv r1, r4 /* return (*val) */ ret
- 65. Por$ng NetBSD • 10°) Add support for interrupts – Write a func$on to register interrupt handlers • 11°) Have a running system clock – Write cpu_initclocks() – Write clock irq handler • Call hardclock()
- 66. Other func$ons to write • Switch context from one thread to another – cpu_switchto(9) • Copy data and abort on page fault – kcopy(9) • Save current context – sebault() • Low level code to finish up fork() opera$on – cpu_lwp_fork(9)
- 67. Other func$ons to write • Block interrupts to protect cri$cal sec$ons – spl(9) • Init CPU and print copyright message – cpu_startup(9) • Determine the root file system device – cpu_rootconf(9) • Etc…
- 68. Por$ng NetBSD • To boot user space – Create dummy ramdisk with /sbin/init – Build kernel with MFS – Insert ramdisk with mdse$mage – Boot it!
- 69. Por$ng NetBSD DEMO
- 70. NetBSD/milkymist virt. Memory Layout Kernel space User space 0 0xffffffff 0xc0000000 0xc8000000 RAM window User stack Kernel stack DDR SDRAM : 128 MB
- 71. RAM window Physical RAM 128 MB Start: PA 0x4000 0000 End: PA 0x4800 0000 Virtual Memory Layout VA: 0xc000 0000 VA: 0xc800 0000 Direct mapping
- 72. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x1234 1000 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC
- 73. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x1234 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC Binary à 0000000011 0000000010 101010111100 10 bits 10 bits 12 bits 1024 entries 1024 entries
- 74. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x1234 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC Binary à 0000000011 0000000010 101010111100 1024 entries 1024 entries
- 75. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x1234 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC Binary à 0000000011 0000000010 101010111100 1024 entries 1024 entries
- 76. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x4123 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC Binary à 0000000011 0000000010 101010111100 1024 entries 1024 entries
- 77. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x4123 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2ABC Binary à 0000000011 0000000010 101010111100 Physical address: 0x4123 1xxx 1024 entries 1024 entries
- 78. Page table Page directory @ 0xc300 0000 0x**** **** 0x**** **** 0x**** **** 0xc400 1000 0x**** **** Page table @ 0xc400 1000 0x**** **** 0x**** **** 0x4123 1xxx 0x**** **** 0x**** **** Virtual address: 0x00C0 2 ABC Binary à 0000000011 0000000010 101010111100 Physical address: 0x4123 1ABC 1024 entries 1024 entries
- 79. Accessing the page table • Kernel running with MMU ON o Direct access through virt. addresses 0xc*** **** • TLB miss handler running with MMU OFF o Computes phys. addresses from RAM window addresses using following formula o PA = VA – 0xC000 0000 + 0x4000 0000
- 80. Follow progress • hOp://sionneau.net/edgebsd/ • hOp://edgebsd.org/edgebsd/wiki/14/port-‐ lm32 • hOp://github.com/fallen/NetBSD • hOp://git.edgebsd.org/gitweb/?p=edgebsd-‐ src.git;a=shortlog;h=refs/heads/fallen-‐port-‐ milkymist (aka hOp://goo.gl/9Mbg7U)
- 81. Thank you! Sébas$en Bourdeauducq, Michael Walle, Robert Swindells, Stefan Kris$ansson, Lars-‐Peter Clausen, Pierre Pronchery, Radoslaw Kujawa, Youri Mouton, MaO Thomas, tech-‐kern@, M-‐ Labs mailing list, and many more.
- 82. QuesLons?