Metasepi team meeting #13: NetBSD driver using Haskell
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Metasepi team meeting #13: NetBSD driver using Haskell

on

  • 1,246 views

Metasepi team meeting #13: NetBSD driver using Haskell

Metasepi team meeting #13: NetBSD driver using Haskell

Statistics

Views

Total Views
1,246
Views on SlideShare
887
Embed Views
359

Actions

Likes
3
Downloads
9
Comments
0

9 Embeds 359

http://metasepi.org 272
http://0.0.0.0 44
https://twitter.com 32
http://www.google.co.jp 3
http://translate.googleusercontent.com 2
https://home.jolicloud.com 2
http://www.metasepi.org 2
http://www.linkedin.com 1
http://127.0.0.1 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Metasepi team meeting #13: NetBSD driver using Haskell Presentation Transcript

  • 1. Metasepi team meeting #13:  #13: NetBSD driver using Haskell Metasepi Project / Kiwamu Okabe
  • 2. Who am I? ☆ http://www.masterq.net/ ☆ Self employed software engineer ☆ Founder of Metasepi Project ☆ A developer of Ajhc Haskell compiler ☆ A Debian Maintainer ☆ 10 years' experience in developing OS using NetBSD.
  • 3. Agenda ☆ [1] Demo ☆ [2] What is Ajhc? ☆ [3] What is Metasepi? ☆ [4] What is compiler to build OS ☆ [5] NetBSD driver using Haskell ☆ [6] Ajhc is the best? ☆ [7] From Arafura to Bohai
  • 4. [1] Demo ☆ NetBSD audio driver play sound ☆ The driver's interrupt handler rewrited using Haskell ☆ Watch the movie at following http://bitly.com/netbsdajhc ☆ The source code at following github.com/metasepi/netbsd-arafura-s1
  • 5. Demo architecture
  • 6. [2] What is Ajhc? http://ajhc.metasepi.org/ ☆ Ajhc := Arafura designed jhc ☆ jhc := John's Haskell Compiler ☆ http://repetae.net/computer/jhc/ ☆ Jhc outputs binary that has lowmemory-footprint and runs fast. ☆ Good for embedded software.
  • 7. Who is John? ☆ John Meacham ☆ http://repetae.net/
  • 8. Why need Ajhc? ☆ GHC is de facto standard on Haskell. ☆ GHC := Glasgow Haskell Compiler ☆ http://www.haskell.org/ghc/ ☆ Why need another Haskell compiler? ☆ To develop kernel named "Metasepi".
  • 9. [3] What is Metasepi? http://metasepi.org/ ☆ Unix-like OS designed by strong type. ☆ Using ML or more strong type lang. Haskell http://www.haskell.org/ OCaml http://caml.inria.fr/ MLton http://mlton.org/ . . . and suchlike.
  • 10. Why need Metasepi? ☆ We have already Linux or Windows. ☆ But the developers are suffering. ☆ If use the kernel changed by you, ☆ you will get many runtime error. ☆ Difficult even to reproduce it.
  • 11. Doesn't OSS have good quality? ☆ "The Cathedral and the Bazaar" ☆ "Given enough eyeballs, all bugs are shallow." http://cruel.org/freeware/cathedral.html ☆ But if you develop your own product reusing OSS...
  • 12. Low quality out of OSS umbrella
  • 13. Type safety ☆ Less runtime errors. ☆ "数理科学的バグ撲滅方法論のすすめ" http://itpro.nikkeibp.co.jp/article/COLUMN/20060915/248230/
  • 14. Kernel wants type desperately ☆ Kernels are developed with C lang. ☆ Error on user space => SEGV ☆ Error on kernel space => Halt! ☆ Should design kernel with the greatest care. ☆ C language is safe?
  • 15. [4] What is compiler to build OS ☆ Need strong type. ☆ Need flexibility such as C language. ☆ Create it if there are not! ☆ From scratch? No thank you... ☆ Look for our compiler base.
  • 16. Want POSIX free compiler Programs to print "hoge" on terminal. The lesser depends on POSIX, the smaller values.
  • 17. Jhc output has only 20 undef $ nm hs.out | grep U U U U U U U U U U U U U U U U U U U U "U " _IO_putc@@GLIBC_2.2.5 __libc_start_main@@GLIBC_2.2.5 _setjmp@@GLIBC_2.2.5 abort@@GLIBC_2.2.5 ctime@@GLIBC_2.2.5 exit@@GLIBC_2.2.5 fflush@@GLIBC_2.2.5 fprintf@@GLIBC_2.2.5 fputc@@GLIBC_2.2.5 fputs@@GLIBC_2.2.5 free@@GLIBC_2.2.5 fwrite@@GLIBC_2.2.5 getenv@@GLIBC_2.2.5 malloc@@GLIBC_2.2.5 memset@@GLIBC_2.2.5 posix_memalign@@GLIBC_2.2.5 realloc@@GLIBC_2.2.5 setlocale@@GLIBC_2.2.5 sysconf@@GLIBC_2.2.5 times@@GLIBC_2.2.5
  • 18. Jhc is translator to C language
  • 19. Easy to cross build
  • 20. Survive burning out Let's develop with dogfooding style. (The method is called "Snatch".)
  • 21. [5] NetBSD driver using Haskell ☆ Snatch target: HD Audio driver sys/dev/pci/hdaudio |-- Makefile |-- ceareg.h |-- eldreg.h |-- files.hdaudio |-- hdafg.c |-- hdafg_dd.c |-- hdafg_dd.h |-- hdaudio.c |-- hdaudio_config.h |-- hdaudio_ids.c |-- hdaudio_ids.h |-- hdaudio_mixer.h |-- hdaudio_pci.c |-- hdaudio_pci.h |-- hdaudioio.h |-- hdaudioreg.h |-- hdaudiovar.h `-- hdmireg.h
  • 22. Rewrited the driver entirety? ☆ No, partly
  • 23. Where is Haskell code? (cont.) netbsd-arafura-s1 # <= based on NetBSD original source tree |-- BUILDING |-- GNUmakefile # <= Wrapper for building kernel |-- Makefile |-- Makefile.inc -- snip -|-- metasepi | |-- sound | | |-- Epopsan-signal.mp3 | | `-- TheMojitoofFluorescence.mp3 | `-- sys | |-- conf | | `-- files.haskell | |-- hsdummy # <= C code to support Ajhc runtime | `-- hssrc # <= Haskell code located | |-- Arch | | `-- I386 | | |-- I386 | | | `-- BusDma.hs | | `-- Include | | |-- BusDefs.hs | | |-- Cpu.hs | | `-- Types.hs
  • 24. Where is Haskell code? (cont.) | | | | | | | | | | | | | | | | | | | | | | | |-| | | | | | | | | | | | | | | |-| | | |-| | Dev |-|-|-| | `-- Auconv.hs AudioIf.hs Ic |-- Ac97reg.hs `-- Ac97var.hs Pci |-- Auich | |-- Intr.hs | `-- Ptr.hs |-- Auich.hs |-- Auichreg.hs `-- Hdaudio |-- Hdaudio.hs |-- Hdaudioreg.hs `-- Hdaudiovar.hs Kern |-- KernMutex.hs |-- SubrKmem.hs `-- SubrPrf.hs Lib `-- Libkern `-- Libkern.hs
  • 25. Where is Haskell code? | | | | | | | | | | |-|-|-|-|-|-|-|-|-|-`-- |-|-| `-- regress rescue sbin share sys tests tools travis-ci usr.bin usr.sbin x11 Main.hs Metasepi `-- EitherIO.hs Sys |-- Audioio.hs |-- Bus.hs |-- Device.hs |-- Errno.hs |-- Proc.hs `-- Types.hs
  • 26. How to build ☆ Install some packages $ sudo apt-get install ffmpeg qemu gcc ☆ Install Ajhc and git clone $ git clone https://github.com/metasepi/netbsd-arafura-s1.git $ cd netbsd-arafura-s1 ☆ Build QEMU image $ make bootcd
  • 27. Run the kernel Run kernel on qemu $ make qemu >> NetBSD/x86 BIOS Boot, Revision 5.9 (from NetBSD 6.1.1_PATCH) >> Memory: 639/1047544 k --snip-Created tmpfs /dev (1490944 byte, 2880 inodes) erase ^?, werase ^W, kill ^U, intr ^C This image contains utilities which may be needed to get you out of a pinch. # Kick play.sh script to play sound # ./play.sh
  • 28. Build process
  • 29. How to call C (figure)
  • 30. How to call C (code) -- File: metasepi/sys/hssrc/Sys/Bus.hs busSpaceRead4 = c_bus_space_read_4 busSpaceWrite4 = c_bus_space_write_4 foreign import ccall "hs_extern.h bus_space_read_4" c_bus_space_read_4 :: BusSpaceTagT -> BusSpaceHandleT -> BusSizeT -> IO Word32 foreign import ccall "hs_extern.h bus_space_write_4" c_bus_space_write_4 :: BusSpaceTagT -> BusSpaceHandleT -> BusSizeT -> Word32 -> IO () -- File: metasepi/sys/hssrc/Arch/I386/Include/BusDefs.hs type BusSizeT = CSize newtype {-# CTYPE "struct bus_space_tag" #-} BusSpaceTag = BusSpaceTag () type BusSpaceTagT = Ptr BusSpaceTag type BusSpaceHandleT = VaddrT
  • 31. Trace pointer chain /* C code */ static int auich_open(void *addr, int flags) { struct auich_softc *sc; sc = (struct auich_softc *)addr; mutex_spin_exit(&sc->sc_intr_lock); sc->codec_if->vtbl->lock(sc->codec_if); // -- snip -- -- Haskell code auichOpen :: Ptr AuichSoftc -> Int -> IO Int auichOpen sc flags = do mutexp <- p_AuichSoftc_sc_intr_lock sc codecif <- peek =<< p_AuichSoftc_codec_if sc lock <- peek =<< p_Ac97CodecIfVtbl_lock =<< peek =<< p_Ac97CodecIf_vtbl codecif mutexSpinExit mutexp call_Ac97CodecIfVtbl_lock lock codecif -- snip --
  • 32. But need assistant code!
  • 33. Tool to generate the code https://github.com/ajhc/struct2hs ☆ Generate Haskell code scanning C $ struct2hs sys/dev/pci/auich_extern_SNATCHED.h | tail offsetOf_Pdevinit_pdev_attach :: Int p_Pdevinit_pdev_attach :: Ptr Pdevinit -> IO (Ptr (Ptr (FunPtr (Int -> IO ())))) p_Pdevinit_pdev_attach p = return $ plusPtr p $ offsetOf_Pdevinit_pdev_attach foreign import ccall "dynamic" call_Pdevinit_pdev_attach :: FunPtr (Int -> IO ()) -> Int -> IO () foreign import primitive "const.offsetof(struct pdevinit, pdev_count)" offsetOf_Pdevinit_pdev_count :: Int p_Pdevinit_pdev_count :: Ptr Pdevinit -> IO (Ptr Int) p_Pdevinit_pdev_count p = return $ plusPtr p $ offsetOf_Pdevinit_pdev_count
  • 34. [6] Ajhc is the best? ☆ No. ☆ Depended on GC ☆ Too buggy compiler https://github.com/ajhc/ajhc/issues ☆ Can't use MonadTrans ☆ Debug hard for strong optimization
  • 35. Before compiling (Haskell) hdaudioIntr :: Ptr HdaudioSoftc -> IO Int hdaudioIntr sc = do intsts <- hdaRead4 sc e_HDAUDIO_MMIO_INTSTS if intsts .&. e_HDAUDIO_INTSTS_GIS /= 0 then hdaudioIntr' sc intsts else return 0 hdaudioIntr' :: Ptr HdaudioSoftc -> Word32 -> IO Int hdaudioIntr' sc intsts = do when (intsts .&. e_HDAUDIO_INTSTS_CIS /= 0) $ do rirbsts <- hdaRead1 sc e_HDAUDIO_MMIO_RIRBSTS when (rirbsts .&. e_HDAUDIO_RIRBSTS_RINTFL /= 0) $ do mutexp <- p_HdaudioSoftc_sc_corb_mtx sc mutexEnter mutexp hdaudioRirbDequeue sc True mutexExit mutexp when (rirbsts .&. (e_HDAUDIO_RIRBSTS_RIRBOIS .|. e_HDAUDIO_RIRBSTS_RINTFL) /= 0) $ hdaWrite1 sc e_HDAUDIO_MMIO_RIRBSTS rirbsts hdaWrite4 sc e_HDAUDIO_MMIO_INTSTS e_HDAUDIO_INTSTS_CIS when (intsts .&. e_HDAUDIO_INTSTS_SIS_MASK /= 0) $ do mutexp <- p_HdaudioSoftc_sc_stream_mtx sc -- snip --
  • 36. After compiling (C language)
  • 37. Debug in kernel ☆ Hardware state is sometime buggy ☆ Not only debug software ☆ but also hardware! ☆ Gap between Haskell and C code ☆ Can't debug pure Haskell code ☆ Need to debug generated C code ☆ Where is more better compiler?
  • 38. [7] From Arafura to Bohai ☆ We are in Arafura iteration ☆ Arafura := Ajhc + Snatch + NetBSD ☆ Use "ATS" compiler on the next ☆ Let's start Bohai iteration! ☆ Bohai := ATS + Snatch + NetBSD
  • 39. What is ATS language? http://www.ats-lang.org/ ☆ Has syntax like ML ☆ Compiled into C ☆ Dependent type ☆ Linear type ☆ Without GC ☆ Without any runtime
  • 40. Japan ATS User Group http://jats-ug.metasepi.org/ ☆ Translating ATS documents ☆ ATS propaganda in Japan ☆ Join us!
  • 41. Hongwei said... (cont.) ☆ Hongwei Xi as ATS author ☆ Associate Professor ☆ at Boston University Date: Mon Dec 23 11:40 JST 2013 Hi Metasepi-chan, I spent quite some time today browsing metasepi.org. I am really interested in your Metasepi project, partly because I myself wanted to implement NetBSD in ATS about 5 years ago. Unfortunately, I never had time to get the project started as I needed to spend so much time on ATS2. By the way, I had planned to use the very same approach which you call "Snatch". I had also considered Minix but I chose NetBSD because it was a real OS.
  • 42. Hongwei said... (cont.) I think I know first-handedly the dilemma you are currently one hand, you want to be able to fully focus on writing the kernel. On the other hand, you also need to add features to constantly to address all kinds of issues that keep popping which undoubtedly makes it very difficult for you to focus. in. On Ajhc up, I would highly recommend that you use ATS to implement NetBSD kernel. Unlike jhc, there is no semantics gap between ATS and C. In particular, they both use the same native unboxed data representation. Once you become familiar with ATS, you can readily visualize the C code that your ATS source is to be compiled into. ATS is truly an ideal language for the kind of "Snatch" approach you want to take to re-write NetBSD. If you take the lead, then I will be happy to "chip in" :) I also spent some time reading documentation on jhc. Personally, I feel that there is simply too much uncertainty to use it in real kernel implementation. Features like GC could make the kernel highly unpredictable, scaring away potential users.
  • 43. Hongwei said... I think that we both believe C is the right language for systems programming. The problem with C is that it is too difficult to write correct C programs. ATS is designed to allow the programmer to correctly write the kind of C programs he or she wanted to write in the first place. While jhc generates C code, the kind of C code it generates may not be suited for kernel. This is what I call a semantics gap. I write this message in the hope that we could join effort in doing something that has not been done up to now: Writing a real kernel in (largely) functional style that can truly deliever safety-wise as well as performance-wise. Cheers, --Hongwei