Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CIF16: Knock, Knock: Unikernels Calling! (Richard Mortier, Cambridge University)

The lightweight and secure nature of Unikernels means that a prime use-case is to customise network behaviour. At the same time, the high-level languages that many are written in means that this sort of low-level coding is opened up to those who might not traditionally consider themselves "systems developers".
MirageOS is a particular unikernel platform built in the OCaml functional programming language. Able to seamlessly target a range of environments, from a local (POSIX) development environment to Xen virtual machines running on the cloud, it is a prime example of the ways that unikernels open up low-level development.
I will briefly introduce MirageOS before walking through an example developing and then running on Xen a simple network proxy using MirageOS. This proxy will implement a basic form of port-knocking, requiring a sequence of TCP connections (SYNs) to be made to the proxy to indicate a target, before permitting an outgoing connection to that target to be made.
Thanks to Thomas Gazagnaire for the material used in the walkthrough!

  • Login to see the comments

  • Be the first to like this

CIF16: Knock, Knock: Unikernels Calling! (Richard Mortier, Cambridge University)

  1. 1. KNOCK,KNOCK: UNIKERNELSCALLING! Richard Mortier University of Cambridge @mort___ Press <esc> to view the slide index, and the <arrow> keys to navigate.
  2. 2. ABOUTME Ph.D. with Systems Research Group at Cambridge University Computer Lab Now faculty at the Cambridge University Computer Lab Previously with University of Nottingham, Microsoft Research, Sprint ATL, startups
  3. 3. ABOUTTHISDECK Slides written in statically type-safe OCaml using Mirage on OSX They are hosted on my laptop as a content-specific webserver, connected to using a browser Their application logic is just a couple of source files, written independently of any OS dependencies Without any source level changes then can also be hosted in the cloud as an x86 Xen unikernel and on ARM Cubieboard2 Binaries small enough to track the entire deployment in Git!
  4. 4. TRADITIONALOPERATINGSYSTEMS A lot of code! Linux: over 25 million lines of code Debian 5.0: 65 million lines of code OSX 10.4: 85 million lines of code
  7. 7. TRADITION Current applications effectively rely on a software stack of 100M+ lines of code software stack Most of that stack is written in C, a long time ago As a result, it's hard to re-use it in different – and new – contexts It's great that we can engineer software to make all this work – but can we do better?
  8. 8. UNIKERNELS! Concepts derived from library OS technology from the 1990s Link application code together with platform libraries at build time Contain only the platform code specifically required by the application Single process, single address space Run anywhere simply by switching out lowest layer libraries during build
  9. 9. MIRAGEOSUNIKERNELS! A unikernel platform built in OCaml Strongly statically type-safe language dating back to the 1970s Very efficient runtime Builds over the Xen Mini-OS, targeting Xen's stable device driver interface Support for other backends in place and in development, including POSIX, KVM, bare-metal, your web browser...
  10. 10. BENEFITS ✓Reduced attack surface Static linking of only required libraries: removes unnecessary services – no Shellshock! Modern, high-level languages: static and runtime analysis, dead-code elimination
  11. 11. BENEFITS ✓Reduced attack surface ✓Increased speed Reduced boot time: can boot inside a TCP connection setup or packet RTT Fewer scheduling layers: lower latency, more predictable performance
  12. 12. BENEFITS ✓Reduced attack surface ✓Increased speed ✓Efficient resource use Reduced memory footprint: a typical stateless MirageOS app is ~10MB of RAM Small on-disk footprint: whole-program optimisation can create a MirageOS DNS server that comes in at ~200kB
  13. 13. BENEFITS ✓Reduced attack surface ✓Increased speed ✓Efficient resource use ✓Immutable Infrastructure Statically link data into your application: if desired, reduces dependency on external components Store outputs in Git: manage via git, introducing new models for update, upgrade, triage Can be sealed: once built, can even enable hardware memory- protection so running image is really immutable
  14. 14. MIRAGEOSWORKFLOW As easy as 1—2—3! 1. Write your OCaml application using the Mirage module types Express its configuration as OCaml code too! $mirageconfigureapp/
  15. 15. MIRAGEOSWORKFLOW As easy as 1—2—3! 1. Write your OCaml application using the Mirage module types Express its configuration as OCaml code too! 2. Compile it and debug under Unix using the miragetool $cdapp $makedepend#installlibrarydependencies $makebuild #buildtheunikernel
  16. 16. MIRAGEOSWORKFLOW As easy as 1—2—3! 1. Write your OCaml application using the Mirage module types Express its configuration as OCaml code too! 2. Compile it and debug under Unix using the miragetool 3. Once debugged, simply retarget it to Xen, and rebuild! $mirageconfigureapp/ $cdapp&&makedepend&&makebuild #rebuild Magic happens via the OCaml module system
  17. 17. MIRAGEOSTECHNOLOGY Capture system dependencies in code and compile them away  
  18. 18. MIRAGEOSTECHNOLOGY A Mirage component usually contains Code parameterised by functors with very limited (Mirage-only) dependencies, and particularly no OS dependencies A collection of libraries where the functors are (fully or partially) applied, suitable for interactive use Functors in the OCaml module system clearly separate OS component dependencies, breaking the monolithic OS down into components
  19. 19. MIRAGEOSTECHNOLOGY Unix/Sockets 8
  20. 20. MIRAGEOSTECHNOLOGY Direct/Static
  22. 22. DEMO:KNOCKKNOCK! The code behind the demo Single MirageOS unikernel Listens on 80/TCPas a web proxy Default: returns a basic hellopage Also has knocking control channel on 32000/TCP Single request RESETto reset knocking state Finally, listens on [32001,32256]/TCPfor knocks Once correct knock sequence received, proxy configured to forward HTTP requests to indicated site instead of returning static page.
  23. 23. UNIKERNELCONFIGURATION openMirage (*Declarewheretheunikernelcodeis*) letmain=foreign "Unikernel.Main" (*thenameofthefunctor*) (console@->stackv4@->job)(*thefunctorsignature*) letpackages= ["mirage-http"](*additionalOPAMpackagesneeded*) letlibraries=["mirage-http"](*additionalocamlfindlibrariesneeded*) (*Registertheapplication*) let()= letstack=generic_stackv4default_consoletap0in register"knocking"~packages~libraries[main$default_console$stack]
  24. 24. PROXYSTATE moduleState:sig typet (**Thetypeforproxyknockingstate.*) valcreate:control_port:int->t (**Createanewproxystate.[control_port]isthecontrolportnumber.*) valports:t->intlist (**[portst]isthesequenceofportknockingstoredinthestate.*) valadd:t->int->unit (**[addtport]add[port]tothecurrentobservedsequenceofports.*) valreset:t->unit (**[resett]resetstheproxystate.*) valcontrol_port:t->int (**[control_portt]retrievesthecontrolportnumber.*) end
  25. 25. DECODINGKNOCKS moduleHostname(C:CONSOLE):sig valdecode:C.t->State.t->stringoption (**[decodecst]resolvesportknocksstoredinproxy[!State][st]into hostname.[c]isaconsoleforlogoutput.*) end=struct letlogcfmt=...(*consolelogging;privatetomodule*) (*theportstable,privatetomodule*) lettable=[ [1] ,""; [1;2],""; ] letdecodect=...(*lookupknocksequencein[table]*) end
  26. 26. UNIKERNEL:HELPERS moduleMain(C:CONSOLE)(S:STACKV4)=struct moduleProxy=Proxy(C)(S) moduleHostname=Hostname(C) typerequest=Reset|Unknownofstring letstring_of_requestbuf= matchString.trim(Cstruct.to_stringbuf)with |"reset"->Reset |req ->Unknownreq letlogcfmt=... (*consolelogger*) letok_or_errordbgc=... (*chaininginvocationswitherrorhandling*) letnot_foundccon=... (*portknocknotrecognised*) letreply_onecstcon=matchHostname.decodectwith |Somehost->proxycsconhost|None->not_foundccon letproxycsconhost=... (*actuallyproxyincomingrequest:createsanHTTPserverlisteningthread thatreceivesincomingHTTPrequestsandproxiesittothetargethost*)
  27. 27. UNIKERNEL:HANDLINGAKNOCK letupdatectportflow= matchport-State.control_porttwith |0-> S.TCPV4.readflow>>=funquestion-> ok_or_error"update"cquestion>>=funbuf-> beginmatchstring_of_requestbufwith |Reset-> logc"Port0:RESET!">>=fun()-> State.resett; S.TCPV4.closeflow |Unknownreq-> logc"Port0:Unknownrequest(%s)"req>>=fun()-> S.TCPV4.closeflow end |port-> logc"Port%d:KNOCKKNOCK!"port>>=fun()-> State.addtport; S.TCPV4.closeflow
  28. 28. UNIKERNEL:ENTRYPOINT letstartcs= logc"PortKnockingProxyUnikernelbooted!n" >>=fun()-> letcontrol_port=32000in lett=State.create~control_portin (*registeraknockhandlerforalltheportsweareinterestedin*) fori=0to256do letport=control_port+iin S.listen_tcpv4s~port(updatectport) done; (*startthewebproxyon80/TCP*) S.listen_tcpv4s~port:80(reply_onecst); S.listens
  29. 29. THEPLUMBING autogenerated code to build the functors Unix with kernel socket networking stack moduleTcpip_stack_socket1=Tcpip_stack_socket.Make(Console_unix) Unix with direct networking stack moduleEthif1=Ethif.Make(Netif) moduleArpv41=Arpv4.Make(Ethif1)(Clock)(OS.Time) moduleIpv41=Ipv4.Make(Ethif1)(Arpv41) moduleUdp1=Udp.Make(Ipv41) moduleTcp1=Tcp.Flow.Make(Ipv41)(OS.Time)(Clock)(Random) moduleTcpip_stack_direct1=Tcpip_stack_direct.Make(Console_unix)(OS.Time) (Random)(Netif)(Ethif1)(Arpv41)(Ipv41)(Udp1)(Tcp1) Xen ... moduleTcpip_stack_direct1=Tcpip_stack_direct.Make(Console_xen)(OS.Time) (Random)(Netif)(Ethif1)(Arpv41)(Ipv41)(Udp1)(Tcp1)
  30. 30. THEPLUMBING plumb together to create the TCP/IP network stack letstackv4_socket1=lazy( let__console_unix_01=Lazy.forceconsole_unix_01in let__udpv4_socket11=Lazy.forceudpv4_socket11in let__tcpv4_socket11=Lazy.forcetcpv4_socket11in __console_unix_01>>=function |`Error_e->fail(Failure"console_unix_01") |`Ok_console_unix_01-> __udpv4_socket11>>=function |`Error_e->fail(Failure"udpv4_socket11") |`Ok_udpv4_socket11-> __tcpv4_socket11>>=function |`Error_e->fail(Failure"tcpv4_socket11") |`Ok_tcpv4_socket11-> letconfig={"stackv4_socket";console=_console_unix_01; interface=(Key_gen.interfaces());mode=()}in Tcpip_stack_socket1.connectconfig_udpv4_socket11_tcpv4_socket11 )
  31. 31. THEPLUMBING plumb together the console and the stack letf11=lazy( let__console_unix_01=Lazy.forceconsole_unix_01in let__stackv4_socket1=Lazy.forcestackv4_socket1in __console_unix_01>>=function |`Error_e->fail(Failure"console_unix_01") |`Ok_console_unix_01-> __stackv4_socket1>>=function |`Error_e->fail(Failure"stackv4_socket1") |`Ok_stackv4_socket1-> Unikernel1.start_console_unix_01_stackv4_socket1 >>=funt->Lwt.return(`Okt) )
  32. 32. THEPLUMBING plumb together the stack and the platform type letmirage1=lazy( let__key1=Lazy.forcekey1in let__f11=Lazy.forcef11in __key1>>=function |`Error_e->fail(Failure"key1") |`Ok_key1-> __f11>>=function |`Error_e->fail(Failure"f11") |`Ok_f11-> Lwt.return_unit )
  33. 33. THEPLUMBING finally, run the unikernel passing it the plumbed devices letrun= let()= lett= Lazy.forcekey1>>=function |`Error_e->exit1 |`Ok_->Lazy.forcemirage1 inrunt
  34. 34. HTTPS://MIRAGE.IO/ Thanks for listening! Questions? Give it a try: Particular thanks to for the demo unikernel! mirage/ Thomas Gazagnaire @mort___