Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CIF16: Tor in Haskell, or How To Write Programs For Unikernels (Adam Wick, Galois Inc.)

You did what, in what, why? In this talk, we’ll talk about Tor, Haskell, and unikernels, and why the technologies make for such a great combination. We’ll then look a little deeper, and walk through the design and construction process of such a project. Why is the first step in building a unikernel to not build a unikernel? Why is resource measurement so important? How the heck to you debug these things? Throughout, we’ll use our Tor implementation as our running example, providing some grounding for the proceedings. Code will be shown, and the pitfalls described. And, in the end, we’ll show how building unikernels isn’t actually all that much different from building anything else … if you take some simple precautions, don’t assume too much, and remember to do things the right way.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

CIF16: Tor in Haskell, or How To Write Programs For Unikernels (Adam Wick, Galois Inc.)

  1. 1. Adam Wick @ CIF16 Tor in Haskell or How to Write Programs for Unikernels
  2. 2. This combination is not crazy. (In fact, it makes a lot of sense.)
  3. 3. © 2016 Galois, Inc.3 © 2016 Galois, Inc.3 Adam Wick @ CIF16 Built the Haskell Lightweight Virtual Machine (HaLVM) Works for (a unikernel) Adam Wick
  4. 4. © 2016 Galois, Inc.4 © 2016 Galois, Inc.4 Why are Unikernels and Tor such a good idea? Unikernels are fundamentally designed to be Lightweight Fast Secure More nodes for the same price, or the same number of nodes for less. Fewer schedulers means lower latency. A much smaller OS stack means less code to get wrong.
  5. 5. So We Did This (And now I’m going to tell you about it.)
  6. 6. But First: Tor
  7. 7. © 2016 Galois, Inc.7 © 2016 Galois, Inc.7 : Anonymity Through Root Vegetables Tor is an anonymous communication layer built on TLS that prevents people from learning who you are interacting with over the Internet.* * Terms and conditions may apply.
  8. 8. © 2016 Galois, Inc.8 © 2016 Galois, Inc.8 Guarantees*: 1. For any single relay node, it is impossible to tell where you are in the chain. 2. The exit node cannot tell how many hops were before it. 3. The entrance node cannot tell how many hops are in front of it. * Terms and conditions may apply.
  9. 9. © 2016 Galois, Inc.9 © 2016 Galois, Inc.9 Terms and Conditions May Apply The security of Tor is dependent on: 1. The correctness of the protocol design. 2. The correctness of the protocol implementation. 3. Ensuring that there are enough nodes that no single entity owns a sufficient percentage of nodes to have a top level view.
  10. 10. And Second: Haskell
  11. 11. © 2016 Galois, Inc.11 © 2016 Galois, Inc.11 It is also known for being popular with people that know more category theory than is good for them, which has given it a certain … reputation. Haskell is a functional programming language: fact :: Integer -> Integer map :: (a -> b) -> [a] -> [b] fact 0 = 1 map _ [] = [] fact x = x * fact (x – 1) map f (x:r) = (f x) : map f r It is mostly known for being lazy, pure, and strongly typed. Very HelpfulHandyMeh
  12. 12. © 2016 Galois, Inc.12 © 2016 Galois, Inc.12 do -- "To authenticate the initiator, the responder MUST check the -- following: -- * The CERTS cell contains exactly one CerType 3 'AUTH' -- certificate. let authCert = exactlyOneAuth certs Nothing authCert' = signedObject (getSigned authCert) -- * The CERTS cell contains exactly one CerType 2 'ID' -- certificate let iidCert = exactlyOneId certs Nothing iidCert' = signedObject (getSigned iidCert) -- * Both certificates have validAfter and validUntil dates -- that are not expired. when (certExpired authCert' now) $ fail "Auth certificate expired." when (certExpired iidCert' now) $ fail "Id certificate expired." -- * The certified key in the AUTH certificate is a 1024-bit RSA -- key. unless (is1024BitRSAKey authCert) $ fail "Auth certificate key is the wrong size." -- * The certified key in the ID certificate is a 1024-bit RSA -- key. unless (is1024BitRSAKey iidCert) $ fail "Identity certificate key is the wrong size." -- * The auth certificate is correctly signed with the key in the -- ID certificate. unless (authCert `isSignedBy` iidCert') $ fail "Auth certificate not signed by identity cert."
  13. 13. © 2016 Galois, Inc.13 © 2016 Galois, Inc.13 I’m Not Here To Sell You Haskell, But: Type Safety + Purity + Safe Data Structures + GC = Fewer bugs per LOC = Less security advisories per LOC
  14. 14. Unikernels You probably know what these are by now.
  15. 15. So: Tor + Haskell + Unikernels?
  16. 16. © 2016 Galois, Inc.16 © 2016 Galois, Inc.16 Terms and Conditions May Apply The security of Tor is dependent on: 1. The correctness of the protocol design. 2. The correctness of the protocol implementation. 3. Ensuring that there are enough nodes that no single entity owns a sufficient percentage of nodes to have a top level view. Can’t really do much for this one. Except that more implementations means more eyes on the specs. Haskell limits memory issues, type safety helps correctness. Unikernels limit attack surface. Unikernels limit resource usage, which allows a lot more relay nodes for the same cost.
  17. 17. Cool! How Did That Work?
  18. 18. © 2016 Galois, Inc.18 © 2016 Galois, Inc.18 So You Want To Build A Unikernel There are five steps to building a Unikernel: 1. Don’t. 2. Test & Measure. 3. Do. 4. Test (Part II) 5. Deploy.
  19. 19. © 2016 Galois, Inc.19 © 2016 Galois, Inc.19 Step #1: Don’t Build a Unikernel Building a unikernel adds a number of complications to the development process (which is already complicated enough). So start by building your application as you normally would, in a language that supports unikernels. (In this case, Haskell.) Build using your normal tools, libraries, and techniques, but: 1. Try to avoid local storage. 2. Try to minimize the number of libraries you pull in. 3. Stay away from libraries that link against C. With Tor, I began by implementing the core parsing and protocol code, sufficient to anonymously look up a hostname. To avoid a linking problem, I also ended up writing my own zlib decompression library. More on that later.
  20. 20. © 2016 Galois, Inc.20 © 2016 Galois, Inc.20 Step #2: Measure & Test What is your application’s peak memory use? Exactly how much space do you need for configuration files, etc.? Does this space need to be writeable? Then TEST, TEST, TEST. Your goal is that in later steps, any bugs you find have to do with the unikernel translation, not your application. In measuring my work with Tor, I discovered that the zlib library I’d written earlier was very … silly ... with regard to memory use. I wish I’d found that here.
  21. 21. © 2016 Galois, Inc.21 © 2016 Galois, Inc.21 Step #3: Do it. Once it works well, and you know about its resource usage, now it’s time to start converting it to a unikernel. 1. Some of your libraries won’t build, and you will either need to fix or replace these. Predicting which ones is possible, but requires some experience and internal knowledge of the library. 2. Disks are expensive and slow; consider using ramdisks. For many of our uses of unikernels, and Tor can be one of them, then the only need for a disk is to pass configuration information: small bits of read-only data. Ramdisks are the way to go. 3. You will need to rewrite your start-up code. You will now need to explicitly instantiate all your devices, for example. 4. Your Edit-Compile-Test loop just got more painful. Sorry. Getting things working on the HaLVM required: • Updates to the TLS library I was using. • Updates to the x.509 library I was using. And it benefited from a bunch of previous work I’d done for other unikernels.
  22. 22. © 2016 Galois, Inc.22 © 2016 Galois, Inc.22 Step #4: Test (Part II) Hopefully, any bugs you hit at this point are bugs in the adaptation to the unikernel, not in your code. But here are some more questions to verify: 1. Does your system come up cleanly, every time, even with flakey device timings? 2. Are you flying safely under your memory bounds? (If so, can you lower them safely?) 3. Can you run your test suite as a unikernel? (If so, do it.) 4. Optional: Does your new unikernel survive migration? This is still in progress. (This is always in progress.)
  23. 23. © 2016 Galois, Inc.23 © 2016 Galois, Inc.23 Step #5: Deploy The unikernel community is working to make deployment to all the standard clouds quicker and easier, but there are documents online that you can use. There are tools for distributing to EC2, but they’re incomplete. Different unikernels are in different places with regard to EC2 and other clouds.
  24. 24. Let’s See Some Code
  25. 25. © 2016 Galois, Inc.25 © 2016 Galois, Inc.25 Let’s Run Some Tor main :: IO () main = runDefaultMain $ flags -> do (MkNS ns, logger) <- initializeSystem flags let options = defaultTorOptions{ … } tor <- startTor ns options addrs <- torResolveName tor "www.whatismypublicip.com" case addrs of [] -> putStrLn ("Could not resolve www.whatismypublicip.com!") ((addr, _ttl) : _) -> do sock <- torConnect tor addr 80 putStrLn ("Connected to " ++ show addr) torWrite sock (buildGet "/") putStrLn ("Wrote GET request.") resp <- readLoop sock putStrLn ("Response: " ++ show resp) torClose sock ReasonDone Parse command line arguments, turn them into Tor configuration options, and start up the network.Start the various daemon threads required to run a Tor entrance node. Anonymously look up a domain name, based on the options provided earlier. Create an anonymous connection and read the response.
  26. 26. © 2016 Galois, Inc.26 © 2016 Galois, Inc.26 Starting Means Booting initializeSystem :: [Flag] -> IO (SomeNetworkStack, String -> IO ()) initializeSystem _ = do con <- initXenConsole xs <- initXenStore ns <- newNetworkStack macstr <- findNIC xs nic <- openNIC xs macstr let mac = read macstr addDevice ns mac (xenSend nic) (xenReceiveLoop nic) deviceUp ns mac ipaddr <- dhcpDiscover ns mac return (MkNS (hansNetworkStack ns), makeLogger ( x -> writeConsole con (x ++ "n"))) Most unikernels will get your memory and such set up, but devices are all up to you. Network setup is all your responsibility.
  27. 27. © 2016 Galois, Inc.27 © 2016 Galois, Inc.27 Abstracting Over Your Network Stack -- |The type of a Tor-compatible network stack. The first type variable is the -- type of a listener socket, the second the type of a standard connection -- socket. data TorNetworkStack lsock sock = TorNetworkStack { connect :: String -> Word16 -> IO (Maybe sock) -- |Lookup the given hostname and return any IP6 (Left) or IP4 (Right) -- addresses associated with it. , getAddress :: String -> IO [TorAddress] , listen :: Word16 -> IO lsock , accept :: lsock -> IO (sock, TorAddress) , recv :: sock -> Int -> IO S.ByteString , write :: sock -> L.ByteString -> IO () , flush :: sock -> IO () , close :: sock -> IO () , lclose :: lsock -> IO () }
  28. 28. © 2016 Galois, Inc.28 © 2016 Galois, Inc.28 QuickCheck! tapHandshakeCheck :: Word32 -> RouterTAP -> TorRNG -> Bool tapHandshakeCheck circId (RouterTAP myRouter priv) g0 = let (g1, (privX, cbody)) = startTAPHandshake myRouter g0 (g2, (dcell, fenc, benc)) = advanceTAPHandshake priv circId cbody g1 Created circIdD dbody = dcell in case completeTAPHandshake privX dbody of Left err -> False Right (fenc', benc') -> (circId == circIdD) && (fenc == fenc') && (benc == benc')
  29. 29. © 2016 Galois, Inc.29 © 2016 Galois, Inc.29 QuickCheck! TorCell Serialization: TorAddress round-trips: [OK, passed 100 tests] TorAddress makes sensible ByteStrings: [OK, passed 100 tests] ExtendSpec serializes: [OK, passed 100 tests] DestroyReason serializes (check #1): [OK, passed 100 tests] DestroyReason serializes (check #2): [OK, passed 100 tests] HandshakeType serializes (check #1): [OK, passed 100 tests] HandshakeType serializes (check #2): [OK, passed 100 tests] RelayEndReason serializes: [OK, passed 100 tests] RelayCell serializes: [OK, passed 100 tests] RelayCell serializes w/ digest: [OK, passed 100 tests] RelayCell serializes w/ digest: [OK, passed 100 tests] Tor certificates serialize: [OK, passed 100 tests] Hybrid encryption tests: Hybrid encryption works when forced: [OK, passed 100 tests] Hybrid encryption works in general: [OK, passed 100 tests] Handshakes: TAP Handshake: [OK, passed 100 tests] NTor Handshake: [OK, passed 100 tests] Make sure we get data formats like. Next step: fuzzing to make sure we’re sufficiently defensive. Test custom crypto. Test handshakes.
  30. 30. What is it like developing unikernels?
  31. 31. © 2016 Galois, Inc.31 © 2016 Galois, Inc.31 Boring • It’s just like normal development when you start. • Then you have to be a little more rigorous about testing and evaluation. • Then you run into a wall with library support. • Except it’s not nearly as much of a problem as it used to be. • With Tor, this meant writing one additional library from scratch (zlib), and then sending patches to a few other authors. • Debugging unikernels is also now more palatable: • printf(), gdbsx, profiling • The big remaining gap: the big web frameworks don’t work. • They require all the deep device and OS integration that causes problems with unikernels. (Mostly)
  32. 32. © 2016 Galois, Inc.32 © 2016 Galois, Inc.32 But What You Get Is More nodes, For less money, With a better security posture. Which is great for , and perhaps your next project.
  33. 33. © 2016 Galois, Inc.33 © 2016 Galois, Inc.33 All trademarks, service marks, trade names, trade dress, product names and logos appearing in these slides are the property of their respective owners, including in some instances Galois, Inc. All rights are reserved. http://github.com/GaloisInc/haskell-tor Adam Wick awick@galois.com Twitter: @acwpdx Any questions?
  34. 34. © 2016 Galois, Inc.34 © 2016 Galois, Inc.34 Unikernels are specialised, single address space machine images constructed using library operating systems. - Wikipedia or Unikernels : Virtual Machines :: Exokernels : Physical Machines or Unikernels are single-process programs compiled to run directly on (usually virtual) hardware, rather than within a full-featured OS.
  35. 35. © 2016 Galois, Inc.35 © 2016 Galois, Inc.35
  36. 36. © 2016 Galois, Inc.36 © 2016 Galois, Inc.36 Lower operating costs Faster response to events Smaller attack surface
  37. 37. © 2016 Galois, Inc.37 © 2016 Galois, Inc.37 Containers and Unikernels Unikernels: The next step in separation: 1. We started with single-task code. 2. Next, we separated some of our tasks into threads. 3. Next, we separated some of our threads into processes. 4. Next, we separated some of our processes into containers. 5. Now, we separated our critical containers into unikernels. Each level provides additional separation properties, at the cost of increased complexity in creating and reasoning about new code.

    Be the first to comment

    Login to see the comments

You did what, in what, why? In this talk, we’ll talk about Tor, Haskell, and unikernels, and why the technologies make for such a great combination. We’ll then look a little deeper, and walk through the design and construction process of such a project. Why is the first step in building a unikernel to not build a unikernel? Why is resource measurement so important? How the heck to you debug these things? Throughout, we’ll use our Tor implementation as our running example, providing some grounding for the proceedings. Code will be shown, and the pitfalls described. And, in the end, we’ll show how building unikernels isn’t actually all that much different from building anything else … if you take some simple precautions, don’t assume too much, and remember to do things the right way.

Views

Total views

1,268

On Slideshare

0

From embeds

0

Number of embeds

117

Actions

Downloads

9

Shares

0

Comments

0

Likes

0

×