Haskell for data science
•
4 likes•2,490 views
Slideshare hasn't imported my notes, so here's the link to the Google Presentation: https://goo.gl/Gl4Vhm Haskell is a statically typed, non strict, pure functional programming language. It is often talked and blogged about, but rarely used commercially. This talk starts with a brief overview of the language, then explains how Haskell is evaluated and how it deals with non-determinism and side effects using only pure functions. The suitability of Haskell for real world data science is then discussed, along with some examples of its users, a small Haskell-powered visualization, and an overview of useful packages for data science. Finally, Accelerate is introduced, an embedded DSL for array computations on the GPU, and an ongoing attempt to use it as the basis for a deep learning package.
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Haskell for data science
Similar to Haskell for data science (20)
Recently uploaded
Recently uploaded (20)
Haskell for data science
- 1. Haskell for Data Science John Cant
- 2. Haskell at a glance ● Purely functional programming language ● Compiled ● Statically typed ● Strongly typed ● Non-strict
- 3. Who on earth would use Haskell for data science? Finance industry Facebook Safety critical systems Bioinformatics Startups Various
- 5. Whirlwind tour of Haskell main = putStrLn "Hello world!"
- 6. Function application main = putStrLn "Hello world!" Function First argument
- 7. Type signatures (+) :: Num a => a -> a -> a Type constraint Arguments Return type
- 8. Partial function application (+) :: Num a => a -> a -> a (+5) :: Num a => a -> a
- 9. Higher order functions map :: (a -> b) -> [a] -> [b] GHCI> map (*2) [0, 1, 2] [0,2,4]
- 10. Data definitions data ChessGame = NotStarted | PlayerTurn Double Player BoardState | CheckMate Player BoardState
- 11. Record syntax data PersonRecord s = PersonRecord { firstName :: s , lastName :: s , personID :: Integer }
- 12. Pattern matching airportSecurity ( PersonRecord "John" n _) = "Water bottles detected, Mr. " ++ n airportSecurity _ = "Please proceed to the departure lounge "
- 13. Pattern matching data [] a = [] | a : [a] map _ [] = [] map f (x:xs) = f x : map f xs
- 14. data JSON = Object [(String, JSON)] | Array [JSON] | String String | Number Double | Bool Bool | Null
- 15. Typeclasses class ToJSON a where toJSON :: a -> JSON instance ToJSON (PersonRecord String) where toJSON (PersonRecord n0 n1 i) = ...
- 16. Evaluation of expressions foo a b = (a+b)*(a+b) foo 3 5 * + + 3 5
- 17. Evaluation of expressions * + + a b * + a b
- 18. Evaluation of expressions + 3 5 * 8 * 64
- 19. Evaluation of expressions bar = PersonRecord “bla” “bla” (bar 3 5) bar 3 5 PersonRecord “bla” “bla”
- 20. Evaluation of expressions bar 3 5 PersonRecord “bla” “bla” STOP!!!!! Success! Weak Head Normal Form
- 22. Pure functions ● Output determined only by inputs ● No side effects => Result independent of evaluation strategy Impure functions ● Randomness ● File IO ● Network ● Call impure functions ● Mutations ● Hard to reason about ● Requires reasoning
- 23. Monads Ordinary value cube :: (Floating a) => a → a cube x = x * x * x Just use the value Monad cubeM :: (Monad m, Floating a) => m a → m a cubeM mx = mx >>= (x → return x * x * x) Just use the value (inside a function you’ve bound to the monad using >>=)
- 24. Various Monad >>= implementations IO monad After the IO is performed Maybe monad If the value is not Nothing Reader, Writer, State monad Immediately List monad For each element cubeM :: (Monad m, Floating a) => m a → m a cubeM mx = mx >>= (x → return x * x * x)
- 25. class Monad m where (>>=) :: m a -> (a -> m b) -> m b return :: a -> m a -- ... Monads ● (In general) No way to extract value ● Result of >>= is m b, so no escape from m! ● Monads can function as tags in your source code
- 26. class Monad m where (>>=) :: m a -> (a -> m b) -> m b return :: a -> m a -- ... Monads ● Return representation of side effects ● Control evaluation order ● Move non-determinism away from pure code ● Tag values resulting from impure computation ● Store information between computations
- 27. Syntactic sugar: Imperative syntax ● Each line evaluated inside a function passed to >>= ● Evaluation order of lines guaranteed ● answer is the name bound to an argument of one of these functions. It is available to functions defined inside this function.
- 30. Libraries required for data science Fast Vectors, Arrays, Linear Algebra Machine learning, Deep learning Probability and statistics Big data Plotting, Graphs, Visualization
- 31. Vectors, Arrays, Linear Algebra Vec, Linear, Repa, Accelerate Use type level literals to encode dimensions of arrays (Repa, Accelerate) Use type level literals to encode length of vectors (Linear, Vec) Accelerate EDSL for running computations on the GPU! Compatability - Use data types from Linear on Accelerate backends
- 32. Machine learning LambdaNet, bpann, hfann, hnn, HaskellNN, instinct, mines, simple-neural-networks HMM, hmm, learning-hmm, markov-processes svm, svm-simple hopfield, deeplearning-hs, dnngraph HLearn genetics, GA, genprog, hgalib, moo, HSGEP, simple-genetic-algorithm, SimpleEA dtw, DynamicTimeWarp KMeans, clustering, heiraclus, Kmeans-vector, gsc-weighting, hps- kmeans, hsgsom, hierarchical-clustering estimator, Kalman
- 33. Probability and statistics ~40 packages
- 34. Big Data No spark library unfortunately Hadron Misc Hadoop libraries Haskell-HBase, ElasticSearch, Cassandra, MongoDB, Redis CloudHaskell Kafka, ZeroMQ Various DB connectors
- 35. Plotting, graphing, visualisation Many plotting libraries OpenGL implementation Elegant DSLs for writing HTML and CSS
- 38. Wide range of different plots (credit: timbod7)
- 39. Example: Density of OpenStreetMap points Raw OSM points. 78Gb uncompressed. 2.9 Billion points. Plot the density of these on a globe. Use Triangular binning because it might look cool
- 40. Data types Point data type. Just use the Vec library Triangle data type. Tuple of points A point that stores extra info, for insertion into a KD Tree
- 41. Data.Trees.KdTree Lets use our own point implementation!
- 45. Bin 3d GPS points onto a triangle in the mesh
- 46. B reaking H askell undefined unsafePerformIO IORef MVar unsafeCoerce trace Use these very carefully!
- 47. Haskell Problems Jargon, complicated types Learning curve
- 48. Haskell Problems Preventable and unpreventable bugs Runtime errors
- 49. Non exhaustive pattern match Prevent with -Wall Crashes at runtime!
- 51. Error/Exception handling error fail Error and Exception catching monads and transformers. ErrorT Maybe Either
- 52. 1/0
- 53. 1/0
- 54. Haskell Problems Space leaks Very easy to accidentally exhaust system memory
- 55. Many types of space leak http://blog.ezyang.com/2011/05/space-leak-zoo/ Enough to need their own zoo! Memory leak Strong reference leak Thunk leak Live variable leak Streaming leak Stack Overflow Selector Leak Optimization induced Leak Thread leak
- 56. Tail recursion length + 1 length + 1 length length len’ 0 len’ Thunk buildup Evaluation (1+(1+(...))) 0 1 len’ 2 Intermediate values not needed! Thunk leak Optimisation possible!
- 57. Performance This slide intentionally left blank C/C++ > Haskell > Scripting languages
- 58. Conduits
- 60. Data.IntMap Strict or Lazy variety? Persistent or Ephemeral? “The implementation is based on big-endian patricia trees. This data structure performs especially well on binary operations like union and intersection. However, my benchmarks show that it is also (much) faster on insertions and deletions when compared to a generic size-balanced map implementation (see Data.Map). ● Chris Okasaki and Andy Gill, "Fast Mergeable Integer Maps", Workshop on ML, September 1998, pages 77-86, http://citeseer.ist.psu. edu/okasaki98fast.html ○ D.R. Morrison, "/PATRICIA -- Practical Algorithm To Retrieve Information Coded In Alphanumeric/", Journal of the ACM, 15(4), October 1968, pages 514-534. “”
- 61. Data.IntMap is a persistent data structure! Result => Horrendous space leak! Fix by periodically rebuilding it. Or, give in and use a mutable vector.
- 63. Accelerate (credit: Trevor L. McDonell)
- 65. Thankyou! Q+A
Editor's Notes
- Introduction to speaker Contents
- Every value is immutable, every function result is deterministic, and without side-effects. Usually compiled to native machine code, mainly by GHC. Can also be compiled to javascript or interpreted. Statically typed. This gives the language a great deal of safety. No implicit conversion between values. Annoying for beginners In a strict language, functions require their arguments to be evaluated. In Haskell, all values are lazy by default, but explicit strictness is allowed.
- The Haskell wiki hints that there is a huge amount of proprietary, closed source financial code written in Haskell. Facebook have been doing a lot of functional programming, for instance React.js, which now has several Haskell implementations or wrappers. However, from what I’ve read, the main area they are using Haskell seems to be their data science team. BAE have used it. There are nearly 80 packages in the package index under the section Bioinformatics, more than most other sections. Oddly enough, Haskell is so underused that it’s easy to find top developers to work on your startup. Google, Microsoft, Intel, and NVidia have all used Haskell.
- Proof that it’s possible to throw something together quickly in Haskell
- What follows is a whirlwind tour of Haskell, skimming through the most important bits, and ignoring the rest of it.
- Putting expressions next to eachother applies the left one to the right one. In the case of multiple arguments, Haskell is left associative.
- This shows the type of the addition operator. The double colon denotes a type signature. Num a and anything to the left of a => is a type constraint. These allow us to be flexible about the types in our program, without compromising type safety
- You don’t have to apply a function to all of its arguments. Here, we’ve partially applied addition to the number 5, and the result was a function that accepts the remaining arguments.
- Higher order functions are functions that operate on or return other functions. Therefore all Haskell functions with arity > 1 are higher order functions. Map is a higher order function because it accepts a function as an argument. [a] denotes a list of type a. Here, we double all elements of a list by partially applying the multiplication operator to the number 2, then using map.
- Just wanted to mention the lambda syntax. The backslash is supposed to look like a lambda.
- This is equivalent to “class” in java/python or “defrecord” in clojure. ChessGame is the type, and NotStarted, PlayerTurn, and CheckMate are its constructors.
- A more convenient syntax for defining data types firstName, lastName, and personID are automatically declared as accessor functions I’ve also introduced a type variable s here. The type of Personrecord s depends on the type s. This might be useful because there are many different ways of representing a string in Haskell.
- _ = don’t care Pattern matching is useful for branching on different constructors and values. Pattern matching is useful for extracting fields for use in the function body Pattern matching is powerful. It works on lists and tuples Useful in many other places in Haskell
- Lists are just ordinary data definitions. They can be constructed by making an empty list, or by consing an element onto a list of similar elements. (:) is just a constructor. Recursive definition of map using pattern matching. At runtime, when map is applied, xs need not have been evaluated yet. This means that xs can be an infinite list! The recursion only stops when the caller stops evaluating the results that map returns. Since the result returned is in Weak Head Normal Form, f, and the recursive part of map are left unevaluated. Rarely have to define functions this way in real life. In most languages, this would be inefficient, and cause a stack overflow for lists of a certain size. In Haskell, unevaluated bindings are represented with thunks, which are lightweight and are stored on the heap.
- Suppose we define a data type to represent an AST for JSON….
- Typeclasses allow you to write type constraints. The purpose of them is polymorphism over types A typeclass is a set of types for which certain functions have been defined. The ‘instance’ declaration makes a type a member of a typeclass, and allows you to define those functions Types can be members of multiple typeclasses Typeclasses are open! Much more flexible than inheritance!
- Expressions can be represented as graphs, where each node is a value, or an unevaluated thunk. The next step is the application of graph reduction rules by the STG machine, which are based on the Lambda Calculus. The STG machine produces assembly language which carries out the operations required by these reductions. STG stands for Spineless Tagless G-machine, where G stands for graph. It lives in GHC and converts the lowest level of Haskell to Assembly language.
- Sharing is applied to avoid recomputing the same values
- The graph is reduced. Function application is a type of reduction. It’s more complicated than this in real life.
- Expressions can be represented as graphs, where each node is a value, or an unevaluated thunk. The next step is the application of graph reduction rules
- If the top level of a graph is a constructor, the graph is said to be in Weak Head Normal Form. This is used as the return value, and the lower levels are evaluated lazily, as needed. This is how laziness is implemented in Haskell
- This symbol is called Bottom. Bottom is an expression that can’t be evaluated, due to an infinite loop or an error. Bottom is a member of every type.
- This is pronounced “bind”
- In this slide I’m comparing monads to ordinary plain values. Monad is a typeclass whose instances define certain functions including >>= and return. The purpose of >>= is to allow us to provide a function that uses the plain value, and allow the Monad implementation to control how that function is called. We choose how to use the value inside the function, but the Monad’s implementation chooses how and when to call our function.
- Different Monad instances implement >>= differently, and will call your function in different ways.
- Can’t extract plain value in general. No way of using a and returning something of any other form than m b, in general.
- If you return a representation of your side effects, they are no longer really side-effects, they are the main return value of your function getLine reads a line from STDIN. The value it returns does not directly contain the value it has read, so it can be considered pure. Furthermore, chained, use of >>= constructs a chain of dependent computations. This guarantees the evaluation order.
- GHCI: Evaluate expressions, determine their types, inspect modules Hoogle: Search for functions by their type signature Cabal ~ a bit like make + package installer Hackage is a package DB. Like most language package databases, contains a large amount of unmaintained, disused, abandonware. Unlike most language-specific package databases, this abandonware has a good chance of still working! Fay, Haste, and GHCJS are Haskell to Javascript compilers.
- Neural nets, Markov models, SVMs, Hopfield network, Restricted Boltzmann Machines, a Convnet, genetic algorithms, dynamic time warping, Many different clustering algorithms, and Kalman filters. Unfortunately, none of the neural networks are capable of using the GPU yet, but one of my side projects is to build a deep learning library on top of Accelerate. The author of deeplearning-hs works for Facebook AI research and has experimented with Accelerate. dnngraph can generate models for Caffe and Torch.
- Many different packages for probability and statistics. You can also call R from Haskell
- No spark library unfortunately. Various libraries for interacting with Hadoop, but only two libraries for running Haskell on Hadoop Hadron. Hadron uses MapReduce streaming, and conduits. It requires Haskell to be installed on every node. Cloud Haskell is a distributed concurrency framework. There are also various connectors to the usual suspects like MySQL and PostgreSQL.
- Haskell has many graph plotting libraries. OpenGL and Haskell are an odd combination, but there are bindings, and they use Monads to represent the internal OpenGL state Haskell is very good for writing declarative DSLs. There are libraries for writing HTML and CSS using the do notation. The result is checked, and it looks very much like HAML or SASS.
- This generates a scatter plot and outputs it to a file. You could easily do this from GHCI
- Here’s some other the other plots this library is capable of. There are many other plotting libraries.
- We’ll generate a spherical mesh using recursion
- Octahedron. This is supposed to give you fairly even triangles. Turns out that if you start with a Tetrahedron, then after the first refinement on a given face, the middle triangle is twice the area of each of the other triangles. Also, Octahedrons are really easy to hardcode.
- The do notation uses the Maybe Monad. If a line returns Nothing, then the rest of the block is skipped. We’ll find the nearest neighbor and inspec the triangles it’s part of, binning the point into the nearest one. We’ll take a break now to look at some hacks and some problems we come across in Haskell
- Here are some functions and types that are considered harmful, but have their uses. Please use them carefully. undefined is bottom and if your program tries to evaluate it, it will crash at runtime. However, it can inhabit any type. This is useful for making dummy values to solve type errors in GHCI. unsafePerformIO extracts a value from an IO monad by performing the IO. Using this function can introduce impurity into pure functions, resulting in undefined behaviour IORef and MVar are mutable variables. Excessive use of these defeats the point of functional programming. unsafeCoerce changes the type of a value without changing the value. This can cause segfaults. trace wraps a unary function, printing out a string when it is evaluated
- Haskell has quite a steep learning curve, because it has too much confusing jargon and type system complications. Monad tutorial fallacy. People imagine that there’s a single explanation they helped them grasp a concept, whereas this explanation was actually the last one they read.
- Despite ensuring type correctness and guarding against race conditions with immutability, run time errors are still possible. You can cause runtime errors by incorrectly using the FFI, or by not covering all cases in a pattern match, or simply by throwing an exception
- If you don’t catch all cases in a pattern match, your program might bork at runtime. Annoyingly, this could be prevented at compile time, but only by setting a compiler flag
- Haskell has Exceptions
- error is sugar for undefined. It’s designed to represent a programming error, and bork with a message. Unfortunately it’s often used, so at some point you might need to catch it. Exceptions, however, are just data types, where the infrastructure required to throw and catch them is provided by a library. fail is a method which must be implemented by all monads. In the IO monad in Haskell 98, fail calls error, but the Maybe monad has a sensible implementation. fail’s presence in the language is generally regarded as a bad design decision and should be avoided. There are many abstractions for dealing with different errors and exceptions, and many differen monads. The monad transformer errorT is a good way of wrapping a monad with error and exception handling. Maybe and Either are elegant ways of handling computations that fail. They carry a result, or alternatively, some failure information. There are too many disparate ways of expressing and handling errors in Haskell.
- If you try and divide by 0, you get infinity rather than an error. Haskell could be safer if division returned a Maybe.
- If you really wanted to, you could redefine division to explicitly handle division by 0, and use -Wall to make you handle it at compile time.
- Space leaks are a problem. It’s very easy to consume crazy amounts of memory in Haskell.
- There are so many different types of space leak that they need their own zoo. You’ll probably come across that page at some point.
- Suppose we want to find the length of a lazy list. The elegant but naive implementation on the left is slow, leaks memory, and overflows the stack. The length of xs will be evaluated first, then the addition of 1. This causes traversal to the end of the list, building up thunks. When the end of the list is reached, the thunks must be evaluated in turn, then they can be freed. This is a lot of work just to perform simple addition. In the implementation on the right, we recurse last rather than first. len’ is tail recursive. This means that the recursive call is the last (or outermost) computation. This can be optimised into iteration which does not build a chain of thunks.
- Haven’t bothered with benchmarks, in case they might not be meaningful. For a given task, Haskell can be slower, because of unnecessary copying that might occur. However, due to laziness, it can skip unnecessary computation, so it’s possible that a Haskell implementation could be faster than a C/C++ implementation!
- Conduits are a way of piping data around in constant memory. Sources are conduits that only produce and Sinks are conduits that only consume. await consumes a value from the input, and yield produces a value. The conduit recurses on itself
- await consumes yield produces Recurse Turns out this leaks memory like a seive, then crashes, unless you have vast amounts of RAM, because of a bad choice of data structure
- Big Endian Patricia Trees are persistent data structures, so IntMap stores it’s entire state history!
- The moral of the story is, it’s sometimes OK to use mutable data structures in Haskell.
- Success! Triangular map of the density of heatmap points in OSM.
- This is how to multiply two matrices in Accelerate, on the GPU. It works by replicating the matrices into 3d arrays, and transposing one of them, so that matrix multiplication is an elementwise product and a summation. In practice, GPU cycles are not wasted replicating the matrices, thanks to fusion that takes place in Accelerate. One of my side projects is to implement symbolic differentiation for Accelerate, so that it’s easy to implement Deep learning, and you don’t have to spend time differentiating by hand when you want to add LSTMs or GRUs to your network.