Probabilistic Programming Systems aim to merge general purpose programming with probabilistic modelling. They provide powerful statistical inference and thus allow developers to focus on the modelling with tools and environments they are comfortable with. These emerging methods are promising additions to the Data Scientist’s toolbox and an interesting, satisfying playground for programming enthusiasts. This talk is an introduction to Probabilistic Programming Systems, their use and value for the industry and Clojure’s great library “Anglican”.

1. Intro to Probabilistic Programming
and Clojure’s Anglican
Dr. Nils Blum-Oeste
EuroClojure 2017, Berlin

2. Who is talking?
Probabilistic Programming:
Why? What? How?
Anglican: Examples

3. Physics
Chemistry
Quantitative Modelling
Complex Systems

4. 😍

5. Smart Home Energy Model

6. Probabilistic Programming
Why? What? How?

7. Predict the future
Infer causes of observations
Improve models from evidence
Why?

8. Compared to other machine learning techniques
incorporates domain knowledge
models are explainable and
understandable
works well with small to medium data
need to build a model
computationally expensive (aka “slow”)
+ -

9. Don’t think of Probabilistic
Programming vs other techniques,
they can complement each other

10. What is a Probabilistic
Programming System?

11. Bayesian Probability: The belief or
confidence in an event occurring

12. Bayes Rule
Marginal Likelihood
PriorLikelihood
Posterior
Hypothesis Evidence

15. Markov chain Monte Carlo
Sampling

17. sample at n fixed positions
too expensive for multiple
dimensions

18. random sample
next sample
random step

19. sample density reflects
probabilities
works for many dimensions

20. Probabilistic Reasoning System
Probabilistic Model
Inference Algorithm
Evidence
Queries
Answer

21. Probabilistic Programming System
Probabilistic Model
Inference Algorithm
Evidence
Queries
Answer
code
provided by PPS
data
code
data

22. Execution model
probabilistic programs are
executed many times for inference
random choices affect the
execution path on each run
computes the distribution from
which execution results are drawn

23. Anglican Examples

24. What is Anglican?
embedded language, subset of
Clojure
same syntax but different
semantics
Clojure, JVM and Anglican
available to each other
Anglican runs wherever
you have a JVM

25. A probabilistic Hello World:
Coin Toss

26. Sampling a fair coin with Anglican

27. Learning the bias of a coin

28. Posterior distribution after observing 20 heads and 20 tails

29. Example: AB-Test

30. Simulate user interactions

31. Probabilistic model

32. Posterior sampling

34. Example: Bayesian Inference
on Game Physics

38. The whole Clojure world available…
No guarantee for success though!

39. PPS beyond Anglican
Clojure, high performance:
Bayadera
Industry Standards: Stan, PyMC
Many PPS: Figaro, WebPPL,
BayesDB, …

40. Probabilistic Programming, my conclusions so far:
Probabilistic Programming Systems: Very low entry barrier to predict, infer
and train domain models, but modelling requires skill
Anglican: Solid solution in Clojure space with great modelling and
prototyping capabilities, but inference is slow and ecosystem is lacking
Research Topic: Expect maturation and extension of the probabilistic
programming techniques

41. Acknowledgements
Anglican Devs: Frank Wood, David Tolpin & all contributors
Speaker Mentor: Jan Stępień
Discussions: Dragan Djuric, Chris Wallace, David Tolpin, Christian Weilbach

42. More Information
http://www.robots.ox.ac.uk/~fwood/anglican/index.html
http://probabilistic-programming.org