Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Probabilistic Programming in Scala

6,346 views

Published on

Published in: Technology, Education
  • Writing essays, letters and stories increases the learning abilities of students. That's why writing is much necessary in schools and colleges. Students should not feel burden writing work and must feel that it's not something which they can't do. It can be done with little pain. Try this HelpWriting.net.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/39sFWPG ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ♥♥♥ http://bit.ly/39sFWPG ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Probabilistic Programming in Scala

  1. 1. Probabilistic Programming in Scala Guy Van den Broeck guy.vandenbroeck@cs.kuleuven.be Computer Science Department Katholieke Universiteit Leuven September 13, 2011
  2. 2. What is a PPL? Probabilistic graphical models (Bayesian networks) important in machine learning, statistics, robotics, vision, biology, neuroscience, artificial intelligence (AI) and cognitive science. Probabilistic Programming Languages unify general purpose programming with probabilistic modeling Examples Functional, extending Scheme (Church) or Scala (Figaro, FACTORIE, ScalaPPL) Logical, extending Prolog (ProbLog, PRISM, BLOG, Dyna) Extending C# (Infer.NET) Tasks Compute probabilities, most likely assignments given observations Learn parameters and programs Applications in natural language processing, computer vision, machine learning, bioinformatics, probabilistic planning, seismic monitoring, . . .
  3. 3. PPLs: The Easy Case Idea behind functional PPLs Any function (A, B, . . .) => R can also operate on probability distributions (Distr [A], Distr [B], . . .) => Distr [R] Starting Point Many library functions operate on Booleans: &&, ||, !, ==, != exists, forall Idea 1. Make a version of those functions that operates on distributions and returns a Distr[Boolean]. 2. Extend with probabilistic data structures. 3. Use them to model complex probability distributions.
  4. 4. PPLs: The Easy Case Idea behind functional PPLs Any function (A, B, . . .) => R can also operate on probability distributions (Distr [A], Distr [B], . . .) => Distr [R] Starting Point Many library functions operate on Booleans: &&, ||, !, ==, != exists, forall Idea 1. Make a version of those functions that operates on distributions and returns a Distr[Boolean]. 2. Extend with probabilistic data structures. 3. Use them to model complex probability distributions.
  5. 5. PPLs: The Easy Case Idea behind functional PPLs Any function (A, B, . . .) => R can also operate on probability distributions (Distr [A], Distr [B], . . .) => Distr [R] Starting Point Many library functions operate on Booleans: &&, ||, !, ==, != exists, forall Idea 1. Make a version of those functions that operates on distributions and returns a Distr[Boolean]. 2. Extend with probabilistic data structures. 3. Use them to model complex probability distributions.
  6. 6. PPLs: The Easy Case Idea behind functional PPLs Any function (A, B, . . .) => R can also operate on probability distributions (Distr [A], Distr [B], . . .) => Distr [R] Starting Point Many library functions operate on Booleans: &&, ||, !, ==, != exists, forall Idea 1. Make a version of those functions that operates on distributions and returns a Distr[Boolean]. 2. Extend with probabilistic data structures. 3. Use them to model complex probability distributions.
  7. 7. PPLs: The Easy Case Idea behind functional PPLs Any function (A, B, . . .) => R can also operate on probability distributions (Distr [A], Distr [B], . . .) => Distr [R] Starting Point Many library functions operate on Booleans: &&, ||, !, ==, != exists, forall Idea 1. Make a version of those functions that operates on distributions and returns a Distr[Boolean]. 2. Extend with probabilistic data structures. 3. Use them to model complex probability distributions.
  8. 8. Examples
  9. 9. Boolean formulae Random Variables are objects (each object independent) val a = Flip(0.3) val b = Flip(0.6) BooleanDistr has member functions that build Formula objects. val xor = a && !b || !a && b Run inference on BooleanDistr println("Probability = " + xor.probability()) Probability = 0.54
  10. 10. Boolean formulae Random Variables are objects (each object independent) val a = Flip(0.3) val b = Flip(0.6) BooleanDistr has member functions that build Formula objects. val xor = a && !b || !a && b Run inference on BooleanDistr println("Probability = " + xor.probability()) Probability = 0.54
  11. 11. Boolean formulae Random Variables are objects (each object independent) val a = Flip(0.3) val b = Flip(0.6) BooleanDistr has member functions that build Formula objects. val xor = a && !b || !a && b Run inference on BooleanDistr println("Probability = " + xor.probability()) Probability = 0.54
  12. 12. Boolean formulae Random Variables are objects (each object independent) val a = Flip(0.3) val b = Flip(0.6) BooleanDistr has member functions that build Formula objects. val xor = a && !b || !a && b Run inference on BooleanDistr println("Probability = " + xor.probability()) Probability = 0.54
  13. 13. 2-state weather HMM abstract class Timestep { def rainy: BooleanDistr def umbrella = If(rainy, Flip(0.9), Flip(0.1)) } object StartState extends Timestep { val rainy = Flip(0.2) } class SuccessorState(predecessor: Timestep) extends Timestep { val rainy = If(predecessor.rainy, Flip(0.5), Flip(0.1)) } var timestep: Timestep = StartState for(i <- 1 until 2000) timestep = new SuccessorState(timestep) println("Probability = " + timestep.umbrella.probability())
  14. 14. 2-state weather HMM abstract class Timestep { def rainy: BooleanDistr def umbrella = If(rainy, Flip(0.9), Flip(0.1)) } object StartState extends Timestep { val rainy = Flip(0.2) } class SuccessorState(predecessor: Timestep) extends Timestep { val rainy = If(predecessor.rainy, Flip(0.5), Flip(0.1)) } var timestep: Timestep = StartState for(i <- 1 until 2000) timestep = new SuccessorState(timestep) println("Probability = " + timestep.umbrella.probability())
  15. 15. 2-state weather HMM abstract class Timestep { def rainy: BooleanDistr def umbrella = If(rainy, Flip(0.9), Flip(0.1)) } object StartState extends Timestep { val rainy = Flip(0.2) } class SuccessorState(predecessor: Timestep) extends Timestep { val rainy = If(predecessor.rainy, Flip(0.5), Flip(0.1)) } var timestep: Timestep = StartState for(i <- 1 until 2000) timestep = new SuccessorState(timestep) println("Probability = " + timestep.umbrella.probability())
  16. 16. Probabilistic Data Structures Probabilistic List: objects are in the list with a certain probability, given by a BooleanDistr trait ListDistr[T] extends Distribution[List[T]]{ def forall(f: T => BooleanDistr) : BooleanDistr def exists(f: T => BooleanDistr) : BooleanDistr } Replace all Boolean by BooleanDistr in member functions Many possibilities (Set,Tree,. . . )
  17. 17. Probabilistic Graphs Viral marketing Learning biological pathways Spread of influence in social networks class Person { val influencedFriends = new ListDistr[Person] def influences(target: Person): BooleanDistr = { if(target == this) True else friends.exists(_.influences(target)) } } val p1,p2,p3,p4,p5,p6 = new Person val influence1to2 = Flip(0.9) n1.influencedFriends += (influence1to2, p2) n2.influencedFriends += (influence1to2, p1) ... println("Probability = " + p1.influences(p4).probability())
  18. 18. Probabilistic Graphs Viral marketing Learning biological pathways Spread of influence in social networks class Person { val influencedFriends = new ListDistr[Person] def influences(target: Person): BooleanDistr = { if(target == this) True else friends.exists(_.influences(target)) } } val p1,p2,p3,p4,p5,p6 = new Person val influence1to2 = Flip(0.9) n1.influencedFriends += (influence1to2, p2) n2.influencedFriends += (influence1to2, p1) ... println("Probability = " + p1.influences(p4).probability())
  19. 19. Probabilistic Graphs Viral marketing Learning biological pathways Spread of influence in social networks class Person { val influencedFriends = new ListDistr[Person] def influences(target: Person): BooleanDistr = { if(target == this) True else friends.exists(_.influences(target)) } } val p1,p2,p3,p4,p5,p6 = new Person val influence1to2 = Flip(0.9) n1.influencedFriends += (influence1to2, p2) n2.influencedFriends += (influence1to2, p1) ... println("Probability = " + p1.influences(p4).probability())
  20. 20. Probabilistic Graphs Viral marketing Learning biological pathways Spread of influence in social networks class Person { val influencedFriends = new ListDistr[Person] def influences(target: Person): BooleanDistr = { if(target == this) True else friends.exists(_.influences(target)) } } val p1,p2,p3,p4,p5,p6 = new Person val influence1to2 = Flip(0.9) n1.influencedFriends += (influence1to2, p2) n2.influencedFriends += (influence1to2, p1) ... println("Probability = " + p1.influences(p4).probability())
  21. 21. Probabilistic Values Model any discrete distribution class ValDistr[T] extends Distribution[T]{ def ==(v: T): BooleanDistr def map[R](f: T => ValDistr[R]): ValDistr[R] } Apply any deterministic function to ValDistr arguments def Apply[A,R](f: (A) => R)(a: ValDistr[A]): ValDistr[R] def Apply[A,B,R](f: (A,B) => R) ... ... Example Two dice: probability that their sum is 8? val die1 = Uniform(1 to 6) val die2 = Uniform(1 to 6) val sum = Apply(_+_)(die1,die2) println("Probability = " + (sum == 8).probability())
  22. 22. Probabilistic Values Model any discrete distribution class ValDistr[T] extends Distribution[T]{ def ==(v: T): BooleanDistr def map[R](f: T => ValDistr[R]): ValDistr[R] } Apply any deterministic function to ValDistr arguments def Apply[A,R](f: (A) => R)(a: ValDistr[A]): ValDistr[R] def Apply[A,B,R](f: (A,B) => R) ... ... Example Two dice: probability that their sum is 8? val die1 = Uniform(1 to 6) val die2 = Uniform(1 to 6) val sum = Apply(_+_)(die1,die2) println("Probability = " + (sum == 8).probability())
  23. 23. Probabilistic Values Model any discrete distribution class ValDistr[T] extends Distribution[T]{ def ==(v: T): BooleanDistr def map[R](f: T => ValDistr[R]): ValDistr[R] } Apply any deterministic function to ValDistr arguments def Apply[A,R](f: (A) => R)(a: ValDistr[A]): ValDistr[R] def Apply[A,B,R](f: (A,B) => R) ... ... Example Two dice: probability that their sum is 8? val die1 = Uniform(1 to 6) val die2 = Uniform(1 to 6) val sum = Apply(_+_)(die1,die2) println("Probability = " + (sum == 8).probability())
  24. 24. 3-state weather HMM object Sunny extends Weather object Foggy extends Weather object Rainy extends Weather abstract class Timestep { def weather: ValDistr[Weather] def umbrella = weather.map{ case Sunny => Flip(0.1) case Foggy => Flip(0.3) case Rainy => Flip(0.8) } } object StartState extends Timestep { val weather = ValDistr((0.3,Sunny), (0.3,Foggy), (0.4,Rainy)) } class SuccessorState(predecessor: Timestep) extends Timestep { val weather = predecessor.weather.map{ case Sunny => ValDistr((0.8,Sunny), (0.15,Foggy), (0.05,Rainy)) case Foggy => ValDistr((0.05,Sunny), (0.3,Foggy), (0.65,Rainy)) case Rainy => ValDistr((0.15,Sunny), (0.35,Foggy), (0.5,Rainy)) } }
  25. 25. Why Scala for Probabilistic Programming? Probabilistic programming as a library No separate compiler or VM mixin DSL Higher-order functions and generics Pass existing deterministic code to probabilistic model Memoization for efficient inference Object-oriented easy to model probabilistic databases Other advantages carry over to the probabilistic case
  26. 26. Thanks

×