Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Jake Mannix, Lead Data Engineer, Lu... by MLconf 495 views
- Sam Steingold, Lead Data Scientist,... by MLconf 626 views
- Jason Baldridge, Associate Professo... by MLconf 655 views
- Jennifer Prendki, Principal Data Sc... by MLconf 674 views
- Igor Markov, Software Engineer, Goo... by MLconf 803 views
- Dr. Erin LeDell, Machine Learning S... by MLconf 703 views

697 views

Published on

In this talk I will present RELOOP, a declarative mathematical programming language embedded into Python. It allows the user to specify mathematical programs before she knows what individuals are in the domain and, therefore, before she knows what variables and constraints exist. It facilitates the formulation of abstract, general knowledge. And, it reveals the rich logical structure underlying many machine learning problems to the solver and, turn, may make it go faster.

With this, people can start to rapidly develop statistical machine learning approaches for complex data. For instance, adding just three lines of RELOOP code makes a linear support vector machines aware of any underlying network that connects the objects to be classified.

Joint work with Martin Mladenov and many others.

Published in:
Technology

No Downloads

Total views

697

On SlideShare

0

From Embeds

0

Number of Embeds

7

Shares

0

Downloads

18

Comments

0

Likes

2

No embeds

No notes for slide

- 1. Declarative Programming for Statistical ML Kristian Kersting Martiin Mladenov TUD Babak Ahmadi PicoEgo Amir Globerson HUJI Martin Grohe RWTH Sriraam Natarajan U. Indiana Leonard Kleinhans TUD Danny Heinrich TUD andmany more… Pavel Tokmakov INRIA Grenoble
- 2. Is there a “-01” flag for Statistical ML? Kristian Kersting Martiin Mladenov TUD Babak Ahmadi PicoEgo Amir Globerson HUJI Martin Grohe RWTH Sriraam Natarajan U. Indiana Leonard Kleinhans TUD Danny Heinrich TUD andmany more… Pavel Tokmakov INRIA Grenoble
- 3. Kristian Kersting - Declarative Programming for Statistical ML
- 4. There is an arms race to “deeply” understand data Kristian Kersting - Declarative Programming for Statistical ML
- 5. Kristian Kersting - Declarative Programming for Statistical ML Take your spreadsheet … Features Objects
- 6. Latent Dirichlet AllocationMatrix Factorization Features Objects … and apply some ML Gaussian Processes Decision Trees/Boosting Autoencoder/Deep Learning and many more … Kristian Kersting - Declarative Programming for Statistical ML Support Vector Machines
- 7. IS IT REALLY THAT SIMPLE? Kristian Kersting - Declarative Programming for Statistical ML
- 8. Guy van den Broeck UCLA
- 9. Guy van den Broeck UCLA card (1,d2) card (1,d3) card (1,pAce) card (52,d2) card (52,d3) card (52,pAce) … … … …
- 10. Guy van den Broeck UCLA card (1,d2) card (1,d3) card (1,pAce) card (52,d2) card (52,d3) card (52,pAce) … … … …
- 11. Guy van den Broeck UCLA No independencies. Fully connected. 22704 states card (1,d2) card (1,d3) card (1,pAce) card (52,d2) card (52,d3) card (52,pAce) … … … … Kristian Kersting - Declarative Programming for Statistical ML
- 12. Guy van den Broeck UCLA A machine will not solve the problem card (1,d2) card (1,d3) card (1,pAce) card (52,d2) card (52,d3) card (52,pAce) … … … …
- 13. Faster modelling Faster inference and learning
- 14. Kristian Kersting - Declarative Programming for Statistical ML Symmetry-Aware Message Passing Compress the model Run message passing inference on the smaller model [Singla, Domingos AAAI’08; Kersting, Ahmadi, Natarajan UAI’09; Ahmadi, Kersting, Mladenov, Natarajan MLJ’13; Mladenov, Globerson, Kersting AISTATS `14, UAI `14; Mladenov, Kersting UAI´15; ...] Big model small model
- 15. Kristian Kersting - Declarative Programming for Statistical ML De Raedt, Kersting, Natarajan, Poole “Statistical Relational Artificial Intelligence”,2016 … the study and design of intelligent agents that act in noisy worlds composed of objects and relations among the objects Statistical Relational AI ScalingUncertainty Logic Graphs Trees Mining And Learning [Getoor, Taskar MIT Press ’07; De Raedt, Frasconi, Kersting, Muggleton, LNCS’08; Domingos, Lowd Morgan Claypool ’09; Natarajan, Kersting, Khot, Shavlik Springer Brief’15; Russell CACM 58(7): 88-97 ’15]
- 16. BUT WAIT A MINUTE! WE WANT TO USE SOME ML, NOT JUST GRAPHICAL MODELS! Kristian Kersting - Declarative Programming for Statistical ML Latent Dirichlet Allocation Matrix Factorization Gaussian Processes Decision Trees/Boosting Autoencoder/Deep Learning and many more … Support Vector Machines
- 17. Let’s say we want to classify publications that cite each other
- 18. This is a quadratic program. If you replace l2- by l1-,l∞-norm you get a linear program Standard ML approach: Support Vector Machines Kristian Kersting - Declarative Programming for Statistical ML [Vapnik ´79; Bennett´99; Mangasarian´99; Zhou, Zhang, Jiao´02, ... ]
- 19. Write down the problem in „paper form“. The machine then compiles automatically into algebraic solver form. Statistical Machine Learning via Declarative Programming [Kersting, Mladenov, Tokmakov AIJ´15, Mladenov, Heinrich, Kleinhans, Gonsio, Kersting DeLBP´16] Logically parameterized variable (set of ground variables) Logically parameterized constraint Logically parameterized objective Data stored externally
- 20. Program1 Data1 Program2 Data2 Program3 Data3 ... Kristian Kersting - Declarative Programming for Statistical ML [Kersting, Mladenov, Tokmakov AIJ´15, Mladenov, Heinrich, Kleinhans, Gonsio, Kersting DeLBP´16] Captures the essence of a problem and can be reused for several problems
- 21. MP1 Declarative Program MP2 MPn Data1 Data2 Datan ... Kristian Kersting - Declarative Programming for Statistical ML [Kersting, Mladenov, Tokmakov AIJ´15, Mladenov, Heinrich, Kleinhans, Gonsio, Kersting DeLBP´16] Captures the essence of a problem and can be reused for several problems
- 22. But wait, publications are citing each other. OMG, I have to use graph kernels! REALLY?
- 23. Simply program some additional constraints [Kersting, Mladenov, Tokmakov AIJ´15, Mladenov, Heinrich, Kleinhans, Gonsio, Kersting DeLBP´16] Kristian Kersting - Declarative Programming for Statistical ML
- 24. http://www-ai.cs.uni-dortmund.de/weblab/static/RLP/html/ … Kristian Kersting - Declarative Programming for Statistical ML Loops and relations get interwined, and models can refer to each other DBMS Interface Using a probabilistic programming language we can even get stochastic relational mathematical programs
- 25. Finally, the „-O1“ flag [Kersting, Mladenov, Tokmakov AIJ 2015, Mladenov, Kleinhans, Kersting 2016] Kristian Kersting - Declarative Programming for Statistical ML (1) Reduce the QP via symmetries (2) Run any solver on the reduced QP
- 26. … and the “-02” flag Kristian Kersting - Declarative Programming for Statistical ML Algebraic Decision Diagrams Formulae parse trees Matrix Free Optimization ( )+ = Optimization with 60 Millions of non-zeros with 12 minutes per log-barrier iteration and actually sublinear in the number of non-zeros
- 27. HIGH-LEVEL LANGUAGES FOR MACHINE LEARNING AND OPTIMIZATION ARE A STEP TOWARDS THE … Conclusions Kristian Kersting - Declarative Programming for Statistical ML
- 28. DEMOCRATIZATION OF MACHINE LEARNING Kristian Kersting - Declarative Programming for Statistical ML Reduces the level of expertise necessary to build optimization applications, makes models faster to write and easier to communicate Facilitate the construction of sophisticated models with rich domain knowledge Speed up solvers by exploiting language properties, compression, and compilation
- 29. Kristian Kersting - Declarative Programming for Statistical ML
- 30. Kristian Kersting - Declarative Programming for Statistical ML

No public clipboards found for this slide

Be the first to comment