Predictive APIs are making it easier to integrate Machine Learning in your apps and to add predictive features to them. Starting with some basics we'll see what the different types of APIs are and we'll give some examples of proprietary predictive APIs. We'll go over some ways of exposing your own predictive models as APIs served by 3rd party platforms, and open source frameworks for creating and serving your own APIs on your infrastructure of choice. We'll give some remarks on recent (and missing) tools to make it easier to use and compare all these APIs. Finally, we'll give some pointers to a Virtual Machine to help you get started with these technologies... Slides from my talk at the Valencian Summer School on Machine Learning (#VSMML15)

1. A developers’ overview of the world of predictive APIs
(proprietary, open source, hybrid)
Louis Dorard
PAPIs.io

2. Predictive APIs basics
Types of Predictive APIs
Deploy own model on 3rd party platform
Open source PAPI frameworks
What’s missing / almost there

3. Predictive APIs basics

5. The two phases of machine learning:
• TRAIN a model
• PREDICT with a model

6. The two methods of predictive APIs:
• TRAIN a model
• PREDICT with a model

7. The two methods of predictive APIs:
• model = create_model(dataset)
• predicted_output =
create_prediction(model, new_input)

8. The two methods of predictive APIs:
• model = create_model(‘training.csv’)
• predicted_output =
create_prediction(model, new_input)

9. from bigml.api import BigML
# create a model
api = BigML()
source = api.create_source('training_data.csv')
dataset = api.create_dataset(source)
model = api.create_model(dataset)
# make a prediction
prediction = api.create_prediction(model, new_input)
print "Predicted output value: ",prediction['object']
['output']
BigML Python

10. # from http://scikit-learn.org/stable/tutorial/
basic/tutorial.html
from sklearn import svm
model = svm.SVC(gamma=0.001, C=100.)
from sklearn import datasets
digits = datasets.load_digits()
model.fit(digits.data[:-1], digits.target[:-1])
model.predict(digits.data[-1])
Scikit Python

11. $ curl https://bigml.io/model?$BIGML_AUTH
-X POST
-H "content-type: application/json"
-d '{"dataset": "dataset/50ca447b3b56356ae0000029"}'
BigML REST http

12. The two methods of predictive APIs:
• model = create_model(‘training.csv’)
• predicted_output =
create_prediction(model, new_input)

13. The two methods of predictive APIs:
• model = create_model(‘training.csv’)
• predicted_output =
create_prediction(model, new_input)

14. Types of Predictive APIs

15. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

16. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

17. “Is this email important?
— Yes/No”

18. Generic Predictive APIs:
• Google Prediction API
• BigML
• Datagami
• PredicSis
• WolframCloud
• Amazon

19. AMAZON GOOGLE PREDICSIS BIGML
ACCURACY 0.862 0.743 0.858 0.790
TRAINING
TIME
135s 76s 17s 5s
TEST TIME 188s 369s 5s 1s
louisdorard.com/blog/machine-learning-apis-comparison

20. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

21. “Which project does this document relate to?
— A/B/C”

22. Text classification APIs:
• uClassify.com
• MonkeyLearn.com
• Cortical.io

23. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

24. “Is this customer going to leave next month?
— Yes/No”

25. “Is this transaction fraudulent?
— Yes/No”

26. Problem-specific Predictive APIs:
• Churn: ChurnSpotter.io, Framed.io
• Fraud detection: SiftScience.com
• Lead scoring: Infer.com
• Personal assistant (Siri): Wit.ai

27. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

28. “What is the sentiment of this tweet?
— Positive/Neutral/Negative”

29. “Is this email spam?
— Yes/No”

30. Fixed-model Predictive APIs:
• Text: Datumbox.com, Semantria.com
• Vision: Indico, AlchemyAPI.com
• Siri-like: maluuba.com

31. ABSTRACTION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

32. CUSTOMIZATION
Algorithmic
Generic
Generic for text
Problem-specific
Fixed-model

33. Deploy your own model
on 3rd party platform

34. EXPERIMENT / DEPLOY
Find model Use model

35. Start with {popular, open source} ML library

36. Experiment in automated environment

42. Experiment on “ScienceCluster”
• Distributed jobs
• Collaborative workspace
• Serialize chosen model
Deploy model as API on “ScienceOps”
• Load balancing
• Auto scaling
• Monitoring (API calls, accuracy)

43. 43
• 1 for serving predictions
• 1 for running ML experiment (i.e. train and evaluate
models on given data)?
• 1 for deploying ML models?
Your API endpoints

44. Open source PAPI
frameworks

46. • “Open source prediction server”
• Based on Spark
• Expose predictive models as (scalable
& robust) APIs

47. MLlib

49. = PAPI+

50. • Train command: “pio train”
• API endpoints:
• Send (new) data
• Send prediction queries

52. Deployment
• Amazon CloudFormation template
→ cluster
• Manual up/down scaling

56. What’s missing /
almost there

57. 57
• Spearmint: “Bayesian optimization” for tuning
parameters → Whetlab → Twitter
• Auto-sklearn: “automated machine learning toolkit
and drop-in replacement for a scikit-learn
estimator”
• See automl.org and challenge
Open Source AutoML?!

58. from sklearn import svm
model = svm.SVC(gamma=0.001, C=100.)
from sklearn import datasets
digits = datasets.load_digits()
model.fit(digits.data[:-1], digits.target[:-1])
model.predict(digits.data[-1])
Scikit Python

59. from sklearn import svm
model = svm.SVC(gamma=0.001, C=100.)
from sklearn import datasets
digits = datasets.load_digits()
model.fit(digits.data[:-1], digits.target[:-1])
model.predict(digits.data[-1])
Scikit Python

60. import autosklearn
model = autosklearn.AutoSklearnClassifier()
from sklearn import datasets
digits = datasets.load_digits()
model.fit(digits.data[:-1], digits.target[:-1])
model.predict(digits.data[-1])
AutoML Scikit

61. 61
• ?
Open Source Auto Scaling?!

62. AMAZON GOOGLE PREDICSIS BIGML
ACCURACY 0.862 0.743 0.858 0.790
TRAINING
TIME
135s 76s 17s 5s
TEST TIME 188s 369s 5s 1s
louisdorard.com/blog/machine-learning-apis-comparison

63. 63
• Requirement:
• train/test splits on local machine
• compute evaluation on local machine
• Solutions
• adapt bigmler and use local evaluations?
• use scikit-learn framework?
Automated Benchmark?!

64. 64
• Python defacto standard: scikit-learn
• “Sparkit-learn aims to provide scikit-
learn functionality and API on PySpark. The main goal of the
library is to create an API that stays close to sklearn’s."
• REST standard: PSI (Protocols & Structures for Inference)
• Pretty similar to BigML API!
• Implementation for scikit available
• Easy benchmarking! Ensembles!
API standards?!

65. 65
• VM with Jupyter notebooks (Python & Bash)
• API wrappers preinstalled: BigML & Google Pred
• Notebook for easy setup of credentials
• Scikit-learn and Pandas preinstalled
• Open source VM provisioning script & notebooks
• Search public Snaps on terminal.com: “machine learning”
Getting started

66. @louisdorard
@papisdotio
louisdorard.com
papis.io