Presented by Kyle Polich, Host at Data Skeptic

1. Are API services
taking over all the interesting
data science problems?

2. Very short “hello world” examples
https://github.com/kylepolich/apis-talk

3. What is an API?
Application Programming Interface
“a set of subroutine definitions, protocols, and
tools for building application software”
- Wikipedia

4. APIs described in this talk
I’m speaking about a subset of this definition.
My use is the colloquial use.
Client
Client
Server
request
response
request
response

5. Example: Wikipedia pageview API
>>> from mwviews.api import PageviewsClient
>>> p = PageviewsClient()
>>> response = p.article_views('en.wikipedia', ['Machine_Learning'],
start='20170101', end='20170107')
>>> print(response)
defaultdict(<type 'dict'>,
{datetime.datetime(2017, 1, 4, 0, 0): {'Machine_Learning': 59},
datetime.datetime(2017, 1, 6, 0, 0): {'Machine_Learning': 54},
datetime.datetime(2017, 1, 2, 0, 0): {'Machine_Learning': 58},
datetime.datetime(2017, 1, 5, 0, 0): {'Machine_Learning': 63},
datetime.datetime(2017, 1, 7, 0, 0): {'Machine_Learning': 42},
datetime.datetime(2017, 1, 1, 0, 0): {'Machine_Learning': 43},
datetime.datetime(2017, 1, 3, 0, 0): {'Machine_Learning': 66}})

6. Some quick technical mentions
PROTOCOLS
HTTP
SERIALIZATION
JSON, XML
AUTHENTICATION
OAUTH
METHODS
GET, POST, PUT, DELETE

7. What’s great about APIs
● Separation of concerns
● Defined service contracts
● Common abstraction layer for operational monitoring
● Versioning and change management
● Buffering, load testing
● Useful tooling for common formats: buffering, load testing, caching, rate
limiting
● Minimize software a data scientist has to write
● Isolated A/B tests

8. Build your project as an API from day one
● Makes seamless integration
● Change management advantages
● Ownership of entire service
● All benefits on previous slide

9. Example: Spam Filtering
● Keyword lists were a failure
● Machine learning solved the problem
● Generalized, robust solutions
● Commoditization
● Federation
● Domain specific challenges still remaining

10. Newer Frontiers for APIs
Computer vision
● Facial recognition
● Object identification
● Logo detection
● Flag inappropriate content
Natural Language Processing
● Sentiment analysis
● Name entity recognition
● Summarization
● Translation
Personalization and Targeting
● Adtech
● Recommender systems
● Content optimization
● Lead scoring
Misc
● Speech to text
● Fraud detection
● Generalized ML tasks
● ???

11. Build, Buy, Acquire
● How long will it take your to build an adequate internal
version?
● What is the cost of expected API usage vs. time cost of
building it?
● Is the API likely to improve in the future faster than your can
improve your system?
● Are there any strategic advantages to an internal build?
● What happens if that API disappears?
● Can you use ensembling?

12. Ensembling
● A term used in many ways
● Leverage multiple models, perhaps built by different algorithms
● Stacking is a special case - Divide dataset, builds layer of models, then one
model on top those
● Random forest is a popular example

13. Two stage system
Internal
Model
3rd party
service API
Feature #1
Feature #2
Feature #3
Feature #4
Feature #5
Sensitive feature #6
Domain specific #7
Other feature #8
First stage output
Output

14. Two stage system
Internal
Model
3rd party
service API
Feature #1
Feature #2
Feature #3 x1
Feature #4
Feature #5
x2
x3
x4
Output

15. Two stage system
Internal
Model
3rd party
service API
Feature #1
Feature #2
Feature #3 x1
Feature #4
Feature #5
x2
x3
x4
Output

16. Facial Recognition
Try available options like Kairos, Microsoft Cognitive Services, Lambda Labs,
Google Cloud Vision API
Use libraries like OpenCV, Caffe, etc.
Read papers and build from scratch
Conduct research and innovate

17. Facial Recognition
Try available options like Kairos, Microsoft Cognitive Services, Lambda Labs,
Google Cloud Vision API
Use libraries like OpenCV, Caffe, etc.
Read papers and build from scratch
Conduct research and innovate
IMO, Facial recognition
is now a commodity

18. Disclaimer
Although I feature some commercial solutions as examples…
● These are not necessarily endorsements
● I’m not employed or compensated by any of these companies
● The services I mention range from adequate to great
● It’s your job to determine the best solution for your needs

19. Microsoft Cognitive Services
● Many services in
text, audio, video,
and image tasks
● You can probably
get $200 to try it
out

20. Microsoft Cognitive Services - Faces API response
[{"faceId":"1a228680-7582-47ac-bc7b-029536e95468",
"faceRectangle":{"top":145,"left":466,"width":67,"height":67},
"faceAttributes":{"age":45.0}},
{"faceId":"d0ee8e65-07fd-4674-9210-d05816205839",
"faceRectangle":{"top":106,"left":729,"width":62,"height":62},
"faceAttributes":{"age":33.8}},
{"faceId":"92696f9e-1f94-4a79-be30-4ff88539d0bb",
"faceRectangle":{"top":129,"left":164,"width":61,"height":61},
"faceAttributes":{"age":70.1}}]

21. Microsoft Cognitive Services

22. ● A source to find free and commercial solutions
● A place to exhibit your work
algorithmia.com market.mashape.com
Algorithmic marketplaces

23. {
"src":"This weekend, I am flying to Dallas
and presenting a talk on API trends.",
"format":"tree", "language":"english"
}
https://algorithmia.com/algorithms/deeplearning/Parsey

24. NOUN++NN ROOT
+-- This DET++DT det
+-- flying VERB++VBG acl:relcl
+-- I PRON++PRP nsubj
+-- am AUX++VBP aux
+-- Dallas PROPN++NNP nmod
| +-- to ADP++IN case
+-- and CONJ++CC cc
+-- presenting VERB++VBG conj
+-- talk NOUN++NN dobj
+-- a DET++DT det
+-- trends. NOUN++NN nmod
+-- on ADP++IN case
+-- API NOUN++NN compound

25. Wrapper libraries

26. indico
“Machine learning for human use”
● Sentiment
● Text tagging
● Language detection
● Political leaning
● Name Entity Recognition
● … several others

27. Indico - keyword api

28. Indico - keyword api

29. Honorable mentions
● Google Cloud Platform - Large variety of services
● IBM Watson - Language, vision, speech, and data
● Wit.ai - Natural language processing
● Diffbot - crawl and extract data from the web
● Many, many more

30. Data APIs vs Data Science Services
Data APIs - extend the features available
Data Science Services - provide a model for
interpreting your features

31. 3rd party data sources can...
● Match on email address
● Provide demographic and psychographic data points
● Up to date content (news, local businesses)
● etc.

32. Disclaimer #2
● I started OpenHouse
● OpenHouse is a free service
● We are a group of volunteers
● Everything is open source

33. OpenHouse
http://gallery.openhouseproject.co

34. Make data accessible

35. OpenHouse
Direct access:
http://api.openhouseproject.co/api/property/?min_price=0&max_price=5000000&m
in_bedrooms=0&max_bedrooms=8&min_bathrooms=0&max_bathrooms=7&min_
building_size=100&max_building_size=4000&close_to=(159.17225089355944,33.
7,-118.2)

36. Weather APIs

37. Microservice architecture
When a large system is constructed from simpler
components called microservices. Each performs a
small, specific function. Microservices communicate
with each other to fulfill higher level objectives.
Process an order:
● Order creation microservice
● Credit card validation service
● Inventory microservice
● Email service
● Fraud detection service
● Personalization service
● Advertising targeting service
● Discount offering service

38. Will I become obsolete?

39. Will I become obsolete?
Probably not.
● Adapt
● Learn methods over tools
● Always be looking at new tools
● Sometimes innovation reduces demand for certain
skills, usually slowly, and almost always creating a new
demand elsewhere
● Learn to recognize what problems will exist in a world
in which your problem is solved

41. Conclusions about 3rd party services
● They’re easily swapped with something else later
● Think about quick wins by tying things together
● Build your project service oriented from day 0
● Consider how your service could be generalized
● Keep an eye out for tools you can use but wouldn’t have
invested in developing

42. Thank you
@DataSkeptic
https://github.com/kylepolich/apis-talk