Seldon @ PAPIs Connect, Valencia, 2016-03-14
•
3 likes•958 views
"Open Source Machine Learning & Predictive APIs" - Alex Housley, Founder & CEO of Seldon @ PAPIs Connect, Valencia, 14th March 2016. http://www.papis.io/connect Abstract: IT decision makers now face an unprecedented challenge — and opportunity — to help their organization build a one-to-one relationship with customers and gain actionable insights. Machine learning and deep learning technologies that were previously reserved for companies such as Google and Amazon are now open-source. But open source machine learning is a fast moving target, with game-changing developments even in the six months since PAPIs 2015. To follow on from my talk in Sydney about our journey taking Seldon from a closed predictive API to an open source machine learning platform, I will provide fresh insight with applied examples to help decision makers stay in control, and identify opportunities for value creation.
Recommended
More Related Content
What's hot
What's hot (11)
Similar to Seldon @ PAPIs Connect, Valencia, 2016-03-14
Similar to Seldon @ PAPIs Connect, Valencia, 2016-03-14 (20)
Recently uploaded
Recently uploaded (20)
Seldon @ PAPIs Connect, Valencia, 2016-03-14
- 1. Machine Intelligence for Enterprise © 2016 Seldon Technologies Ltd. This work is licensed under a Creative Commons Attribution 4.0 International Public License
- 2. SELDON.IO ALEX HOUSLEY FOUNDER & CEO, SELDON @AHOUSLEY Open Source Machine Learning and Predictive APIs 14/03/2016
- 4. SELDON.IO
- 5. SELDON.IO Our Journey 2011 2014 2015 2016 Content Recommendation Social Sharing Data scientists in short supply Organizations want more control and on premise solutions Open Source Machine Learning Recommendation and Prediction Microservices API Platform agnostic with no lock-in Deploy on premise or in the cloud FinTech Products Barclays Proof of Concert Machine Intelligence Ecosystem +
- 6. SELDON.IO Economic Social Technological 2016 the breakout year for open machine intelligence ● Lower compute costs (CPU and GPU) ● Disruptive start-ups ● Data privacy and compliance ● Consumer expectations ● Workforce automation: 58% of job activities. ● Data scientists and decision makers want control ● Commodification of ML/AI technologies ● 2 billion smartphones; 13 billion connected devices. ● Exponential data
- 7. SELDON.IO 2016 the breakout year for open machine intelligence
- 8. SELDON.IO Optimization Feature extraction and training Monitor Impact on KPIs Industry Models and Microservices Constant feedback loop PredictionsModelsIngest Data Machine Intelligence Pipeline
- 9. SELDON.IO How to add machine intelligence to your company Build In-House Predictive API Open Source Machine Learning Control Model Evaluation Time Data Scientists Cost $$$✔️ Too Many $$✗ Limited / Unknown $✔️ Industry Models + Your Own
- 11. SELDON.IO February 2015—Open Source Launch Documentation SaaSIntegration Sales CommunityLicense
- 12. SELDON.IO 9 Months Later… (h/t @NathanBenaich)
- 13. SELDON.IO Community 1. Fix Documentation 2. Help Each Other 3. Fix Bugs
- 14. SELDON.IO 1. User Clusters - improve relevance in high churn services. 2. Tag Affinity - focused tag-based associations. 3. Latent Factor Models - best for lower churn service. 4. Item Activity Correlation - built for static slowly changing historical items. 5. Topic Models - built for sites needing long tail recommendation. 6. Association Rules - basket analysis to suggest the next best action. 7. Content Similarity - rich metadata and high sparsity across items. Recommendation Algorithms
- 15. SELDON.IO • Cascade/combine multiple algorithms to cover different users and use cases • control relevance, popularity, diversity • control interactiveness of recommendations • Combine algorithm results - e.g. weighted scores, rank combine. • Run A/B and Multivariate tests with no redeploy • Select algorithm strategies via API tags • to handle user cohorts: mobile users, desktop, tablet • to provide multiple content recommendations per page: site-wide, intersection • Change all configuration in real time with no redeployment. Advanced Optimization
- 16. SELDON.IO start test best 1 2 3 N Selecting the best model ● Evaluation of multiple strategies in parallel using multi-armed bandit. ● Adaptive as context changes - i.e. time of day, special event. ● The latest winning test strategy (1...N) is promoted to best.
- 17. SELDON.IO • Stream events in real-time • (i.e. metadata associated with transactions) • Create supervised learning pipelines: • Classification - yes/no (binary) or categorize (multi-class) • Regression - predicting a continuous value • Pluggable Algorithms • Vowpal Wabbit • XGBoost • Keras • Your algorithm! General Purpose Prediction
- 18. SELDON.IO Seldon Microservices Seldon Server Your App or Service GET /predict GET /recommend P1 Vowpal Wabbit P2 XGBoost POST /users /items /actions GET /recommend POST /events GET /predict Internal Microservices API External REST API R1 IBM Watson RN Your algorithm!
- 20. SELDON.IO Fully Integrated Open-Source Ecosystem Microservices API Open Source Libraries Closed APIs Global Models
- 22. SELDON.IO Seldon 2.0 • Flexible design • Apache Kafka as hub • data pushed to DBs and processing units as needed • Stream algorithms via Flink (or Spark) dependent on latency requirements • Batch algorithms via Spark (or Flink) • Low latency front end scoring Zookeeper for state and control • Luigi pipeline • Docker Swarm for deployment • Python single machine for agile algorithm development
- 23. SELDON.IO 2016 is the breakout year for open source machine intelligence. Data scientists and decision makers want more control. Open source helps organizations focus on the last 10% of the problem.
- 26. SELDON.IO How do you make money?(the elephant in the room) OS Integration Services SaaS Marketplace
Editor's Notes
- Install the Lato font for the best experience: http://www.fontsquirrel.com/fonts/LATO
- My name is Alex Housley founder and CEO, Seldon. The last time I gave this talk was in Sydney, so I’m happy to halve my carbon footprint by having the chance to share this with the PAPIs community in Valencia. Promise I’ll travel by bicycle when PAPIs comes to London. We released Seldon’s open source predictive platform in Feb last year after four years of R&D from an exceptional team of data scientists. Seldon is tried and tested as a closed API in demanding enterprise environment serving billions of recommendations every month. I will talk about how OS helped us to survive and thrive. This is not meant to be a an evangelical talk about how I have “seen the light”, I want you share with you some of the things we have learned from moving closed to open.
- [set the scene] You’re in a hot, Sweaty board room, mid-Summer, We were a SaaS predictive API with two PRODUCTS: Recommendation Engine and Predictive Social Sharing. FACED WITH 2 OPTIONS: 1. GET ACQUI-HIRED 2. DO SOMETHING DIFFERENT We had spent the prev 9 months speaking to potential acquirers. During this process I had a front row seat on the priorities of some of world’s leading media, ecommerce, technology companies. COMMON PATTERN emerged: large companies putting data science and prediction at the very top if their strategic agenda for investment. They were acquisitive and recruiting expensive teams. Made me think: WHAT WOULD BE THE IMPACT OF MAKING THE FULL PREDICTIVE STACK OPEN SOURCE?
- On a wet and windy day in October 2014, our team were sitting on Brighton Beach, discussing the bigger picture of what we could achieve with Seldon. We had come a long way over the previous three years and were serving content recommendations to hundreds of millions of people every month. However, we believed that continuing to ship a black box solution would increasingly face obstacles in adoption by enterprises as machine learning technology became increasingly commoditized, and new applications were developed and adopted everywhere. So we did the most disruptive thing we could imagine and open-sourced the platform and algorithms that we had spent many years, and a couple of million pounds, building. We knew there was a risk that our competitors would take what makes us valuable, but we also knew that the bigger risk would have been to stop innovating. So we took the leap and pivoted in one of the most exciting ways a technology company can go.
- Seldon started as a content recommendation engine. Seldon is tried and tested as a closed API in demanding enterprise environment serving billions of recommendations every month, mostly recommending articles on news websites. In 2014 we took the business in a new open-source direction to create Seldon. 2016 – we’re building out a fintech product and are aiming to establish a POC with Barclays. From early discussions there are many parts of the bank that can benefit from Seldon. Joke about Risk Weighted Asset.
- There are a number of interconnected market forces at work that means 2016 will be a tipping point for machine intelligence: 250 billion billion (250 x 10^18) transistors were produced in 2014. Every second of that year, on average, 8 trillion transistors were produced. That figure is about 25 times the number of stars in the Milky Way. (according to Moores law production should now have doubled.) 58% of job activities can be automated. 47% of jobs will be taken over cognitive machines in the next 10 years 2 billion smartphones;13 billion connected devices. Seldon was one of the first companies to open source a machine learning platform. But last year we saw Google open-source TensorFlow, IBM donated SystemML to Apache. Elon Musk and Sam Altman form a non-profit AI research org called OpenAI. Open-source is a huge benefit to enterprise, particularly banks where data privacy and compliance are particularly important, as it gives full control with an on premise deployment.
- Credit: http://thenewstack.io/ Credit kenming_wang @ Flickr
- Seldon isn’t an OS library, it’s an end-to-end machine learning PIPELINE. Include best of Open Source and algorithms built ourselves. 1. CONNECT YOUR DATA - Ingesting behavioural data from events that contains metadata and context such as device and location. 2. MODEL BUILDING - Multiple models are built based on desired Goals (Could be a KPI or an action/event). Behavioral data plus algorithms are used to train the predictive models. computationally inefficient to store all the possible alternatives… realtime behavioural data needs to update the models in real-time. value not in the algorithm, the value is in the model. INDUSTRY MODEL - working in media, advertising and ecommerce. 3. OUTPUTS - there are currently two outputs for Seldon - one is a recommendation and another is a prediction (score). FEEDBACK LOOP - Models are optimised in a recursive way… FURTHER INFO Seldon pulls in behavioral data from any digital environment, builds predictive models and outputs recommendations and predictions at SCALE. But have built a generic platform with a broad range of applications including finance, insurance and healthcare.
- Build in house: DEMAND - there are far fewer for truly skilled machine learning and AI developers than big data engineers. Improving internal data science capabilities is increasingly important for companies. So they are hiring or aquahiring teams of data scientists. 3rd party: DATA SECURITY - there are many companies with data control policies that require the hosting of consumer data behind their firewall, which a flexible open source solution will allow. Sometimes there’s no transparency on algorithms. Open Source. MARKET DISRUPTION Open source technologies such as Docker, Hadoop, and Apache Spark, have superseded proprietary operating systems and databases. Meanwhile, most vendors higher up the data science stack (i.e. providing predictive analytics, recommendations, and machine learning APIs) are effectively licensing black box solutions. Seldon wants to reduce barriers to entry and get the Seldon’s technology that we believe in the hands of as many developers as possible. Businesses wanting to solve their own / domain-specific problems. ADD PRIVACY….
- WHY? Data scientists want more CONTROL to solve the problems specific to their business. PARATOS LAW in action – people should be spending 90% of their time solving the 10% of the domain-specific problems that make the biggest impact on their business, but otherwise data scientists are focusing on the remaining 90% and wasting time on reinventing the wheel. I’ll share with you a quick analogy with this DJ MIXER: Each channel on the mixer represents a predictive model. The controls represent hyper-parameters. Data scientist is the DJ listening to the audience and adjusting the controls accordingly. Seldon gives the DJ super powers. Enabling them to play all of the stages of a festival at the same time, so your audiences is not stuck listening to Lionel Richie (collaborative filtering) if you want to listen to Metallica (matrix factorization) But remember, you can always tap the DJ on the shoulder and make special requests.
- OPEN SOURCE SETUP. SaaS platform grows roots, provisioning usually internal process with continuous integration and deployment. Rarely setup new infrastructure from scratch. VIRTUAL MACHINE. DOCUMENTATION. SaaS businesses don’t need as much. Docs on Github so people can commit changes – first pull requests. Documentation gets the highest engagement to see how to use it. [show documentation] CHOOSING A LICENSE. Reason for Apache 2 vs LGPL/GPL – better for business because they don’t have to make modifications open source. SAAS – open source is a great driver of SaaS customers. Many companies want to start with SaaS and have a longer term plan to move on-prem and work with custom algos. SALES CYCLE – INBOUND: more leads via open-source. Previously contract before getting tech in hands of developers. TRACKING – companies much further along the funnel because they don’t have to sign up to get started. VALUE CHAIN – where we sit now we have disrupted ourselves. TIMING (OS more strategic and sometimes SaaS is a better option). COMMUNITY – newsletter, github, detail release notes, clean codebase, future: events, etc. Important for us. - As SaaS: classically delivering endpoint - In comparison as open source: enable looking into configuration --> make product & onboarding more streamlined - Documentation important for activating developers --> transfer docs/pdf to Github - Interestingly: first pull requests about fixing documentation - Create demo apps to show possibilities - Huge thing: community --> changed way of communication - Inbound instead of outbound - License issues --> solved with Apache 2 license - Sales cycle got longer - Didn’t lose any customers through going open - Found that open source is a good distribution channel - Find right business/revenue model for open source - Deployment --> cloud vs. on premise - Build ecosystem around proactive community & potentially work together with competitors - Architecture enables microservices & API --> interfaces with other ML services - Integration with other open source libraries & closed APIs - Change in cost structure - Conflict in support (free vs. SaaS)? - Measuring open source engagement?
- There are INFINITE algorithm configurations to choose from. So which ones are best for my business? Example of when different models used: high/low churn, days of week. Userbase or product mix changes. Seasonal changes. YOU CAN USE YOUR OWN ALGORITHMS AND MODELS. The model selection will vary depending on the user type. OLD WAY - AB testing. each test used to be a manual process measure the impact of recommendations on KPIs such as CTR, conversion rate, etc.. NEW WAY – CONTINUOUS TESTING of all models that diverts more traffic towards the model performing best at that specific point in time. Called a MULTI-ARMED BANDIT, inspired by a strategy to play a room full of slot machines. Give a user case of which algorithms would be used high churn news environment. You want to make the case that it’s not 1 algorithm/model but a combination of different models that will maximise your KPI. And that’s why you need Seldon. We A/B test from set of INDUSTRY MODELS to find a combination of model selections that work best for your business for a given user at a given time. TIME SAVED in choosing the best model. Seldon increases the productivity of your data science teams and helps your business to increase profits through rapid prototyping and better KPI performance.
- Recommendations using EXTERNAL REST API Predictions takes JSON of /events data and provides regression and classification outputs. Microservices /predictions with Vowpal Wabbit Example /recommendations microservices – including Collaborative Deep Learning from the KDD talk. Example of community member using Microservices to test various matrix factorization implementation.
- Two dimensions: horizontal vs vertical (market focus) and product (scale) vs service (consulting) After open-source Seldon in the bottom right, providing services on top of a horizontal platform. ”Integration Services” or “Customer Funded Development” Companies generally seek a position in the top right, unicorn territory. But it doesn’t make sense for us to jump there directly. So we’re first making it much easier to deploy Seldon’s product through SaaS and optimising setup on our horizontal platform and carefully picking some areas to focus on. Our aim is to spawn many unicorns. https://thenounproject.com/term/mule/28242/
- MICROSERVICES – make the Seldon stack completely pluggable with third party code / algos developed by your data scientists IN ANY LANGUAGE (R, Python, etc) that can be put into production and utilise the same pipeline as core Seldon algorithms. OPEN SOURCE LIBRARIES – we already used microservices to connect an OS library called Vowpal Wabbit (Microsoft research) to power our new predict endpoint. This enables regression, binary and multi-class classification. We are connected to the leading machine intelligence libraries such as Torch. CLOSED APIs – third parties who provide We can leverage third party APIs such as IBM Watson for personality insights or text to speech. ENTERPRISE DISTRIBUTIONS – Seldon are already planning integrations with some of the leading enterprise distributions.
- 4. SELDON CONTAINER INFRASTRUCTURE MAIN COMPONENTS – REST API SERVERS, ZOOKEEPER, FLUENTD, KAFKA, SPARK, MEMCACHE, JDBC database… VIRTUAL MACHINE - Setting up infrastructure is complex. Portable - Developer can download it and straight away access the infrastructure. Movie demo. DOCKER / AMIs - technology that allows you to use on different platforms (docker container = shipping container. Part of our infrastructure in each container). Deploy on premise, cloud (AWS, Google) or SaaS. [encourage people to register for AMIs]
- WORK IN PROGRESS Flexible design to allow different techniques to attack the problem Stream based - Apache Kafka as hub data pushed to DBs and processing units as needed Stream algorithms via Flink (or Spark) dependent on latency requirements Batch algorithms via Spark (or Flink) Low latency front end scoring systems evolution of Seldon server with likely input from trading systems expertise Zookeeper for state and control, Luigi pipeline Docker Swarm for deployment Python single machine for agile algorithm development scikit-learn, pandas, pyseldon etc.
- Data scientists and SALES Come and work at Barclays Accelerator
- BEFORE TAKING SOME QUESTIONS, I’LL FINISH WITH A QUESTION. Can we change the way in which people view their competition? London is become a centre of excellence for AI. And companies that collaborate as part of an ecosystems have a competitive advantage. Since Seldon went OS had an open door policy about speaking to “competitors”. Download Seldon’s open source, VM or AMI... visit seldon.io or develors head on over to docs.seldon.io Contact me if you would like to help out or to discuss how we can help add machine intelligence into your business. Or other AI companies that want to partner. THANK YOU
- (quite often this is the elephant in the room – it’s often the first question we get asked) OS is revolutionising the way in which we do business. Since we released Seldon in Feb it is already starting to be used by world’s largest companies. FREE - First, don’t expect more than a small percentage of activated users to pay. ADVISORY. ML is complicated and more projects have unique requirements. Advisory services, enables us to spend more time with customers, understanding their problems so ultimately leads to building a better platform. MLAAS. Finally, we have a SaaS platform! Many companies find us via the OS but SaaS the best starting point. Companies like the MIGRATION PATH [can reference later in call for open models] ECOSYSTEM. Third party technology, APIs and models. Also UNIFY AND MONETISE the DATA. Value in creating an ecosystem. Seldon can offer distribution and monetization.