Uploaded byEdunomica
144 views

Paige Roberts: Shortcut MLOps with In-Database Machine Learning

The document discusses MLOps (Machine Learning Operations) and the challenges associated with operationalizing machine learning, arguing that in-database machine learning (in-db ML) can address these issues. It outlines how in-db ML simplifies the machine learning workflow by performing data analysis and model training directly within databases, thereby eliminating the need for data transfer and enhancing efficiency. The advantages of in-db ML include improved speed, scalability, and security, as well as the capability to process large datasets without downsampling.

Embed presentation

Shortcut MLOps with In-Database Machine Learning Paige Roberts Vertica Open Source Relations Manager
Agenda MLOps – Machine Learning Operations Challenges with Operationalizing ML Is In-Database ML Possible? How Does In-DB ML Help MLOps? How Does In-DB ML Work? Questions
Advanced Analytics Use Cases are Everywhere It’s not an option; competing on data is the new style of business. Network Optimization Energy Optimization
Standard MLOps Cycle 4
Challenges with operationalizing ML
A lot of machine learning projects never make it to production https://www.redapt.com/blog/why-90-of-machine- learning-models-never-make-it-to-production https://www.forbes.com/sites/forbestechcouncil/2019/04/03/why- machine-learning-models-crash-and-burn-in-production/ https://thenewstack.io/add-it-up-how-long- does-a-machine-learning-deployment-take/ https://info.algorithmia.com/hubfs/2019/Whitepapers/ The-State-of-Enterprise-ML- 2020/Algorithmia_2020_State_of_Enterprise_ML.pdf https://towardsdatascience.com/why-90-percent-of-all-machine- learning-models-never-make-it-into-production-ce7e250d5a4a
Data science journey – Now sample extraction ODBC/JDBC Data Preparation & Model Training Model Deployment & Management
Separation of Dev and Prod Environments is Problematic Development Environment  Notebooks – Interactive  Python – Pandas, SciKit  Tensorflow  Training  Batch  One Instance or single computer  Small sample – fits in memory  No worries about resource mgmt Production Environment No notebook – Automated Java, Scala JS, C++ Prediction/Inference Streaming  Many nodes or instances  Full data set – could be huge  Concurrent jobs, resource mgmt, autoscaling 8 Tools: Scale:
Challenges Added Cost Additional hardware and software required. Lots of components to make a whole solution. Requires Down Sampling Cannot process large data sets due to memory and computational limitations, resulting in inaccurate predictions Slower Time to Development Higher turnaround times for model building/scoring and need for moving, transforming large volumes of data between systems Slower Time to Deployment Inability to quickly deploy predictive models into production. Often must be rebuilt in another technology so it will scale. 7
In-Database Machine Learning End-to-end data science workflow in place SELECT IMPUTE SELECT NORMALIZE SELECT RF_CLASSIFIER SELECT ROC SQL User Defined Functions Distributed Analytical Database
Is in-database ML really possible?
8 Databases that Support In-Database Machine Learning  Vertica  Amazon Redshift *  BlazingSQL  Google BigQuery *  IBM Db2 *  Kinetica  Microsoft SQL Server *  Oracle * https://www.infoworld.com/article/3607762/8-databases-supporting-in-database-machine-learning.html * Machine learning is an add-on or integrated product purchased with the database
Many analytical databases have full set of ML functions https://www.vertica.com/python/ https://github.com/vertica/VerticaPy Correlations Heteroscedascity Trend & Stationarity Normality Cramer’s V Biserial Point Kendall Spearman Pearson Breush-Pagan Goldfeld-Quandt White’s Lagrange Engle Augmented Dickey-Fuller Mann-Kendall Normaltest Regular Joins Joins XGBoost Random Forest Tree Based Models Linear Regression Logistic Regression LinearSVC Linear Models KMeans Bisecting KMeans Clustering Time Series Joins Spatial Joins
Deployment/ Management End-to-end machine learning process in database 14 Data Analysis/ Exploration Data Preparation Modeling Evaluation Business Understanding In-Database Scoring Speed at Scale Security Statistical Summary Sessionization Pattern Matching Date/Time Algebra Window/Partition Date Type Handling Sequences and more… Outlier Detection Normalization Imbalanced Data Processing Sampling Test/Validation Split Time Series Missing Value Imputation and more… K-Means Support Vector Machines Logistic, Linear, Ridge Regression Naïve Bayes Cross Validation and more… Model-level Stats ROC Tables Error Rate Lift Table Confusion Matrix R-Squared MSE XGBoost Filtering Feature Selection Correlation Matrices Table-Like Management Versioning Authorization and more… Random Forests and more… Principal Component Analysis PMML Import/Export TensorFlow Import AutoML
Modern analytical databases analyze data in many formats – Semi-structured, streaming, text, complex, … data formats optimized for long-term storage Database format optimized for read/analyze performance Historical Recent Unified Analytics Analyze data where it sits.  Web logs  Sensor, IOT data  Time series  Click streams  Purchase history  Geospatial  … Streaming CSV Cloud storage, Hadoop (HDFS), S3 on prem data formats optimized for streaming messaging ROS
How does in-database ML help?
Advantages of In-Database Machine Learning 17 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer
Advantages of In-Database Machine Learning 18 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance
Advantages of In-Database Machine Learning 19 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management
Advantages of In-Database Machine Learning 20 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users
Advantages of In-Database Machine Learning 21 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities
Advantages of In-Database Machine Learning 22 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities Avoiding maintenance cost of a separate system
Advantages of In-Database Machine Learning 23 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities Avoiding maintenance cost of a separate system Identical Dev, Prod environments
How does in-database ML work?
In-Database Random Forest A B C r1 r2 r3 NODE 1 1 NODE 2 2 NODE 3 3 2 A B C Tree1 r1 r2 A B C Tree2 r1 r3 A B C Tree3 r2 r3 3 Tree1 Tree2 Tree3 A B C r1 r3 A B C r2 r3 A B C r1 r2 TRAIN TRAIN TRAIN MERGE TABLE SPLIT DISTRIBUTE 1
In-Database Extreme Gradient Boosting A COMPUTE SUMMARY STATISTICS NODE 1 1 B COMPUTE SUMMARY STATISTICS NODE 2 2 C COMPUTE SUMMARY STATISTICS NODE 3 3 NODE 1 1 NODE 2 2 NODE 3 3 A B C FIND BEST SPLIT A B C FIND BEST SPLIT A B C FIND BEST SPLIT 1 2 ... ? Tree1 TreeN 3 ITERATIVE PROCESS TRAIN EXPORT SQL + memModel BINARY 4 JSON
SARIMA In-Database SARIMA X Y 0 s+2 s+4 s+6 s+8 s+10 s+12 V 2V 3V trend seasonality AR MA ARIMA value prediction confidence interval 1 2 3 4 seasonal decomposition NODE 1 NODE N X Xt Xt-1 Xt-2 𝜃𝜃𝑖𝑖 0 ≤ 𝑖𝑖 ≤ 𝑘𝑘𝑘 X Xt Xt-1 Xt-2 𝛼𝛼𝑗𝑗 0 ≤ 𝑗𝑗 ≤ 𝑘𝑘𝑘 𝜃𝜃𝑖𝑖 𝑘𝑘𝑘 ≤ 𝑖𝑖 ≤ 𝑞𝑞 𝛼𝛼𝑗𝑗 𝑘𝑘𝑘 ≤ 𝑗𝑗 ≤ 𝑝𝑝 COMPUTE COMPUTE A Simple Noniterative Estimator for Moving Average Models John W. Galbraith & Victoria Zinde-Walsh
Integration with other ML technologies Geospatial Event Series Time series Text Analytics Pattern Matching Regression User-Defined Functions Machine Learning In-DB Data Science Pojo Frozen Graph SQL + memModel EXPORT EXPORT EXPORT EXPORT IMPORT SQL + memModel BINARY EXPORT XGBoost Random Forest ElasticNet Lasso Ridge KMeans Isolation Forest SVM JSON PCA
Example Notebooks https://www.vertica.com/python/examples/ 29
Q&A VerticaPy: https://vertica.com/python Paige Roberts Open Source Relations Manager E: Paige.Roberts@microfocus.com Try: https://www.vertica.com/try
Thank You
Previously, we used Apache Spark. Using Apache Spark cost 80% of the time from data I/O, only 20% of the time on machine learning. Now, we switched to using Vertica in-database machine learning and saved 80-90% data reading time. So, it’s had huge benefits in saving time and costs for model training. Xiaobo Ge, CTO, EOITek 32
Using Vertica’s User-Defined Extension, we have already built a bespoke lowpass filter to run against our data – that ensures top analytical performance without having to move the data to other systems or tools. Vertica analytics and machine learning ultimately delivers an edge that leads to more points, podiums, and wins for Jaguar Racing. Phil Charles 33 Technical Manager Jaguar Formula E racing team
What People Are Saying 34 “Vertica is the path to production for machine learning.” “With Vertica in Eon Mode’s sub-clustering ability, we can also give more computation power to our users as needed. This is really helpful for our machine learning analysts.” “We’re using machine learning and deep learning, technologies, in the cloud, to consume, that data, leveraging things like Python and TensorFlow to build out and then operationalize those predictive models ...” “Vertica is a key technology, bringing in those diverse, data sets and building flexible, data models, and being able to do it quickly and at scale …” “If we have a particular use case, we can easily write our own functions in any language – Java, Python, R – and deploy this in Vertica, harnessing the power of parallel computing.” “Others are forced to sample. This is about using all of your data which is fantastic.”
Analyze Data In Many Formats – Semi-structured, streaming, text, complex, … data formats optimized for long-term storage data format optimized for read/analyze performance Historical Recent Unified Analytics Analyze data where it sits.  Web logs  Sensor, IOT data  Time series  Click streams  Purchase history  Geospatial  … Streaming CSV Cloud storage, Hadoop (HDFS), S3 on prem data formats optimized for streaming messaging ROS
Distributed databases automatically parallelize algorithm training Deliver accurate predictive analytics at speed and scale MPP architecture is a great fit – ML functions run in parallel across hundreds of nodes in a Vertica cluster Tree1 Tree2 Node1 Tree3 Tree4 Tree2n-1 Tree2n Node2 Noden
High Performance + High Concurrency 37 Get data quickly enough to act upon it, explore your data interactively, and enable everyone to make their own data-driven decisions Enable everyone to make their own data-driven decisions. Get data quickly enough to act on it. Explore data interactively. Scale Data Volumes Scale Users and Jobs SQL Database + + Analytics & ML Query Engine
Vertica ML is fast and scalable • Leverages MPP infrastructure and scale-out architecture for parallel high performance • No data movement across the system • No need to downsample, so accuracy is boosted Manage resources for many users or jobs Use a familiar interface • Manage, train, and deploy ML models using simple SQL calls • Integrate ML functions with other tools via SQL • Or use Jupyter and Python for familiar data science Proven infrastructure • No additional hardware and software to manage • Open integration with other solutions • Requires fewer people to setup and maintain • Resource isolation for multiple user sessions • Natural segmentation of user environments and privileges • Configurable memory usage for ML functions In-Database Machine Learning Advantages Enterprise-Grade Scalability Concurrency Productivity
VerticaPy – Python API for Machine Learning at Scale 28 Benefits: Jupyter/Python – popular interactive tool of choice Graphs and visualizations – even on large data sets Data stays in database – security, integrity Distributed Engine - heavy computation done by optimized MPP engine Open source – you can contribute! Manage models – store in DB Results where they’re needed – available in DB for other analyses, visualization, or driving applications and AI
VerticaPy  Python library  Open source  https://github.com/vertica/VerticaPy  Pandas and Scikit Learn functionality  Conduct data science projects in Vertica – high performance, high scale analytical database  Takes advantage of built-in parallel analytics and machine learning  You write Python in a Jupyter Notebook, the database executes it on the full data set  Uses vertica-python native Python client for Vertica created by Uber with contributions from Twitter, Palantir, Etsy, Kayak, Gooddata, … - https://github.com/vertica/vertica-python 40
Bruce Yen, WW Business Intelligence Leader “Our Vertica platform is instrumental in many areas of our business—creating predictive algorithms, serving up product recommendations, powering insight to our mobile apps, and generating daily reports and ad- hoc queries. It’s crucial for enabling us to be more agile with data.”
Challenge - Analyze large volumes of endpoint data to detect indications of cyber attacks Result - Provides near real-time data insertion across hundreds of thousands of endpoints; results in enterprise-grade threat detection quality - Ensures rapid querying, essential for analysts to outpace attackers - Delivers higher detection rates and dramatically reduces false positives - Enables analysts to perform complex queries over large volumes of data; presents results in easy-to-understand interface - Provides credible, robust, and scalable big data technology that will be trusted by its customers Cyberbit Surpasses Conventional Security Systems Vertica uses big data, behavioral analysis, and machine learning to optimize detection and response
Our Vertica platform is instrumental in many areas of our business—creating predictive algorithms, serving up product recommendations, powering insight to our mobile apps, and generating daily reports and ad- hoc queries. It’s crucial for enabling us to be more agile with data. Bruce Yen 43 WW Business Intelligence Leader Guess
Advantages of Python + MPP Analytical Database MPP Scale Clusters with no name node or other single point of failure allow unlimited scale Speed and Concurrency Query optimization and resource management across multiple nodes Features ML algorithm parallelization, moving windows, geospatial analysis, time series joins, fast data prep... Open Architecture Integration with many other applications - BI, ETL, Kafka, Spark, Data Science Labs … Broad Utility Many functionalities - one of the most broadly useful programming languages. Flexibility It Many right paths to do things, a lot of freedom, works on many platforms. Ease of Use High level of abstraction makes Python one of the easiest programming languages. Strong Community Most data scientists master Python. Many useful packages (pandas, scikit, …)
Linear regression Analytic database functions include machine learning algorithms K-means Logistic regression Naive Bayes Random Forest SVM Predict customer retention Forecast sales revenues Customer segmentation Predict sensor failure Classify gene expression data for drug discovery Refine keywords to improve Click Through Rate (CTR)
20 Bring your R, TensorFlow, and Python code inside the database – analyze the data in place.
Online Examples
Predictive Maintenance Demo 48 Analyze sensor data from cooling towers across the US , enabling equipment manufacturers to predict and prevent equipment failure
Flight Tracker Demo 49 Vertica operates at the “edge” with flight track detail. Sensor data is collected using a Raspberry pi with radio receiver and antenna. Data is loaded into Vertica as thousands of records per second and builds to billions of flight data points collected within a 250-mile radius. https://www.vertica.com/blog/blog-post-series-using-vertica-track- commercial-aircraft-near-real-time/
Customer Examples
 Huge improvements in stability and performance after moving to Vertica  24 mins on Spark, 3 mins in Vertica  Can incorporate other data like weather to optimize predictive thermostat efficiency after moving to Vertica ML  Citing speed of analytics, ease of use when coding in SQL, and improvements in the accuracy of models after moving workloads to Vertica ML  Solving issues that were previously unsolvable  Minimal hardware, software, and personnel investments when differentiating with data science. 51
Moving data science workloads from Spark on Hadoop to in-database Improvements in stability and performance Creating customer segmentation via clustering algorithms on a 15 million customer dataset took 24 mins on Spark - 3 mins in database Concurrently running other algorithms without performance impact Cardlytics partners with more than 1,500 financial institutions to run their online and mobile banking rewards programs, which gives us a robust view into where and when consumers are spending their money.
Fidelis Cybersecurity protects the world's most sensitive data by identifying and removing attackers no matter where they're hiding on your network and endpoints. 53 Data science team was experiencing challenges with performance while using Spark ML Moving workloads from Spark ML to in-database ML provided: Speed of analytics Ease of use when coding in SQL Increased accuracy of models
Some Vertica IoT Customer Resources Case Studies  Anritsu ROI case study: https://www.vertica.com/wp- content/uploads/2017/01/r24-HPE-Vertica-ROI-case-study-Anritsu.pdf  Infographic of ROI: https://www.vertica.com/wp- content/uploads/2017/03/Anritsu-v2.pdf  Nimble Storage ROI case study: https://www.vertica.com/wp- content/uploads/2017/08/Nimble-Storage-ROI.pdf  Optimal+ case study: https://www.vertica.com/wp- content/uploads/2017/06/Optimal-MF-rebrand-FINAL-lo-res.pdf  *Climate Corp case study: https://www.vertica.com/wp- content/uploads/2019/01/Climate-Corp_Success-Story-FINAL.pdf Webcasts – Data Disruptors  Philips: https://www.brighttalk.com/webcast/10477/277693  Climate Corp: https://www.brighttalk.com/webcast/8913/336201  Nimble Storage (HPE InfoBright): https://www.brighttalk.com/webcast/8913/330769  Zebrium: https://www.brighttalk.com/webcast/8913/332838  Simpli.fi: https://www.brighttalk.com/webcast/8913/354325/simpli-fi- delivers-advertising-insights-on-billions-of-streaming-bid-messages Videos  Optimal+: https://www.youtube.com/watch?v=IZkkoy5ZT1M&feature=youtu.be  Anritsu: https://www.youtube.com/watch?v=QZ5vWqblVXU&feature=youtu.be 54

More Related Content

PDF
Machine Learning Platformization & AutoML: Adopting ML at Scale in the Enterp...
byEd Fernandez
 
PPTX
END-TO-END MACHINE LEARNING STACK
byJan Wiegelmann
 
PDF
Large-Scale Machine Learning at Twitter
bynep_test_account
 
PDF
알리바바 클라우드 PAI (machine learning Platform for AI)
byAlibaba Cloud Korea
 
PPTX
Flux - Open Machine Learning Stack / Pipeline
byJan Wiegelmann
 
PDF
Challenges of Deep Learning in the Automotive Industry and Autonomous Driving
byJan Wiegelmann
 
PPTX
Machine learning and Deep learning on edge devices using TensorFlow
byAditya Bhattacharya
 
PPTX
Aditya Bhattacharya - Enterprise DL - Accelerating Deep Learning Solutions to...
byAditya Bhattacharya
 
Machine Learning Platformization & AutoML: Adopting ML at Scale in the Enterp...
byEd Fernandez
 
END-TO-END MACHINE LEARNING STACK
byJan Wiegelmann
 
Large-Scale Machine Learning at Twitter
bynep_test_account
 
알리바바 클라우드 PAI (machine learning Platform for AI)
byAlibaba Cloud Korea
 
Flux - Open Machine Learning Stack / Pipeline
byJan Wiegelmann
 
Challenges of Deep Learning in the Automotive Industry and Autonomous Driving
byJan Wiegelmann
 
Machine learning and Deep learning on edge devices using TensorFlow
byAditya Bhattacharya
 
Aditya Bhattacharya - Enterprise DL - Accelerating Deep Learning Solutions to...
byAditya Bhattacharya
 

What's hot

PDF
Ml infra at an early stage
byNick Handel
 
PPTX
Mohamed Sabri: Operationalize machine learning with Kubeflow
byLviv Startup Club
 
PPTX
Computer vision-must-nit-silchar-ml-hackathon-2019
byAditya Bhattacharya
 
PDF
Unifying Twitter around a single ML platform - Twitter AI Platform 2019
byKarthik Murugesan
 
PDF
ML Infra for Netflix Recommendations - AI NEXTCon talk
byFaisal Siddiqi
 
PDF
Monitoring AI with AI
byStepan Pushkarev
 
PDF
DIY Analytics with Apache Spark
byAdam Roberts
 
PDF
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
PPTX
Ai use cases
bySparsh Agarwal
 
PDF
MLOps with Kubeflow
bySaurabh Kaushik
 
PDF
MLSEV Virtual. From my First BigML Project to Production
byBigML, Inc
 
PDF
Introduction to ML.NET
byGianni Rosa Gallina
 
PDF
Big Data at Speed
bymarkgrover
 
PPTX
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
byJosh Patterson
 
PDF
Cloud Computing Was Built for Web Developers—What Does v2 Look Like for Deep...
byDatabricks
 
PPTX
IBM Meetup on November 1, 2018: Machine Learning made easy with Watson Studio
bySvetlana Levitan, PhD
 
PDF
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
byDatabricks
 
PDF
Distributed Deep Learning with Hadoop and TensorFlow
byJan Wiegelmann
 
PDF
Near real-time anomaly detection at Lyft
bymarkgrover
 
PDF
Machine Learning at Scale with MLflow and Apache Spark
byDatabricks
 
Ml infra at an early stage
byNick Handel
 
Mohamed Sabri: Operationalize machine learning with Kubeflow
byLviv Startup Club
 
Computer vision-must-nit-silchar-ml-hackathon-2019
byAditya Bhattacharya
 
Unifying Twitter around a single ML platform - Twitter AI Platform 2019
byKarthik Murugesan
 
ML Infra for Netflix Recommendations - AI NEXTCon talk
byFaisal Siddiqi
 
Monitoring AI with AI
byStepan Pushkarev
 
DIY Analytics with Apache Spark
byAdam Roberts
 
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
byBill Liu
 
Ai use cases
bySparsh Agarwal
 
MLOps with Kubeflow
bySaurabh Kaushik
 
MLSEV Virtual. From my First BigML Project to Production
byBigML, Inc
 
Introduction to ML.NET
byGianni Rosa Gallina
 
Big Data at Speed
bymarkgrover
 
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
byJosh Patterson
 
Cloud Computing Was Built for Web Developers—What Does v2 Look Like for Deep...
byDatabricks
 
IBM Meetup on November 1, 2018: Machine Learning made easy with Watson Studio
bySvetlana Levitan, PhD
 
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
byDatabricks
 
Distributed Deep Learning with Hadoop and TensorFlow
byJan Wiegelmann
 
Near real-time anomaly detection at Lyft
bymarkgrover
 
Machine Learning at Scale with MLflow and Apache Spark
byDatabricks
 

Similar to Paige Roberts: Shortcut MLOps with In-Database Machine Learning

PDF
Python + MPP Database = Large Scale AI/ML Projects in Production Faster
byPaige_Roberts
 
PPTX
rsec2a-2016-jheaton-morning
byJeff Heaton
 
PPTX
BIG DATA AND MACHINE LEARNING
byUmair Shafique
 
PDF
ML crash course
bymikaelhuss
 
PDF
ML.pdf
bySamuelAwuah1
 
PPTX
Lectuhhhhhhhhhhhhhhhhhhhhhhbbbhhhre 1.pptx
byHekmatyarZahir
 
PDF
What is Machine Learning and How Does it Work.pdf
byAPAC Entrepreneur
 
PDF
Demystifying Machine Learning - How to give your business superpowers.
by10x Nation
 
PPTX
Machine Learning presentation of diploma
bysp915411
 
PDF
Machine learning it is time...
bySandip Chatterjee
 
PPTX
machine learning introduction notes foRr
bySanaMateen7
 
PPTX
Moving from BI to AI : For decision makers
byzekeLabs Technologies
 
PDF
Using Machine Learning to Accelerate Revenue
byPaul Johnston
 
PPTX
Lecture 01-02 ANN BS IT (Overview).pptxx
byAmishAli2
 
PDF
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
byLucas Jellema
 
PPTX
Machine Learning and Its Real-World Impact
byCBitss Technologies
 
PPTX
The 4 Machine Learning Models Imperative for Business Transformation
byRocketSource
 
PDF
What is Machine Learning in Simple Terms and Why It Matters Today | IABAC
byIABAC
 
PDF
A Compendium of Various Applications of Machine Learning
byIRJET Journal
 
PDF
Introduction to machine learning and applications (1)
byManjunath Sindagi
 
Python + MPP Database = Large Scale AI/ML Projects in Production Faster
byPaige_Roberts
 
rsec2a-2016-jheaton-morning
byJeff Heaton
 
BIG DATA AND MACHINE LEARNING
byUmair Shafique
 
ML crash course
bymikaelhuss
 
ML.pdf
bySamuelAwuah1
 
Lectuhhhhhhhhhhhhhhhhhhhhhhbbbhhhre 1.pptx
byHekmatyarZahir
 
What is Machine Learning and How Does it Work.pdf
byAPAC Entrepreneur
 
Demystifying Machine Learning - How to give your business superpowers.
by10x Nation
 
Machine Learning presentation of diploma
bysp915411
 
Machine learning it is time...
bySandip Chatterjee
 
machine learning introduction notes foRr
bySanaMateen7
 
Moving from BI to AI : For decision makers
byzekeLabs Technologies
 
Using Machine Learning to Accelerate Revenue
byPaul Johnston
 
Lecture 01-02 ANN BS IT (Overview).pptxx
byAmishAli2
 
The Art of Intelligence – A Practical Introduction Machine Learning for Orac...
byLucas Jellema
 
Machine Learning and Its Real-World Impact
byCBitss Technologies
 
The 4 Machine Learning Models Imperative for Business Transformation
byRocketSource
 
What is Machine Learning in Simple Terms and Why It Matters Today | IABAC
byIABAC
 
A Compendium of Various Applications of Machine Learning
byIRJET Journal
 
Introduction to machine learning and applications (1)
byManjunath Sindagi
 

More from Edunomica

PDF
Daniel Samaan: ChatGPT and the Future of Work
byEdunomica
 
PDF
Fanni Kadocsa: Unlocking the power of capability building: Maximizing the imp...
byEdunomica
 
PDF
Zack Johnson: Session title: People Analytics: the epicenter of management an...
byEdunomica
 
PPTX
Anita Zbieg: How to make data actionable? Lessons from the teams on how to tu...
byEdunomica
 
PDF
Aizhan Tursunbayeva: The ethics of people analytics: risks, opportunities and...
byEdunomica
 
PDF
Kevin Martin: The New Corporate Currency
byEdunomica
 
PPTX
Catherine Coppinger: Why Anchor Days Are Sinking Productivity & What to Do Ab...
byEdunomica
 
PPTX
Kevin Martin: Empowering Your Board with the People Analytics That Matter
byEdunomica
 
PDF
Aizhan Tursunbayeva: Navigating Opportunities and Risks: A Responsible Approa...
byEdunomica
 
PPTX
Alan Susi: Organizational Health: A People Team’s path to Minimum Viable ‘Wo...
byEdunomica
 
PDF
Cole Napper: Are you ready for generative AI in people analytics?
byEdunomica
 
PPTX
Fahim Karim: Attrition Prevention
byEdunomica
 
PDF
Taras Filatov: Building your own metaverse & NFT app
byEdunomica
 
PDF
Alex Poon: Should you gamify community contributions?
byEdunomica
 
PDF
Julio Holon: Decentralised colaboration
byEdunomica
 
PDF
Startup Presentation: Gaianet
byEdunomica
 
PDF
Shawn Grubb: Minnows v. whales: Quadratic Governance to the rescue
byEdunomica
 
PDF
Joachim Stroh: Hypha DAO, the 3rd generation of DAOs
byEdunomica
 
PDF
Vikram Aditya: Biggest Opportunity Areas in the DAOverse
byEdunomica
 
PDF
Tamara Helenius: The Commons are Coming
byEdunomica
 
Daniel Samaan: ChatGPT and the Future of Work
byEdunomica
 
Fanni Kadocsa: Unlocking the power of capability building: Maximizing the imp...
byEdunomica
 
Zack Johnson: Session title: People Analytics: the epicenter of management an...
byEdunomica
 
Anita Zbieg: How to make data actionable? Lessons from the teams on how to tu...
byEdunomica
 
Aizhan Tursunbayeva: The ethics of people analytics: risks, opportunities and...
byEdunomica
 
Kevin Martin: The New Corporate Currency
byEdunomica
 
Catherine Coppinger: Why Anchor Days Are Sinking Productivity & What to Do Ab...
byEdunomica
 
Kevin Martin: Empowering Your Board with the People Analytics That Matter
byEdunomica
 
Aizhan Tursunbayeva: Navigating Opportunities and Risks: A Responsible Approa...
byEdunomica
 
Alan Susi: Organizational Health: A People Team’s path to Minimum Viable ‘Wo...
byEdunomica
 
Cole Napper: Are you ready for generative AI in people analytics?
byEdunomica
 
Fahim Karim: Attrition Prevention
byEdunomica
 
Taras Filatov: Building your own metaverse & NFT app
byEdunomica
 
Alex Poon: Should you gamify community contributions?
byEdunomica
 
Julio Holon: Decentralised colaboration
byEdunomica
 
Startup Presentation: Gaianet
byEdunomica
 
Shawn Grubb: Minnows v. whales: Quadratic Governance to the rescue
byEdunomica
 
Joachim Stroh: Hypha DAO, the 3rd generation of DAOs
byEdunomica
 
Vikram Aditya: Biggest Opportunity Areas in the DAOverse
byEdunomica
 
Tamara Helenius: The Commons are Coming
byEdunomica
 

Recently uploaded

PDF
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
PPTX
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
PPTX
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
PPTX
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
PPTX
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
PDF
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
PDF
chapter one introduction to Probability theory.pdf
bySeid42
 
PPTX
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
PPTX
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
PDF
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
PDF
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
PDF
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
DOCX
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
PPTX
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
PPTX
I need all your support to get the data for the sections like photo gallery o...
byhsharsha1
 
PPTX
Organizational Structure and Design Topic 4
bydavenieralayaay
 
PPTX
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
PDF
Trustworthy AI : Governance of AI through Ethics
byKarim BaĂŻna
 
PDF
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
PDF
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
chapter one introduction to Probability theory.pdf
bySeid42
 
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
I need all your support to get the data for the sections like photo gallery o...
byhsharsha1
 
Organizational Structure and Design Topic 4
bydavenieralayaay
 
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
Trustworthy AI : Governance of AI through Ethics
byKarim BaĂŻna
 
Khan Traders Fish Meal – MSDS Material Safety Data Sheet
byKhan Traders Karachi
 
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 

Paige Roberts: Shortcut MLOps with In-Database Machine Learning

  • 1.
    Shortcut MLOps withIn-Database Machine Learning Paige Roberts Vertica Open Source Relations Manager
  • 2.
    Agenda MLOps – MachineLearning Operations Challenges with Operationalizing ML Is In-Database ML Possible? How Does In-DB ML Help MLOps? How Does In-DB ML Work? Questions
  • 3.
    Advanced Analytics UseCases are Everywhere It’s not an option; competing on data is the new style of business. Network Optimization Energy Optimization
  • 4.
  • 5.
  • 6.
    A lot ofmachine learning projects never make it to production https://www.redapt.com/blog/why-90-of-machine- learning-models-never-make-it-to-production https://www.forbes.com/sites/forbestechcouncil/2019/04/03/why- machine-learning-models-crash-and-burn-in-production/ https://thenewstack.io/add-it-up-how-long- does-a-machine-learning-deployment-take/ https://info.algorithmia.com/hubfs/2019/Whitepapers/ The-State-of-Enterprise-ML- 2020/Algorithmia_2020_State_of_Enterprise_ML.pdf https://towardsdatascience.com/why-90-percent-of-all-machine- learning-models-never-make-it-into-production-ce7e250d5a4a
  • 7.
    Data science journey– Now sample extraction ODBC/JDBC Data Preparation & Model Training Model Deployment & Management
  • 8.
    Separation of Devand Prod Environments is Problematic Development Environment  Notebooks – Interactive  Python – Pandas, SciKit  Tensorflow  Training  Batch  One Instance or single computer  Small sample – fits in memory  No worries about resource mgmt Production Environment No notebook – Automated Java, Scala JS, C++ Prediction/Inference Streaming  Many nodes or instances  Full data set – could be huge  Concurrent jobs, resource mgmt, autoscaling 8 Tools: Scale:
  • 9.
    Challenges Added Cost Additional hardware andsoftware required. Lots of components to make a whole solution. Requires Down Sampling Cannot process large data sets due to memory and computational limitations, resulting in inaccurate predictions Slower Time to Development Higher turnaround times for model building/scoring and need for moving, transforming large volumes of data between systems Slower Time to Deployment Inability to quickly deploy predictive models into production. Often must be rebuilt in another technology so it will scale. 7
  • 10.
    In-Database Machine Learning End-to-enddata science workflow in place SELECT IMPUTE SELECT NORMALIZE SELECT RF_CLASSIFIER SELECT ROC SQL User Defined Functions Distributed Analytical Database
  • 11.
    Is in-database MLreally possible?
  • 12.
    8 Databases thatSupport In-Database Machine Learning  Vertica  Amazon Redshift *  BlazingSQL  Google BigQuery *  IBM Db2 *  Kinetica  Microsoft SQL Server *  Oracle * https://www.infoworld.com/article/3607762/8-databases-supporting-in-database-machine-learning.html * Machine learning is an add-on or integrated product purchased with the database
  • 13.
    Many analytical databaseshave full set of ML functions https://www.vertica.com/python/ https://github.com/vertica/VerticaPy Correlations Heteroscedascity Trend & Stationarity Normality Cramer’s V Biserial Point Kendall Spearman Pearson Breush-Pagan Goldfeld-Quandt White’s Lagrange Engle Augmented Dickey-Fuller Mann-Kendall Normaltest Regular Joins Joins XGBoost Random Forest Tree Based Models Linear Regression Logistic Regression LinearSVC Linear Models KMeans Bisecting KMeans Clustering Time Series Joins Spatial Joins
  • 14.
    Deployment/ Management End-to-end machine learningprocess in database 14 Data Analysis/ Exploration Data Preparation Modeling Evaluation Business Understanding In-Database Scoring Speed at Scale Security Statistical Summary Sessionization Pattern Matching Date/Time Algebra Window/Partition Date Type Handling Sequences and more… Outlier Detection Normalization Imbalanced Data Processing Sampling Test/Validation Split Time Series Missing Value Imputation and more… K-Means Support Vector Machines Logistic, Linear, Ridge Regression Naïve Bayes Cross Validation and more… Model-level Stats ROC Tables Error Rate Lift Table Confusion Matrix R-Squared MSE XGBoost Filtering Feature Selection Correlation Matrices Table-Like Management Versioning Authorization and more… Random Forests and more… Principal Component Analysis PMML Import/Export TensorFlow Import AutoML
  • 15.
    Modern analytical databasesanalyze data in many formats – Semi-structured, streaming, text, complex, … data formats optimized for long-term storage Database format optimized for read/analyze performance Historical Recent Unified Analytics Analyze data where it sits.  Web logs  Sensor, IOT data  Time series  Click streams  Purchase history  Geospatial  … Streaming CSV Cloud storage, Hadoop (HDFS), S3 on prem data formats optimized for streaming messaging ROS
  • 16.
  • 17.
    Advantages of In-DatabaseMachine Learning 17 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer
  • 18.
    Advantages of In-DatabaseMachine Learning 18 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance
  • 19.
    Advantages of In-DatabaseMachine Learning 19 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management
  • 20.
    Advantages of In-DatabaseMachine Learning 20 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users
  • 21.
    Advantages of In-DatabaseMachine Learning 21 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities
  • 22.
    Advantages of In-DatabaseMachine Learning 22 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities Avoiding maintenance cost of a separate system
  • 23.
    Advantages of In-DatabaseMachine Learning 23 NODE 1 NODE 2 NODE N Schema Tables Models Schema Tables Models Schema Tables Models Network Eliminating overhead of data transfer Data Security and Provenance Model Storage and Management Serving concurrent users Highly scalable ML functionalities Avoiding maintenance cost of a separate system Identical Dev, Prod environments
  • 24.
  • 25.
    In-Database Random Forest AB C r1 r2 r3 NODE 1 1 NODE 2 2 NODE 3 3 2 A B C Tree1 r1 r2 A B C Tree2 r1 r3 A B C Tree3 r2 r3 3 Tree1 Tree2 Tree3 A B C r1 r3 A B C r2 r3 A B C r1 r2 TRAIN TRAIN TRAIN MERGE TABLE SPLIT DISTRIBUTE 1
  • 26.
    In-Database Extreme GradientBoosting A COMPUTE SUMMARY STATISTICS NODE 1 1 B COMPUTE SUMMARY STATISTICS NODE 2 2 C COMPUTE SUMMARY STATISTICS NODE 3 3 NODE 1 1 NODE 2 2 NODE 3 3 A B C FIND BEST SPLIT A B C FIND BEST SPLIT A B C FIND BEST SPLIT 1 2 ... ? Tree1 TreeN 3 ITERATIVE PROCESS TRAIN EXPORT SQL + memModel BINARY 4 JSON
  • 27.
    SARIMA In-Database SARIMA X Y 0 s+2s+4 s+6 s+8 s+10 s+12 V 2V 3V trend seasonality AR MA ARIMA value prediction confidence interval 1 2 3 4 seasonal decomposition NODE 1 NODE N X Xt Xt-1 Xt-2 𝜃𝜃𝑖𝑖 0 ≤ 𝑖𝑖 ≤ 𝑘𝑘𝑘 X Xt Xt-1 Xt-2 𝛼𝛼𝑗𝑗 0 ≤ 𝑗𝑗 ≤ 𝑘𝑘𝑘 𝜃𝜃𝑖𝑖 𝑘𝑘𝑘 ≤ 𝑖𝑖 ≤ 𝑞𝑞 𝛼𝛼𝑗𝑗 𝑘𝑘𝑘 ≤ 𝑗𝑗 ≤ 𝑝𝑝 COMPUTE COMPUTE A Simple Noniterative Estimator for Moving Average Models John W. Galbraith & Victoria Zinde-Walsh
  • 28.
    Integration with otherML technologies Geospatial Event Series Time series Text Analytics Pattern Matching Regression User-Defined Functions Machine Learning In-DB Data Science Pojo Frozen Graph SQL + memModel EXPORT EXPORT EXPORT EXPORT IMPORT SQL + memModel BINARY EXPORT XGBoost Random Forest ElasticNet Lasso Ridge KMeans Isolation Forest SVM JSON PCA
  • 29.
  • 30.
    Q&A VerticaPy: https://vertica.com/python PaigeRoberts Open Source Relations Manager E: Paige.Roberts@microfocus.com Try: https://www.vertica.com/try
  • 31.
  • 32.
    Previously, we usedApache Spark. Using Apache Spark cost 80% of the time from data I/O, only 20% of the time on machine learning. Now, we switched to using Vertica in-database machine learning and saved 80-90% data reading time. So, it’s had huge benefits in saving time and costs for model training. Xiaobo Ge, CTO, EOITek 32
  • 33.
    Using Vertica’s User-DefinedExtension, we have already built a bespoke lowpass filter to run against our data – that ensures top analytical performance without having to move the data to other systems or tools. Vertica analytics and machine learning ultimately delivers an edge that leads to more points, podiums, and wins for Jaguar Racing. Phil Charles 33 Technical Manager Jaguar Formula E racing team
  • 34.
    What People AreSaying 34 “Vertica is the path to production for machine learning.” “With Vertica in Eon Mode’s sub-clustering ability, we can also give more computation power to our users as needed. This is really helpful for our machine learning analysts.” “We’re using machine learning and deep learning, technologies, in the cloud, to consume, that data, leveraging things like Python and TensorFlow to build out and then operationalize those predictive models ...” “Vertica is a key technology, bringing in those diverse, data sets and building flexible, data models, and being able to do it quickly and at scale …” “If we have a particular use case, we can easily write our own functions in any language – Java, Python, R – and deploy this in Vertica, harnessing the power of parallel computing.” “Others are forced to sample. This is about using all of your data which is fantastic.”
  • 35.
    Analyze Data InMany Formats – Semi-structured, streaming, text, complex, … data formats optimized for long-term storage data format optimized for read/analyze performance Historical Recent Unified Analytics Analyze data where it sits.  Web logs  Sensor, IOT data  Time series  Click streams  Purchase history  Geospatial  … Streaming CSV Cloud storage, Hadoop (HDFS), S3 on prem data formats optimized for streaming messaging ROS
  • 36.
    Distributed databases automaticallyparallelize algorithm training Deliver accurate predictive analytics at speed and scale MPP architecture is a great fit – ML functions run in parallel across hundreds of nodes in a Vertica cluster Tree1 Tree2 Node1 Tree3 Tree4 Tree2n-1 Tree2n Node2 Noden
  • 37.
    High Performance +High Concurrency 37 Get data quickly enough to act upon it, explore your data interactively, and enable everyone to make their own data-driven decisions Enable everyone to make their own data-driven decisions. Get data quickly enough to act on it. Explore data interactively. Scale Data Volumes Scale Users and Jobs SQL Database + + Analytics & ML Query Engine
  • 38.
    Vertica ML is fastand scalable • Leverages MPP infrastructure and scale-out architecture for parallel high performance • No data movement across the system • No need to downsample, so accuracy is boosted Manage resources for many users or jobs Use a familiar interface • Manage, train, and deploy ML models using simple SQL calls • Integrate ML functions with other tools via SQL • Or use Jupyter and Python for familiar data science Proven infrastructure • No additional hardware and software to manage • Open integration with other solutions • Requires fewer people to setup and maintain • Resource isolation for multiple user sessions • Natural segmentation of user environments and privileges • Configurable memory usage for ML functions In-Database Machine Learning Advantages Enterprise-Grade Scalability Concurrency Productivity
  • 39.
    VerticaPy – PythonAPI for Machine Learning at Scale 28 Benefits: Jupyter/Python – popular interactive tool of choice Graphs and visualizations – even on large data sets Data stays in database – security, integrity Distributed Engine - heavy computation done by optimized MPP engine Open source – you can contribute! Manage models – store in DB Results where they’re needed – available in DB for other analyses, visualization, or driving applications and AI
  • 40.
    VerticaPy  Python library Open source  https://github.com/vertica/VerticaPy  Pandas and Scikit Learn functionality  Conduct data science projects in Vertica – high performance, high scale analytical database  Takes advantage of built-in parallel analytics and machine learning  You write Python in a Jupyter Notebook, the database executes it on the full data set  Uses vertica-python native Python client for Vertica created by Uber with contributions from Twitter, Palantir, Etsy, Kayak, Gooddata, … - https://github.com/vertica/vertica-python 40
  • 41.
    Bruce Yen, WW BusinessIntelligence Leader “Our Vertica platform is instrumental in many areas of our business—creating predictive algorithms, serving up product recommendations, powering insight to our mobile apps, and generating daily reports and ad- hoc queries. It’s crucial for enabling us to be more agile with data.”
  • 42.
    Challenge - Analyze largevolumes of endpoint data to detect indications of cyber attacks Result - Provides near real-time data insertion across hundreds of thousands of endpoints; results in enterprise-grade threat detection quality - Ensures rapid querying, essential for analysts to outpace attackers - Delivers higher detection rates and dramatically reduces false positives - Enables analysts to perform complex queries over large volumes of data; presents results in easy-to-understand interface - Provides credible, robust, and scalable big data technology that will be trusted by its customers Cyberbit Surpasses Conventional Security Systems Vertica uses big data, behavioral analysis, and machine learning to optimize detection and response
  • 43.
    Our Vertica platformis instrumental in many areas of our business—creating predictive algorithms, serving up product recommendations, powering insight to our mobile apps, and generating daily reports and ad- hoc queries. It’s crucial for enabling us to be more agile with data. Bruce Yen 43 WW Business Intelligence Leader Guess
  • 44.
    Advantages of Python+ MPP Analytical Database MPP Scale Clusters with no name node or other single point of failure allow unlimited scale Speed and Concurrency Query optimization and resource management across multiple nodes Features ML algorithm parallelization, moving windows, geospatial analysis, time series joins, fast data prep... Open Architecture Integration with many other applications - BI, ETL, Kafka, Spark, Data Science Labs … Broad Utility Many functionalities - one of the most broadly useful programming languages. Flexibility It Many right paths to do things, a lot of freedom, works on many platforms. Ease of Use High level of abstraction makes Python one of the easiest programming languages. Strong Community Most data scientists master Python. Many useful packages (pandas, scikit, …)
  • 45.
    Linear regression Analytic databasefunctions include machine learning algorithms K-means Logistic regression Naive Bayes Random Forest SVM Predict customer retention Forecast sales revenues Customer segmentation Predict sensor failure Classify gene expression data for drug discovery Refine keywords to improve Click Through Rate (CTR)
  • 46.
    20 Bring your R, TensorFlow,and Python code inside the database – analyze the data in place.
  • 47.
  • 48.
    Predictive Maintenance Demo 48 Analyzesensor data from cooling towers across the US , enabling equipment manufacturers to predict and prevent equipment failure
  • 49.
    Flight Tracker Demo 49 Verticaoperates at the “edge” with flight track detail. Sensor data is collected using a Raspberry pi with radio receiver and antenna. Data is loaded into Vertica as thousands of records per second and builds to billions of flight data points collected within a 250-mile radius. https://www.vertica.com/blog/blog-post-series-using-vertica-track- commercial-aircraft-near-real-time/
  • 50.
  • 51.
     Huge improvementsin stability and performance after moving to Vertica  24 mins on Spark, 3 mins in Vertica  Can incorporate other data like weather to optimize predictive thermostat efficiency after moving to Vertica ML  Citing speed of analytics, ease of use when coding in SQL, and improvements in the accuracy of models after moving workloads to Vertica ML  Solving issues that were previously unsolvable  Minimal hardware, software, and personnel investments when differentiating with data science. 51
  • 52.
    Moving data scienceworkloads from Spark on Hadoop to in-database Improvements in stability and performance Creating customer segmentation via clustering algorithms on a 15 million customer dataset took 24 mins on Spark - 3 mins in database Concurrently running other algorithms without performance impact Cardlytics partners with more than 1,500 financial institutions to run their online and mobile banking rewards programs, which gives us a robust view into where and when consumers are spending their money.
  • 53.
    Fidelis Cybersecurity protectsthe world's most sensitive data by identifying and removing attackers no matter where they're hiding on your network and endpoints. 53 Data science team was experiencing challenges with performance while using Spark ML Moving workloads from Spark ML to in-database ML provided: Speed of analytics Ease of use when coding in SQL Increased accuracy of models
  • 54.
    Some Vertica IoTCustomer Resources Case Studies  Anritsu ROI case study: https://www.vertica.com/wp- content/uploads/2017/01/r24-HPE-Vertica-ROI-case-study-Anritsu.pdf  Infographic of ROI: https://www.vertica.com/wp- content/uploads/2017/03/Anritsu-v2.pdf  Nimble Storage ROI case study: https://www.vertica.com/wp- content/uploads/2017/08/Nimble-Storage-ROI.pdf  Optimal+ case study: https://www.vertica.com/wp- content/uploads/2017/06/Optimal-MF-rebrand-FINAL-lo-res.pdf  *Climate Corp case study: https://www.vertica.com/wp- content/uploads/2019/01/Climate-Corp_Success-Story-FINAL.pdf Webcasts – Data Disruptors  Philips: https://www.brighttalk.com/webcast/10477/277693  Climate Corp: https://www.brighttalk.com/webcast/8913/336201  Nimble Storage (HPE InfoBright): https://www.brighttalk.com/webcast/8913/330769  Zebrium: https://www.brighttalk.com/webcast/8913/332838  Simpli.fi: https://www.brighttalk.com/webcast/8913/354325/simpli-fi- delivers-advertising-insights-on-billions-of-streaming-bid-messages Videos  Optimal+: https://www.youtube.com/watch?v=IZkkoy5ZT1M&feature=youtu.be  Anritsu: https://www.youtube.com/watch?v=QZ5vWqblVXU&feature=youtu.be 54