Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AI Modernization at AT&T
and the application to
Fraud with Databricks
Mark Austin, VP Data Science, AT&T
Prince Paulraj, A...
Agenda
§ AT&T’s History in AI
§ Fraud AI Application
§ AI Modernization & Strategy
▪ Create AI
▪ Deploy & Serve AI
▪ Monit...
01
02
03
04
05
AT&T’s: History in Transformative AI/Data Science…
1950
1955
“Artificial
Intelligence” term first
coined, A...
Mobility Fraud: Organized Crime Stealing “iPhones”
4
Identity Theft:
Identities stolen via
social engineering
or otherwise...
5
Year 1: Fraud stops using rules only
Year 3: Fraud stops using AI/ML (+25 Models)
Deploying Real-Time AI/ML in addition ...
6
Develop
Features
Productionize
Model Pipeline
Discover
Data
Build Models Deploy
Model
Integrate
Model
Monitor
Model
Can ...
Create AI
Getting the best features and models…
▪ Batch data pipeline
▪ Near real-time data
pipeline
▪ Streaming feature
p...
8
Pinnacle crowd sourcing
Competitive/collaborative internal platform
Pinnacle– AI as a team sport yields ~29% improvement...
Deploy & Serve AI
Deploying models to score at 10M+/day @50ms latency
▪ Model Lineage and
Versioning
▪ A/B Model
Framework...
10
Atlantis – Multi-Pipeline Feature Store for Machine Learning
Databricks ML Pipeline
Snowflake ML Pipeline
Pinnacle ML P...
11
Feature Store Benefits:
Ensures the “same features” are used in training and serving, preventing serving ML loss
Traini...
Monitor AI
Needs to cover data, model, infra, and process
▪ Detecting Data
Quality issues
▪ Data drifting -
Feature value ...
13
Watchtower – An end-to-end Data, Features and Models Monitoring Engine
Cases created for Mitigation,
Alerts, Notificati...
SIFT
Learn Document Evaluate
Use case
ML Model
MLDB
ML Project Metadata
Data Catalog AIaaS
Model Catalog
Model Metadata
Di...
Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

AI Modernization at AT&T and the Application to Fraud with Databricks

Download to read offline

AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.

  • Be the first to like this

AI Modernization at AT&T and the Application to Fraud with Databricks

  1. 1. AI Modernization at AT&T and the application to Fraud with Databricks Mark Austin, VP Data Science, AT&T Prince Paulraj, AVP, Data Insights, AT&T
  2. 2. Agenda § AT&T’s History in AI § Fraud AI Application § AI Modernization & Strategy ▪ Create AI ▪ Deploy & Serve AI ▪ Monitor AI ▪ Govern AI § Conclusion and Opportunities
  3. 3. 01 02 03 04 05 AT&T’s: History in Transformative AI/Data Science… 1950 1955 “Artificial Intelligence” term first coined, AT&T, IBM, Harvard, Dartmouth Shannon, AT&T Bell Labs “Programming a Computer to Play Chess” 1970’s Unix, C,C++, and Statistical Programming (S) which becomes 1980’s-90’s Neural Network foundational work on Conv Neural Nets, AT&T 2000’s AT&T Wins Netflix Recommender Competition AI on Tech, Media, Telecom Applied AI/ML 3
  4. 4. Mobility Fraud: Organized Crime Stealing “iPhones” 4 Identity Theft: Identities stolen via social engineering or otherwise and used to obtain handset to resell overseas Gaming Fraud: “Gaming Customer” has no intention to pay and uses their “credit” to sell new iPhone (and other devices) to fraud crime ring Illegal Unlocks: Bribing or impersonating Call center employees to unlock phones that are under contract Retail, Care, Digital xxxx _ password login BILLION DOLLAR Mobility Fraud Industry affecting all US Carriers Combatting Fraud using Realtime ML/AI
  5. 5. 5 Year 1: Fraud stops using rules only Year 3: Fraud stops using AI/ML (+25 Models) Deploying Real-Time AI/ML in addition to Rules is Effective Year 2: Fraud stops using AI/ML (5 Models) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Fraud Events NOT Stopped 2018 2019 2020 Rules only ML+Rules Year 1 Year 2 Year 3 ML1 ML2 ML3 ML4 ML5 ML6 ML7-8 ML9 ML10 ML11-12 ML13 -14 ML15-16 ML17-18 ML19-20 ML21-22 ML23-26 Combatting Fraud using Realtime ML/AI The Technical Challenge: • Scoring >10M transactions/day • Scoring latency < 50ms • 100’s of real-time features • >4x as many batch features
  6. 6. 6 Develop Features Productionize Model Pipeline Discover Data Build Models Deploy Model Integrate Model Monitor Model Can you put these features into production? Can you put this model into production? Can you integrate this model and app? Can you setup monitoring? Can you give me the access to the raw data? Can I use this data for this model? Create AI Deploy & Serve AI Monitor AI AIaaS - AI Modernization using Can we visualize model outcome & insights Can you build the best model? Govern AI Can you get the data fast? Can you notify for model retraining?
  7. 7. Create AI Getting the best features and models… ▪ Batch data pipeline ▪ Near real-time data pipeline ▪ Streaming feature pipeline ▪ Speed to market ▪ Code once and model many ▪ Merged or derived features sets across enterprise ▪ Collaboration • Share/Reuse Features • Creating Features ▪ Model and hyperopt experiment scorecard ▪ Model Repository • Experiment & Catalog ▪ Transformative features (autoML) • Best Model Enterprise Atlantis The Enterprise Feature Store Model Benchmarking by Individuals & Robots Model Benchmarking Amongst “the Crowd” 7
  8. 8. 8 Pinnacle crowd sourcing Competitive/collaborative internal platform Pinnacle– AI as a team sport yields ~29% improvement Pinnacle by the numbers…. 219 Competitions 1,101 people 4 Automl bots 29% avg. improvement! >4,750 models benchmarked ML 8 Pinnacle crowd sourcing Competitive/collaborative internal platform Pinnacle– AI as a team sport yields ~29% improvement Pinnacle by the numbers…. 219 Competitions 1,101 people 4 Automl bots 29% avg. improvement! >4,750 models benchmarked ML
  9. 9. Deploy & Serve AI Deploying models to score at 10M+/day @50ms latency ▪ Model Lineage and Versioning ▪ A/B Model Framework ▪ Model Lifecycle management ▪ Higher throughput and scalability ▪ Feature versioning ▪ Time travel model evaluation ▪ Backfilling features • Model Offline Training • Model Deployment ▪ Lightening fast ▪ High scalability ▪ Consistent features ▪ High availability ▪ Feature Time to live • Model Online Scoring ▪ Metadata management ▪ Discoverability ▪ Access Control ▪ Feature Health & statistics ▪ Compliance & Legal • Feature Governance Atlantis The Enterprise Feature Store 9
  10. 10. 10 Atlantis – Multi-Pipeline Feature Store for Machine Learning Databricks ML Pipeline Snowflake ML Pipeline Pinnacle ML Pipeline H2O Driverless ML Pipeline Feature Engineering Pipelines Offline/Online Feature Store Real-Time Data Batch Data Raw Data Model Scoring (mS) Model Training Jupyter ML Pipeline Palantir ML Pipeline
  11. 11. 11 Feature Store Benefits: Ensures the “same features” are used in training and serving, preventing serving ML loss Training Features Serving Features Blue (serving predicted) Offline/Static N/A Green (serving actual of Blue ) Offline/Static Online Red (serving actual when train/serve in sync (Feature Store) Online Online Using Feature Store For Train/Serve Improves Lift ~2X @first decile
  12. 12. Monitor AI Needs to cover data, model, infra, and process ▪ Detecting Data Quality issues ▪ Data drifting - Feature value is missing or invalid ▪ Data readiness - Out of SLA ▪ Model Drifting ▪ Performance Issues ▪ Production Model Governance ▪ Visualization of model health • Model • Data ▪ Out of SLA response ▪ CPU, RAM, I/O usage ▪ Application, mS or VM goes down ▪ Network and connectivity issues ▪ Correlation of system and model performance • Infrastructure ▪ Drag and drop to create custom rules ▪ Auto remediations like re-training or rollback model version ▪ Workflow of actions ▪ Predict the Root cause • Process AI Watchtower – An end-to-end Data, and Models Monitoring & Decision engine 12
  13. 13. 13 Watchtower – An end-to-end Data, Features and Models Monitoring Engine Cases created for Mitigation, Alerts, Notifications Apply ML models & Business Rules Setup Monitoring Real Time Data pipelines Decision Engine Any data pipelines into Event Hub Is available for monitoring Take Auto/Manual Actions Downstream Systems/Feedback Action & Remediation How? Drifting Data Rules Model Actions BYO Doing it in a Self-Service Way Multi-tenant subscription API Integration to platforms Self-service experience On-Perm & Cloud availability Model 3 – Percent Scored Model 4 - Percent Scored Model 5 - Percent Scored Model 6 - Percent Scored Model 7 - Percent Scored Model 1 – Percent Scored Model 2 – Percent Scored
  14. 14. SIFT Learn Document Evaluate Use case ML Model MLDB ML Project Metadata Data Catalog AIaaS Model Catalog Model Metadata Disparate impact Debiased Model Feature importance Drift assessment Bias Detection Bias Mitigation Explainability Data Drift Open Open-Source Tools Vendor Tools Output Business decision Privacy (PRR) Process Legal Privacy Process Docs Govern AI Using AT&T’s System for Investigating Fairness and Transparency (SIFT) 14 Atlantis Feature Store
  15. 15. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

AT&T has been involved in AI from the beginning, with many firsts; “first to coin the term AI”, “inventors of R”, “foundational work on Conv. Neural Nets”, etc. and we have applied AI to hundreds of solutions. Today we are modernizing these AI solutions in the cloud with the help of Databricks and a variety of in-house developments. This talk will highlight our AI modernization effort along with its application to Fraud which is one of our biggest benefitting applications.

Views

Total views

106

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×