SlideShare a Scribd company logo

Apache Superset at Airbnb

•
•
B
Bill Liu

Presented at AI NEXTCon Seattle 1/17-20, 2018 http://aisea18.xnextcon.com join our free online AI group with 50,000+ tech engineers to learn and practice AI technology, including: latest AI news, tech articles/blogs, tech talks, tutorial videos, and hands-on workshop/codelabs, on machine learning, deep learning, data science, etc..

Technology
1 of 24
MAXIME BEAUCHEMIN / MAY 2017 Apache Superset a modern, enterprise-ready business intelligence web application
Slide Title Here Optional subtitle goes here • Cereal Entrepreneur: Creative. Embraces constraints. Solution-oriented. Tenacious. • Be a Host: Collaborative. Anticipates the needs of others. Prepared. Authentic. Listens. • Embrace the Adventure: Flexible. Risk tolerant. Always learning. Curious. Open-minded. • Simplify: Prioritizes. Distills a problem to its essence. Makes and communicates clear decisions. • Champion the Mission: Passionate. Committed. Optimistic. Puts the Airbnb community first. • Every Frame Matters: Thinks holistically. Rigorous about quality. Appreciates the details and 
 prioritizes the right ones. MaximeBeauchemin
* explore your data! * create interactive dashboards * share discoveries A modern, enterprise-ready business intelligence web application
The Superset Vision Any visualizationAny database microscopic amount of work
Event Logs MySQL Dumps Gold Hive Cluster HDFS Spark Cluster Airpal Airflow Scheduling Presto Cluster Silver Hive Cluster HDFS Replication Kafka Sqoop Tableau S3 Superset ! Druid
* Data is too strategic to depend on vendors! * Tableau doesn’t support Presto & Druid * Tableau extracts don’t scale well * Buying means lock-in and increasing costs * We need deep integration with our stack * We’re builders not buyers!
Live demo!
The Stack * Python backend * Flask App Builder (authentication, roles, CRUD, …) * Pandas for rich analytics * SqlAlchemy (multi dialect SQL toolkit) * Javascript frontend * React / Redux * ES6 / Webpack / npm * d3.js! * nvd3.org
Security * Provided by Flask AppBuilder (python web framework) * Easily integrate with: OpenID, LDAP, REMOTE_USER, OAUTH, or use the builtin database * Ships with 3 roles: * Admin (all access) * Alpha (all access but cannot alter permissions) * Gamma (per-datasource / table access) * Fine grain controls to create new roles
A thin Semantic Layer * Verbose names and long descriptions for columns and metrics * Add calculated fields and metrics as SQL expression * Set how individual columns are exposed
Caching! * Provided by flask-cache * Backends: memcache, redis, filesystem, memory, … * cascading timeout configuration * UI is upfront about staleness * allows to force-refresh
Open Source!
* Grow a community! Joining the ASF * UX -> smoothen common flows * Improve SQL Lab * Ship visualizations & controls as React.js components * DSL for the semantic layer What’s next?
github.com/airbnb/superset
Q?

Recommended

Open Source DataViz with Apache Superset
Open Source DataViz with Apache SupersetOpen Source DataViz with Apache Superset
Open Source DataViz with Apache SupersetCarl W. Handlin
 
DAST in CI/CD pipelines using Selenium & OWASP ZAP
DAST in CI/CD pipelines using Selenium & OWASP ZAPDAST in CI/CD pipelines using Selenium & OWASP ZAP
DAST in CI/CD pipelines using Selenium & OWASP ZAPsrini0x00
 
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...Abhay Bhargav
 
Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema
 
Peeling the Onion: Making Sense of the Layers of API Security
Peeling the Onion: Making Sense of the Layers of API SecurityPeeling the Onion: Making Sense of the Layers of API Security
Peeling the Onion: Making Sense of the Layers of API SecurityMatt Tesauro
 
facebook architecture for 600M users
facebook architecture for 600M usersfacebook architecture for 600M users
facebook architecture for 600M usersJongyoon Choi
 
Secure Code Review 101
Secure Code Review 101Secure Code Review 101
Secure Code Review 101Narudom Roongsiriwong, CISSP
 
OWASP Top 10 API Security Risks
OWASP Top 10 API Security RisksOWASP Top 10 API Security Risks
OWASP Top 10 API Security RisksIndusfacePvtLtd
 

Recommended

Open Source DataViz with Apache Superset
Open Source DataViz with Apache SupersetOpen Source DataViz with Apache Superset
Open Source DataViz with Apache SupersetCarl W. Handlin
 
DAST in CI/CD pipelines using Selenium & OWASP ZAP
DAST in CI/CD pipelines using Selenium & OWASP ZAPDAST in CI/CD pipelines using Selenium & OWASP ZAP
DAST in CI/CD pipelines using Selenium & OWASP ZAPsrini0x00
 
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...
An Attacker's View of Serverless and GraphQL Apps - Abhay Bhargav - AppSec Ca...Abhay Bhargav
 
Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Apache Superset - open source data exploration and visualization (Conclusion ...
Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema
 
Peeling the Onion: Making Sense of the Layers of API Security
Peeling the Onion: Making Sense of the Layers of API SecurityPeeling the Onion: Making Sense of the Layers of API Security
Peeling the Onion: Making Sense of the Layers of API SecurityMatt Tesauro
 
facebook architecture for 600M users
facebook architecture for 600M usersfacebook architecture for 600M users
facebook architecture for 600M usersJongyoon Choi
 
Secure Code Review 101
Secure Code Review 101Secure Code Review 101
Secure Code Review 101Narudom Roongsiriwong, CISSP
 
OWASP Top 10 API Security Risks
OWASP Top 10 API Security RisksOWASP Top 10 API Security Risks
OWASP Top 10 API Security RisksIndusfacePvtLtd
 
How to submit ios app in Appstore
How to submit ios app in AppstoreHow to submit ios app in Appstore
How to submit ios app in AppstoreNandini Gautam
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elkRushika Shah
 
Weaponizing Your DevOps Pipeline
Weaponizing Your DevOps PipelineWeaponizing Your DevOps Pipeline
Weaponizing Your DevOps PipelinePuma Security, LLC
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEMJohn Hubbard
 
12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready Solutions12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready SolutionsKashif Ali Siddiqui
 
NoSql Injection
NoSql InjectionNoSql Injection
NoSql InjectionNSConclave
 
Introduction to Kong API Gateway
Introduction to Kong API GatewayIntroduction to Kong API Gateway
Introduction to Kong API GatewayYohann Ciurlik
 
Application Monitoring using Datadog
Application Monitoring using DatadogApplication Monitoring using Datadog
Application Monitoring using DatadogMukta Aphale
 
Inside SecOps at bet365
Inside SecOps at bet365 Inside SecOps at bet365
Inside SecOps at bet365 Splunk
 
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)OWASP Ottawa
 
Source Code Analysis with SAST
Source Code Analysis with SASTSource Code Analysis with SAST
Source Code Analysis with SASTBlueinfy Solutions
 
Microservices & API Gateways
Microservices & API Gateways Microservices & API Gateways
Microservices & API Gateways Kong Inc.
 
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...Magno Logan
 
How OpenShift SDN helps to automate
How OpenShift SDN helps to automateHow OpenShift SDN helps to automate
How OpenShift SDN helps to automateIlkka Tengvall
 
Efficient API delivery with APIOps
Efficient API delivery with APIOpsEfficient API delivery with APIOps
Efficient API delivery with APIOpsSven Bernhardt
 
Secure Design: Threat Modeling
Secure Design: Threat ModelingSecure Design: Threat Modeling
Secure Design: Threat ModelingCigital
 
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...AlienVault
 
Introduction To OWASP
Introduction To OWASPIntroduction To OWASP
Introduction To OWASPMarco Morana
 
API Design- Best Practices
API Design- Best PracticesAPI Design- Best Practices
API Design- Best PracticesPrakash Bhandari
 
BugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed AdamBugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed AdamMohammed Adam
 
SMACK Stack 1.1
SMACK Stack 1.1SMACK Stack 1.1
SMACK Stack 1.1Joe Stein
 
Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCacheUnleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCacheAmazon Web Services
 

More Related Content

What's hot

How to submit ios app in Appstore
How to submit ios app in AppstoreHow to submit ios app in Appstore
How to submit ios app in AppstoreNandini Gautam
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elkRushika Shah
 
Weaponizing Your DevOps Pipeline
Weaponizing Your DevOps PipelineWeaponizing Your DevOps Pipeline
Weaponizing Your DevOps PipelinePuma Security, LLC
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEMJohn Hubbard
 
12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready Solutions12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready SolutionsKashif Ali Siddiqui
 
NoSql Injection
NoSql InjectionNoSql Injection
NoSql InjectionNSConclave
 
Introduction to Kong API Gateway
Introduction to Kong API GatewayIntroduction to Kong API Gateway
Introduction to Kong API GatewayYohann Ciurlik
 
Application Monitoring using Datadog
Application Monitoring using DatadogApplication Monitoring using Datadog
Application Monitoring using DatadogMukta Aphale
 
Inside SecOps at bet365
Inside SecOps at bet365 Inside SecOps at bet365
Inside SecOps at bet365 Splunk
 
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)OWASP Ottawa
 
Source Code Analysis with SAST
Source Code Analysis with SASTSource Code Analysis with SAST
Source Code Analysis with SASTBlueinfy Solutions
 
Microservices & API Gateways
Microservices & API Gateways Microservices & API Gateways
Microservices & API Gateways Kong Inc.
 
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...Magno Logan
 
How OpenShift SDN helps to automate
How OpenShift SDN helps to automateHow OpenShift SDN helps to automate
How OpenShift SDN helps to automateIlkka Tengvall
 
Efficient API delivery with APIOps
Efficient API delivery with APIOpsEfficient API delivery with APIOps
Efficient API delivery with APIOpsSven Bernhardt
 
Secure Design: Threat Modeling
Secure Design: Threat ModelingSecure Design: Threat Modeling
Secure Design: Threat ModelingCigital
 
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...AlienVault
 
Introduction To OWASP
Introduction To OWASPIntroduction To OWASP
Introduction To OWASPMarco Morana
 
BugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed AdamBugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed AdamMohammed Adam
 

What's hot (20)

How to submit ios app in Appstore
How to submit ios app in AppstoreHow to submit ios app in Appstore
How to submit ios app in Appstore
 
Log analysis using elk
Log analysis using elkLog analysis using elk
Log analysis using elk
 
Weaponizing Your DevOps Pipeline
Weaponizing Your DevOps PipelineWeaponizing Your DevOps Pipeline
Weaponizing Your DevOps Pipeline
 
The Elastic Stack as a SIEM
The Elastic Stack as a SIEMThe Elastic Stack as a SIEM
The Elastic Stack as a SIEM
 
12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready Solutions12 factor app - Core Guidelines To Cloud Ready Solutions
12 factor app - Core Guidelines To Cloud Ready Solutions
 
NoSql Injection
NoSql InjectionNoSql Injection
NoSql Injection
 
Introduction to Kong API Gateway
Introduction to Kong API GatewayIntroduction to Kong API Gateway
Introduction to Kong API Gateway
 
Application Monitoring using Datadog
Application Monitoring using DatadogApplication Monitoring using Datadog
Application Monitoring using Datadog
 
Inside SecOps at bet365
Inside SecOps at bet365 Inside SecOps at bet365
Inside SecOps at bet365
 
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
 
Source Code Analysis with SAST
Source Code Analysis with SASTSource Code Analysis with SAST
Source Code Analysis with SAST
 
Microservices & API Gateways
Microservices & API Gateways Microservices & API Gateways
Microservices & API Gateways
 
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
DevSecOps - Integrating Security in the Development Process (with memes) - Ma...
 
How OpenShift SDN helps to automate
How OpenShift SDN helps to automateHow OpenShift SDN helps to automate
How OpenShift SDN helps to automate
 
Efficient API delivery with APIOps
Efficient API delivery with APIOpsEfficient API delivery with APIOps
Efficient API delivery with APIOps
 
Secure Design: Threat Modeling
Secure Design: Threat ModelingSecure Design: Threat Modeling
Secure Design: Threat Modeling
 
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
How to Normalize Threat Intelligence Data from Multiple Sources - Tech Talk T...
 
Introduction To OWASP
Introduction To OWASPIntroduction To OWASP
Introduction To OWASP
 
API Design- Best Practices
API Design- Best PracticesAPI Design- Best Practices
API Design- Best Practices
 
BugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed AdamBugBounty Roadmap with Mohammed Adam
BugBounty Roadmap with Mohammed Adam
 

Similar to Apache Superset at Airbnb

SMACK Stack 1.1
SMACK Stack 1.1SMACK Stack 1.1
SMACK Stack 1.1Joe Stein
 
Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCacheUnleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCacheAmazon Web Services
 
Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCache Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCache Amazon Web Services
 
Above the cloud joarder kamal
Above the cloud joarder kamalAbove the cloud joarder kamal
Above the cloud joarder kamalJoarder Kamal
 
Michael stack -the state of apache h base
Michael stack -the state of apache h baseMichael stack -the state of apache h base
Michael stack -the state of apache h basehdhappy001
 
Cloud Native Camel Riding
Cloud Native Camel RidingCloud Native Camel Riding
Cloud Native Camel RidingChristian Posta
 
Planning to Fail #phpuk13
Planning to Fail #phpuk13Planning to Fail #phpuk13
Planning to Fail #phpuk13Dave Gardner
 
Choosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overviewChoosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet
 
Docker - Demo on PHP Application deployment
Docker - Demo on PHP Application deployment Docker - Demo on PHP Application deployment
Docker - Demo on PHP Application deployment Arun prasath
 
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache Accumulo
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache AccumuloReal-Time Distributed and Reactive Systems with Apache Kafka and Apache Accumulo
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache AccumuloJoe Stein
 
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...Accumulo Summit
 
Chicago Microservices Integration Talk
Chicago Microservices Integration TalkChicago Microservices Integration Talk
Chicago Microservices Integration TalkChristian Posta
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesAdrian Cockcroft
 
2012-03-15 What's New at Red Hat
2012-03-15 What's New at Red Hat2012-03-15 What's New at Red Hat
2012-03-15 What's New at Red HatShawn Wells
 
Lessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeYevgeniy Brikman
 
Web Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud PlatformWeb Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud PlatformSudhir Tonse
 
Run Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesRun Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesBernd Ocklin
 
Integration in the age of DevOps
Integration in the age of DevOpsIntegration in the age of DevOps
Integration in the age of DevOpsAlbert Wong
 
Microservices and Friends
Microservices and FriendsMicroservices and Friends
Microservices and FriendsYun Zhi Lin
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCScalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson
 

Similar to Apache Superset at Airbnb (20)

SMACK Stack 1.1
SMACK Stack 1.1SMACK Stack 1.1
SMACK Stack 1.1
 
Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCacheUnleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCache
 
Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCache Unleash the Power of Redis with Amazon ElastiCache
Unleash the Power of Redis with Amazon ElastiCache
 
Above the cloud joarder kamal
Above the cloud joarder kamalAbove the cloud joarder kamal
Above the cloud joarder kamal
 
Michael stack -the state of apache h base
Michael stack -the state of apache h baseMichael stack -the state of apache h base
Michael stack -the state of apache h base
 
Cloud Native Camel Riding
Cloud Native Camel RidingCloud Native Camel Riding
Cloud Native Camel Riding
 
Planning to Fail #phpuk13
Planning to Fail #phpuk13Planning to Fail #phpuk13
Planning to Fail #phpuk13
 
Choosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overviewChoosing PaaS: Cisco and Open Source Options: an overview
Choosing PaaS: Cisco and Open Source Options: an overview
 
Docker - Demo on PHP Application deployment
Docker - Demo on PHP Application deployment Docker - Demo on PHP Application deployment
Docker - Demo on PHP Application deployment
 
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache Accumulo
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache AccumuloReal-Time Distributed and Reactive Systems with Apache Kafka and Apache Accumulo
Real-Time Distributed and Reactive Systems with Apache Kafka and Apache Accumulo
 
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...
Accumulo Summit 2015: Real-Time Distributed and Reactive Systems with Apache ...
 
Chicago Microservices Integration Talk
Chicago Microservices Integration TalkChicago Microservices Integration Talk
Chicago Microservices Integration Talk
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
 
2012-03-15 What's New at Red Hat
2012-03-15 What's New at Red Hat2012-03-15 What's New at Red Hat
2012-03-15 What's New at Red Hat
 
Lessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure code
 
Web Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud PlatformWeb Scale Applications using NeflixOSS Cloud Platform
Web Scale Applications using NeflixOSS Cloud Platform
 
Run Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesRun Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in Kubernetes
 
Integration in the age of DevOps
Integration in the age of DevOpsIntegration in the age of DevOps
Integration in the age of DevOps
 
Microservices and Friends
Microservices and FriendsMicroservices and Friends
Microservices and Friends
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCScalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
 

More from Bill Liu

Walk Through a Real World ML Production Project
Walk Through a Real World ML Production ProjectWalk Through a Real World ML Production Project
Walk Through a Real World ML Production ProjectBill Liu
 
Redefining MLOps with Model Deployment, Management and Observability in Produ...
Redefining MLOps with Model Deployment, Management and Observability in Produ...Redefining MLOps with Model Deployment, Management and Observability in Produ...
Redefining MLOps with Model Deployment, Management and Observability in Produ...Bill Liu
 
Productizing Machine Learning at the Edge
Productizing Machine Learning at the EdgeProductizing Machine Learning at the Edge
Productizing Machine Learning at the EdgeBill Liu
 
Transformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroTransformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroBill Liu
 
Deep AutoViML For Tensorflow Models and MLOps Workflows
Deep AutoViML For Tensorflow Models and MLOps WorkflowsDeep AutoViML For Tensorflow Models and MLOps Workflows
Deep AutoViML For Tensorflow Models and MLOps WorkflowsBill Liu
 
Metaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at NetflixMetaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at NetflixBill Liu
 
Practical Crowdsourcing for ML at Scale
Practical Crowdsourcing for ML at ScalePractical Crowdsourcing for ML at Scale
Practical Crowdsourcing for ML at ScaleBill Liu
 
Building large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBuilding large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBill Liu
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsBill Liu
 
Big Data and AI in Fighting Against COVID-19
Big Data and AI in Fighting Against COVID-19Big Data and AI in Fighting Against COVID-19
Big Data and AI in Fighting Against COVID-19Bill Liu
 
Highly-scalable Reinforcement Learning RLlib for Real-world Applications
Highly-scalable Reinforcement Learning RLlib for Real-world ApplicationsHighly-scalable Reinforcement Learning RLlib for Real-world Applications
Highly-scalable Reinforcement Learning RLlib for Real-world ApplicationsBill Liu
 
Build computer vision models to perform object detection and classification w...
Build computer vision models to perform object detection and classification w...Build computer vision models to perform object detection and classification w...
Build computer vision models to perform object detection and classification w...Bill Liu
 
Causal Inference in Data Science and Machine Learning
Causal Inference in Data Science and Machine LearningCausal Inference in Data Science and Machine Learning
Causal Inference in Data Science and Machine LearningBill Liu
 
Weekly #106: Deep Learning on Mobile
Weekly #106: Deep Learning on MobileWeekly #106: Deep Learning on Mobile
Weekly #106: Deep Learning on MobileBill Liu
 
Weekly #105: AutoViz and Auto_ViML Visualization and Machine Learning
Weekly #105: AutoViz and Auto_ViML Visualization and Machine LearningWeekly #105: AutoViz and Auto_ViML Visualization and Machine Learning
Weekly #105: AutoViz and Auto_ViML Visualization and Machine LearningBill Liu
 
AISF19 - On Blending Machine Learning with Microeconomics
AISF19 - On Blending Machine Learning with MicroeconomicsAISF19 - On Blending Machine Learning with Microeconomics
AISF19 - On Blending Machine Learning with MicroeconomicsBill Liu
 
AISF19 - Travel in the AI-First World
AISF19 - Travel in the AI-First WorldAISF19 - Travel in the AI-First World
AISF19 - Travel in the AI-First WorldBill Liu
 
AISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeAISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeBill Liu
 
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...Bill Liu
 
Toronto meetup 20190917
Toronto meetup 20190917Toronto meetup 20190917
Toronto meetup 20190917Bill Liu
 

More from Bill Liu (20)

Walk Through a Real World ML Production Project
Walk Through a Real World ML Production ProjectWalk Through a Real World ML Production Project
Walk Through a Real World ML Production Project
 
Redefining MLOps with Model Deployment, Management and Observability in Produ...
Redefining MLOps with Model Deployment, Management and Observability in Produ...Redefining MLOps with Model Deployment, Management and Observability in Produ...
Redefining MLOps with Model Deployment, Management and Observability in Produ...
 
Productizing Machine Learning at the Edge
Productizing Machine Learning at the EdgeProductizing Machine Learning at the Edge
Productizing Machine Learning at the Edge
 
Transformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to HeroTransformers in Vision: From Zero to Hero
Transformers in Vision: From Zero to Hero
 
Deep AutoViML For Tensorflow Models and MLOps Workflows
Deep AutoViML For Tensorflow Models and MLOps WorkflowsDeep AutoViML For Tensorflow Models and MLOps Workflows
Deep AutoViML For Tensorflow Models and MLOps Workflows
 
Metaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at NetflixMetaflow: The ML Infrastructure at Netflix
Metaflow: The ML Infrastructure at Netflix
 
Practical Crowdsourcing for ML at Scale
Practical Crowdsourcing for ML at ScalePractical Crowdsourcing for ML at Scale
Practical Crowdsourcing for ML at Scale
 
Building large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudiBuilding large scale transactional data lake using apache hudi
Building large scale transactional data lake using apache hudi
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its Applications
 
Big Data and AI in Fighting Against COVID-19
Big Data and AI in Fighting Against COVID-19Big Data and AI in Fighting Against COVID-19
Big Data and AI in Fighting Against COVID-19
 
Highly-scalable Reinforcement Learning RLlib for Real-world Applications
Highly-scalable Reinforcement Learning RLlib for Real-world ApplicationsHighly-scalable Reinforcement Learning RLlib for Real-world Applications
Highly-scalable Reinforcement Learning RLlib for Real-world Applications
 
Build computer vision models to perform object detection and classification w...
Build computer vision models to perform object detection and classification w...Build computer vision models to perform object detection and classification w...
Build computer vision models to perform object detection and classification w...
 
Causal Inference in Data Science and Machine Learning
Causal Inference in Data Science and Machine LearningCausal Inference in Data Science and Machine Learning
Causal Inference in Data Science and Machine Learning
 
Weekly #106: Deep Learning on Mobile
Weekly #106: Deep Learning on MobileWeekly #106: Deep Learning on Mobile
Weekly #106: Deep Learning on Mobile
 
Weekly #105: AutoViz and Auto_ViML Visualization and Machine Learning
Weekly #105: AutoViz and Auto_ViML Visualization and Machine LearningWeekly #105: AutoViz and Auto_ViML Visualization and Machine Learning
Weekly #105: AutoViz and Auto_ViML Visualization and Machine Learning
 
AISF19 - On Blending Machine Learning with Microeconomics
AISF19 - On Blending Machine Learning with MicroeconomicsAISF19 - On Blending Machine Learning with Microeconomics
AISF19 - On Blending Machine Learning with Microeconomics
 
AISF19 - Travel in the AI-First World
AISF19 - Travel in the AI-First WorldAISF19 - Travel in the AI-First World
AISF19 - Travel in the AI-First World
 
AISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeAISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the Edge
 
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
AISF19 - Building Scalable, Kubernetes-Native ML/AI Pipelines with TFX, KubeF...
 
Toronto meetup 20190917
Toronto meetup 20190917Toronto meetup 20190917
Toronto meetup 20190917
 

Recently uploaded

When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...Elena Simperl
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Alison B. Lowndes
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoTAnalytics
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupCatarinaPereira64715
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxAbida Shariff
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCzechDreamin
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...Product School
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeCzechDreamin
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka DoktorováCzechDreamin
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomCzechDreamin
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...Product School
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Thierry Lestable
 

Recently uploaded (20)

When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 

Apache Superset at Airbnb

  • 1. MAXIME BEAUCHEMIN / MAY 2017 Apache Superset a modern, enterprise-ready business intelligence web application
  • 2. Slide Title Here Optional subtitle goes here • Cereal Entrepreneur: Creative. Embraces constraints. Solution-oriented. Tenacious. • Be a Host: Collaborative. Anticipates the needs of others. Prepared. Authentic. Listens. • Embrace the Adventure: Flexible. Risk tolerant. Always learning. Curious. Open-minded. • Simplify: Prioritizes. Distills a problem to its essence. Makes and communicates clear decisions. • Champion the Mission: Passionate. Committed. Optimistic. Puts the Airbnb community first. • Every Frame Matters: Thinks holistically. Rigorous about quality. Appreciates the details and 
 prioritizes the right ones. MaximeBeauchemin
  • 3. * explore your data! * create interactive dashboards * share discoveries A modern, enterprise-ready business intelligence web application
  • 4. The Superset Vision Any visualizationAny database microscopic amount of work
  • 5. Event Logs MySQL Dumps Gold Hive Cluster HDFS Spark Cluster Airpal Airflow Scheduling Presto Cluster Silver Hive Cluster HDFS Replication Kafka Sqoop Tableau S3 Superset ! Druid
  • 6. * Data is too strategic to depend on vendors! * Tableau doesn’t support Presto & Druid * Tableau extracts don’t scale well * Buying means lock-in and increasing costs * We need deep integration with our stack * We’re builders not buyers!
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17. The Stack * Python backend * Flask App Builder (authentication, roles, CRUD, …) * Pandas for rich analytics * SqlAlchemy (multi dialect SQL toolkit) * Javascript frontend * React / Redux * ES6 / Webpack / npm * d3.js! * nvd3.org
  • 18. Security * Provided by Flask AppBuilder (python web framework) * Easily integrate with: OpenID, LDAP, REMOTE_USER, OAUTH, or use the builtin database * Ships with 3 roles: * Admin (all access) * Alpha (all access but cannot alter permissions) * Gamma (per-datasource / table access) * Fine grain controls to create new roles
  • 19. A thin Semantic Layer * Verbose names and long descriptions for columns and metrics * Add calculated fields and metrics as SQL expression * Set how individual columns are exposed
  • 20. Caching! * Provided by flask-cache * Backends: memcache, redis, filesystem, memory, … * cascading timeout configuration * UI is upfront about staleness * allows to force-refresh
  • 22. * Grow a community! Joining the ASF * UX -> smoothen common flows * Improve SQL Lab * Ship visualizations & controls as React.js components * DSL for the semantic layer What’s next?
  • 24. Q?