Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Modernizing Enterprise Analytics
The IT Story
Tableau overview
Tell me and I forget;
Show me and I may remember;
Involve me and I’ll understand.
Chinese Proverb
The people who know the data should be
empowered to ask questions of the data
old school
old school
People are smart and computers are tools to
augment their intelligence and creativity
break free
Flow, flexibility, and freedom
are the keys to creative thinking
Driving change
not just discovering insights
We can’t solve problems by using
the same kind of thinking we used
when we cr...
Culture of Analytics
Change isn’t coming. It’s here.
Business users are demanding self service…wherever they are.
Their data is everywhere and ...
Self-service @ scale
Data Visual Analytics Cloud
Mobile Fast, easy, beautiful
Transformation is happening now….
People Process
Technology
01000100
01000001
DATA
01000001
01010100
We need to re-imagine...
1. Governance
2. Security
3. Scalability
4. Availability
5. Monitoring
6. Management
Self Service at Scale
The Trial…
You download the server trial, start installer, hits “Next” a bunch of times
You have a Tableau Server!! Now wh...
A Day In the Life of IT
From Getting Started to Enterprise
Network, Storage
Infrastructure Systems
Application / Services
Monitoring,Management,
Governance,Scalability,Availability,...
Tableau ServerData Clients
Command Line
Tools
Browser/Mobile
Tableau Desktop
SQL
User Tier
Storage Tier
Management
Tier
Tableau ServerData Clients
Base Install Responsible for monitoring various components, detecting
failures, and executing f...
Tableau ServerData Clients
Gateway
Base Install
Receives incoming client requests and directs them to the
appropriate serv...
Tableau ServerData Clients
Gateway
App Server
Base Install
Includes two processes – one that renders the web portal
(vizpo...
Tableau ServerData Clients
Gateway
Base Install
Repository Search & Browse App Server
Command Line
Tools
Browser/Mobile
Ta...
Tableau ServerData Clients
Gateway
Base Install
App ServerRepository Search & Browse
Active Directory/SAML
Command Line
To...
Tableau ServerData Clients
Gateway
Base Install
DataSourceDrivers
App ServerRepository Search & Browse
Active Directory/SA...
Tableau ServerData Clients
Gateway
Base Install
DataSourceDrivers
VizQL Server
Cache Server
App Server
Loads and renders v...
Tableau ServerData Clients
Gateway
Base Install
DataSourceDrivers
VizQL Server
Cache Server
Data EngineFile Store
App Serv...
Tableau ServerData Clients
Gateway
Base Install
DataSourceDrivers
VizQL Server
Cache Server
Data EngineFile Store
Backgrou...
Tableau ServerData
DataSourceDrivers
Clients
Gateway
VizQL Server
Data EngineFile Store
Data Server
Base Install
Cache Ser...
Tableau ServerData
DataSourceDrivers
Clients
Gateway
VizQL Server
Data EngineFile Store
Data Server
Base Install
Cache Ser...
Active Repository
HTTP(S)Server
Gateway, etc.
Cluster Controller
Coordination
VizPortal
File Store
Passive Repository
HTTP...
User Authentication
SAML
Kerberos
Row Level Security - Kerberos
A
B
Deployment Architectures
• Single Machine, Default Installation
• Use Sample Workbooks Included
• Published your home grown workbook
Trial Deployme...
Simple and Small - Production Deployment
• Single Machine Deployment
– 1x8 Core
– 8GB Per Core RAM
– 5MBPS IOPS or More
• ...
Medium Deployment
• Multi-Machine Deployment
– 2x8 Core Machines
• Trade Offs:
– Small increase in complexity for
companie...
Primary Node
Base
Install
Worker Node 1 Worker Node 2
Gateway
Search
VizQL Server
Cache Server
Data Server
*
Data Engine
F...
An Enterprise Deployment Architecture
Database
Untrusted Zone
(Internet)
Public
DMZ
App
Zone
Intranet ZoneDB Zone
Maps
Rev...
Tableau Server Scalability
Scalability
Scales outScales up
Tableau architecture is designed for scale
Data Refresh Frequency for Effective Business Decisions
AnalyticsUseforEffectiveBusinessDecisions
Data Refresh Frequency for Effective Business Decisions
AnalyticsUseforEffectiveBusinessDecisions
Low
(once a day)
1.
Exam...
Data Refresh Frequency for Effective Business Decisions
AnalyticsUseforEffectiveBusinessDecisions
Moderate
(once an hour)
...
Data Refresh Frequency for Effective Business Decisions
AnalyticsUseforEffectiveBusinessDecisions
High
(every second)
9.
E...
AnalyticsUseforEffectiveBusinessDecisions
High
(every second)
7.
Examples:
WW Data Exploration
Tableau Public (US Presiden...
AnalyticsUseforEffectiveBusinessDecisions
High
(every second)
7.
Examples:
WW Data Exploration
Tableau Public (US Presiden...
High
(every second)
7.
Examples:
WW Data Exploration
Tableau Public (US Presidential
Election) 30KViews/hour
8.
Examples:
...
PERFORMANCE
Improvements across the product
Query
Improvements
Data Engine
Improvements
Server
Improvements
Parallel Query Vectorizati...
Performance Comparison
A test dashboard
with a100 million
rows of flight data
took ~25 secs in 8.3
The same dashboard,
tak...
Connection pool architecture
Connection pool
Connection group
Connection
Connection
Connection group
Connection
Connection...
High Availability
9
35 days
9
4 days
9
8 hours
9
50 mins
9
5 mins
%
CoordinationCoordination
Active Repository
HTTP(S)Server
Gateway, etc.
Cluster Controller
VizPortal
File Store
Coordinatio...
Coordination
Active Repository
HTTP(S)Server
Gateway, etc.
VizPortal
File Store
Passive RepositoryActive Repository
HTTP(S...
• JavaScript API: Integrate visualizations in web applications
– Drive Mark Selections, Apply / Remove Filters
– Two Way E...
Enterprise Heterogeneous Connectivity
Over 40 specialized connectors out of the
box and ODBC
Out of the box support for Bi...
Server Architecture Deep Dive
Gateway
VizQL Server
Data Server
(Extracts)
Postgres
Data Engine
Extracts
Customer Data
Source
Published data source
(live...
Request Flow – Admin Management
Gateway
Application Server
(JAVA)
Search Service
SOLR
Postgres
JSON -RPC
Gateway
API Services (aka
WGServer)
SOLR Postgres
Request Flow - REST API
Gateway
Data Server
(Extracts)
Postgres
Request Flow - Published Data Server
Data Engine
Extracts
Customer Data
Source
Pub...
Backgrounder
Postgres
Data Engine
Same as Web
Visualization
Request Flow
Refresh Extract
Request Flow – Backgrounder
Tableau ServerData
DataSourceDrivers
Clients
Gateway
VizQL Server
Data EngineFile Store
Data Server
Base Install
Cache Ser...
An Enterprise Deployment Architecture
Database
Untrusted Zone
(Internet)
Public
DMZ
App
Zone
Intranet ZoneDB Zone
Maps
Rev...
Does not mean simplistic
Tableau architecture drives enterprises
Scaling Analytic Culture
with Tableau Drive
Is this how you feel (now)?
So why do it?
Tableau is different.
It can help you create better workplaces by
building analytic culture.
Does the “report factory” model work for anyone?
Requiremen
ts Gathering
Developme
nt
Planning
User
Acceptanc
e
Test
Produ...
Is your effort appreciated?
But can’t the process be “tweaked” using Agile?
Should the business users
move in with development?
Planning
Developme
nt
...
What happens when business users do the development?
Self-service collapses phases
of the agile process, allowing
real-tim...
Self service model: IT = enabler
IT
Business
Users
?
?
?
??
?
?
?
?
??
?
?
Report factory model: IT = bottleneck
Tableau = Disrupter
But is this a
BIG deal or a small deal?
Our customers have been telling us for
years that it’s a big deal, a really big de...
“Try it, you’ll like it”
Fun aside… what value is this bringing to my
users and organization?
We work in a knowledge economy
Intangibles (Human Capital contribution) as % of S&P 500 market cap.
1975 2015
17% 84%
Knowledge work cannot be forced
Analytic
Culture
Thinking
Knowledge
Participation
Engagement
Analytic Culture
• A shared, baseline understanding of the business: who, what, when,
where, why, how.Knowledge
• Empower ...
Curiosity: Wanting to know
Anxiety: Needing to know
Anxiety: Needing to know
Anxiety: Needing to know
Report Factory
- By technical specialists who often don’t
have business context knowledge
- Using specialized skills and complex
tools
- ...
- Business-aligned subject matter experts
with analytic skills
- Run the “Center of Evangelism”
- Participate in promotion...
Metcalfe’s Law
value ∝ n2
Sharing information makes organizations smarter,
exponentially.
Knowledge allows sense making
“Core”
Contextual
Knowledge
New
Information
Filtering
Validation
Synthesis
Capuchin monkey fairness experiment
https://www.youtube.com/watch?v=-KSryJXDpZo
Fairness and workplace morale
“Without data, opinion
prevails. Where
opinion prevails,
whoever has power is
king.”
Scientists seek the truth through data
Google decides with data
Systems Thinking
Simplistic isn’t sufficient
By Nicolaus Copernicus
The Earth revolves around
the sun.
(Applause)
“All you need
to know
Is ...
Execution is more than understanding
Analytic Culture
• A shared, baseline understanding of the business: who, what, when,
where, why, how.Knowledge
• Empower ...
Feeling left out?
Democracy (vs. Monarchy)
Toyota Andon System
Toyota Way
Analytic Culture
• A shared, baseline understanding of the business: who, what, when,
where, why, how.Knowledge
• Empower ...
Not Carrots, Not Sticks
Not about the Perks
Maslow
Eupsychian Management
Thinkers about thinking
Abraham Maslow
Mihaly Csikszentmihalyi
Peter Drucker
Martin Seligman
(and many more)
• Autonomy
• Mastery
• Purpose
• Community
The Four Amigos of Engagement
Engaged Not Engaged
• Autonomous
• Challenged/Growing
• Communal
• Purposeful
• Blocked
• Stuck
• Isolated
• Meaningless
Engagement Pays
Most organizations aren’t doing so well…
In developed countries, enagement hovers
around 20% on average.
Employee Value Proposition
#1 Demotivator: Road Blocks
“People are most satisfied
with their jobs (and therefore
most motivated) when those
jobs give...
#1 Demotivator: Road-blocks
“[W]e discovered the progress
principle: Of all the things that can
boost emotions, motivation...
Flow matters
Our humanity is expressed in our choices
Our humanity is expressed in our choices
Which workplace reflects our humanity?
Zappos
Traditional BI is disengaging. It is inhumane.
Analytic Culture
• A shared, baseline understanding of the business: who, what, when,
where, why, how.Knowledge
• Empower ...
Where will the next great idea come from?
Critical thinking Evaluate
• Judge
• Compare
• Contrast
• Critique
• Choose
• Rate
• Select
Synthesize
• Compose
• Origina...
Brain exercise
Foundational skill-set, “A Liberal Art”
Cicero
Socrates
David S. Moore
“Rich setting for problem solving and group
work.”
Applied, experiential, active learning
Thinking
Knowledge
Participation
Engagement
“Organic Growth”
Analytics 4 Fun != Analytics @ Scale
Analytics for Fun Analytics at Scale
Individual effort Community effort
Self-starter,...
Deliberate, programmatic support
Why deliberate, programmatic support?
Why deliberate, programmatic support?
Why deliberate, programmatic support?
Drive is a roadmap to scale your analytic culture
http://www.tableau.com/drive
Drive’s Big Ideas
• Business owns the creative and analytical work.
• IT is empowered to do what they do best, better.
• G...
A partnership that works
IT Role
• Operations
• Infrastructure
• Systems
• Security
• Data
• Production environment
Busine...
Drive best practices
Getting Started, Properly
– Own the “getting started” experience
and do it right.
Skills Pyramid
– Develop champions throughout the
organization and enable users.
Analysis Not Replication
– Follow a repeatable process to translate business
questions into data projects.
Balance Control with Agility
– There is a difference between managed data
discovery and traditional BI lockdown.
Teamwork or Bust
– Bridge the gap between business and IT.
Make this strategic
– Build-out the Centers of Evangelism and
Operations.
Measure and Monitor
– Create a feedback loop to quantify business impact.
• Discovery
• Prototyping
• Best Practices development
• Custom training
• Helpdesk
• Scale-out
• Assessment
• Events
Serv...
Tableau’s Mission
Help people see and
understand their data.
Ask a Tableau champion
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
IT Summit - Modernizing Enterprise Analytics: the IT Story
Upcoming SlideShare
Loading in …5
×

IT Summit - Modernizing Enterprise Analytics: the IT Story

2,554 views

Published on

IT Summit - Modernizing Enterprise Analytics: the IT Story

  1. 1. Modernizing Enterprise Analytics The IT Story
  2. 2. Tableau overview
  3. 3. Tell me and I forget; Show me and I may remember; Involve me and I’ll understand. Chinese Proverb
  4. 4. The people who know the data should be empowered to ask questions of the data
  5. 5. old school old school People are smart and computers are tools to augment their intelligence and creativity
  6. 6. break free Flow, flexibility, and freedom are the keys to creative thinking
  7. 7. Driving change not just discovering insights We can’t solve problems by using the same kind of thinking we used when we created them. Albert Einstein
  8. 8. Culture of Analytics
  9. 9. Change isn’t coming. It’s here. Business users are demanding self service…wherever they are. Their data is everywhere and they have questions. Databases Big Data Spreadsheets Application Data Cloud
  10. 10. Self-service @ scale Data Visual Analytics Cloud Mobile Fast, easy, beautiful
  11. 11. Transformation is happening now…. People Process Technology 01000100 01000001 DATA 01000001 01010100 We need to re-imagine our IT processes and how we support our business
  12. 12. 1. Governance 2. Security 3. Scalability 4. Availability 5. Monitoring 6. Management Self Service at Scale
  13. 13. The Trial… You download the server trial, start installer, hits “Next” a bunch of times You have a Tableau Server!! Now what??
  14. 14. A Day In the Life of IT From Getting Started to Enterprise
  15. 15. Network, Storage Infrastructure Systems Application / Services Monitoring,Management, Governance,Scalability,Availability, Security Service Desk (ITIL) APIs/Extensibility/ Integration In IT We have too much on our plate. Infrastructure teams are driving toward private clouds, embracing converged infrastructure and have little time to understand every application they have to deploy, monitor and manage. Every application needs integration to the enterprise technology fabric that takes time and effort. And all of this needs to be monitored and managed end to end.
  16. 16. Tableau ServerData Clients Command Line Tools Browser/Mobile Tableau Desktop SQL User Tier Storage Tier Management Tier
  17. 17. Tableau ServerData Clients Base Install Responsible for monitoring various components, detecting failures, and executing failover when needed. In distributed installations, responsible for ensuring there is a quorum for making decisions during failover. Manages the licensing of Tableau Server through periodic compliance checks. Command Line Tools Browser/Mobile Tableau Desktop SQL
  18. 18. Tableau ServerData Clients Gateway Base Install Receives incoming client requests and directs them to the appropriate service for action. Acts as a load balancer, routing traffic across multiple service instances. Command Line Tools Browser/Mobile Tableau Desktop SQL
  19. 19. Tableau ServerData Clients Gateway App Server Base Install Includes two processes – one that renders the web portal (vizportal) and one that handles REST APIs (wgserver). Processes logins, content searches, content and permission management, uploads/downloads and other tasks not related to visualizing data. Repository Command Line Tools Browser/Mobile Tableau Desktop SQL Stores Tableau Server metadata: users, group assignments, permissions, projects, etc. Also stores flat files (TWB, TDS). Responds to queries from other services when they need metadata. Holds audit data for performance reporting. Has a SQL interface so external applications can connect (read-only).
  20. 20. Tableau ServerData Clients Gateway Base Install Repository Search & Browse App Server Command Line Tools Browser/Mobile Tableau Desktop SQL Handles fast search, filter, retrieval , and display of content metadata on the server.
  21. 21. Tableau ServerData Clients Gateway Base Install App ServerRepository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL If used, verifies authentication in conjunction with the App Server and Repository.
  22. 22. Tableau ServerData Clients Gateway Base Install DataSourceDrivers App ServerRepository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL Drivers need to be installed for each data source (32-bit or 64-bit, depending on installed version of Tableau Server). Downloads and more details at http://www.tableau.com/support/drivers
  23. 23. Tableau ServerData Clients Gateway Base Install DataSourceDrivers VizQL Server Cache Server App Server Loads and renders views, computes and executes queries. Repository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL The query cache used to be local to each service but now it is distributed and shared across the server cluster. The cache speeds user experience across many scenarios. VizQL Server, Backgrounder, and Data Server make requests to the Cache Server before hitting the data source.
  24. 24. Tableau ServerData Clients Gateway Base Install DataSourceDrivers VizQL Server Cache Server Data EngineFile Store App Server Stores and services queries to data extracts (TDE). Invoked when a data extract is published or viewed. Repository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL Installed with the Data Engine. Automatically replicates extracts across data engine nodes.
  25. 25. Tableau ServerData Clients Gateway Base Install DataSourceDrivers VizQL Server Cache Server Data EngineFile Store Backgrounder App ServerRepository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL Runs maintenance tasks to ensure Tableau Server is running efficiently. When the Data Engine is used, also handles scheduled data refreshes. Handles tasks initiated via TABCMD.
  26. 26. Tableau ServerData DataSourceDrivers Clients Gateway VizQL Server Data EngineFile Store Data Server Base Install Cache ServerBackgrounder App Server Invoked when a data source is published via Tableau Desktop. Serves as proxy for queries to the actual data source (file, DB server or extract host). Enables centralized metadata management for data sources and an additional layer of access control. Allows multiple workbooks to use the same data extract. Allows centralized driver deployment. Repository Search & Browse Active Directory/SAML Command Line Tools Browser/Mobile Tableau Desktop SQL
  27. 27. Tableau ServerData DataSourceDrivers Clients Gateway VizQL Server Data EngineFile Store Data Server Base Install Cache ServerBackgrounder Active Directory/SAML App ServerRepository Search & Browse Command Line Tools Browser/Mobile Tableau Desktop SQL
  28. 28. Active Repository HTTP(S)Server Gateway, etc. Cluster Controller Coordination VizPortal File Store Passive Repository HTTP(S)Server Worker1 Search & Browse Worker2 HTTP(S)Server Cluster Controller Coordination File Store Worker3 Data Engine Gateway, etc. Cluster Controller Coordination VizQL Server File Store Search & Browse Data Engine Backgrounder HTTP(S)Server Primary Cluster Controller Coordination Gateway Search & Browse Licensing Loading a viz Backgrounder
  29. 29. User Authentication SAML Kerberos
  30. 30. Row Level Security - Kerberos A B
  31. 31. Deployment Architectures
  32. 32. • Single Machine, Default Installation • Use Sample Workbooks Included • Published your home grown workbook Trial Deployment / Prototyping Load testing is not recommended with trial deployments (tuned for trial)
  33. 33. Simple and Small - Production Deployment • Single Machine Deployment – 1x8 Core – 8GB Per Core RAM – 5MBPS IOPS or More • Trade Offs: – Easy to manage and administer one node. – Good for small teams with little to no IT support – Hardware and Software are single point of failure, higher risk of down time – Likely hood of shared resource (RAM,DISK etc.) contention increases with increased usage over time Primary Node Gateway Search VizQL Server Cache Server Data Server * Data Engine File Store BaseInstall Backgrounder Repository Application Server * ** ** ** ** ** ** * * 1x8 Core Machine Higher Risk Deployment Gateway, Repository, Application Server, Data Engine become single point of failures on single machine systems Backgrounder is CPU and Disk intensive by design. Can starve other server processes with increased workload Adding additional server processes will come at the cost of user scale and performance.
  34. 34. Medium Deployment • Multi-Machine Deployment – 2x8 Core Machines • Trade Offs: – Small increase in complexity for companies/teams with no IT support – Improved availability with 2 machines, at process level – Repository still single point of failure – Scalable to a certain degree, under peak loads likelihood of shared resource (RAM,DISK etc.) contention increases Primary Node Gateway Search VizQL Server Cache Server Data Server * Data Engine File Store BaseInstall Backgrounder Repository Application Server * * ** **** ** * * * * Worker Node BaseInstall Gateway VizQL Server Cache Server Data Server * Application Server * ** **** ** Added gateway*, reduces risk Added worker alleviates RAM, Disk contentions Repository remains single point of failure Backgrounder can compete with resources with VizQL, Data Engine and Repository 1x8 Core Machine 1x8 Core Machine Lower Risk Deployment, Increased Availability *Assumes ELB
  35. 35. Primary Node Base Install Worker Node 1 Worker Node 2 Gateway Search VizQL Server Cache Server Data Server * Data Engine File Store BaseInstall Application Server * * ** ** ** ** * Repository (active) * Gateway VizQL Server Cache Server Data Server * Data Engine File Store BaseInstall Application Server * **** **** ** ** * Search * Gateway Backgrounder (N to 2N) **** Extract Heavy Production Deployment 501-1000 Users 1x8 Core Physical or VM 64GB + 4GB = 68 GB RAM 1x8 Core 1x8 Core 2 Additional backgrounders for higher extract 1 Additional Worker 2 Additional VizQL for user load 2 Additional Cache Servers 2 Additional Data Engines 1x8 Core
  36. 36. An Enterprise Deployment Architecture Database Untrusted Zone (Internet) Public DMZ App Zone Intranet ZoneDB Zone Maps Reverse Proxy Shadow Sync Policy ServerClient SSO Firewall
  37. 37. Tableau Server Scalability
  38. 38. Scalability Scales outScales up Tableau architecture is designed for scale
  39. 39. Data Refresh Frequency for Effective Business Decisions AnalyticsUseforEffectiveBusinessDecisions
  40. 40. Data Refresh Frequency for Effective Business Decisions AnalyticsUseforEffectiveBusinessDecisions Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI Low (once a day)
  41. 41. Data Refresh Frequency for Effective Business Decisions AnalyticsUseforEffectiveBusinessDecisions Moderate (once an hour) 5. Examples Patient Capacity Dealer Management Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI Low (once a day) Moderate (once an hour)
  42. 42. Data Refresh Frequency for Effective Business Decisions AnalyticsUseforEffectiveBusinessDecisions High (every second) 9. Examples: Air Traffic Controller Finance Trade Execution Moderate (once an hour) 5. Examples Patient Capacity Dealer Management Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI Low (once a day) Moderate (once an hour) Always (Live)
  43. 43. AnalyticsUseforEffectiveBusinessDecisions High (every second) 7. Examples: WW Data Exploration Tableau Public (US Presidential Election) 30KViews/hour 8. Examples: Sales Quota Dashboard, Tableau on TV 9. Examples: Air Traffic Controller Monitoring Finance Trade Execution Moderate (once an hour) 4. Examples Daily Store Inventory Insurance Customer Analysis Marketing (targeting) 5. Examples Patient Capacity Dealer Management 6. Examples: Support Escalation Dashboard Finance Portfolio Dashboard Fraud Investigation Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI 2. Examples: Who’s Hot Sales Lead Tracking 3. Examples: Highway Web Traffic Dashboards Low (once a day) Moderate (once an hour) Always (Live) Data Refresh Frequency for Effective Business Decisions
  44. 44. AnalyticsUseforEffectiveBusinessDecisions High (every second) 7. Examples: WW Data Exploration Tableau Public (US Presidential Election) 30KViews/hour 8. Examples: Sales Quota Dashboard, Tableau on TV 9. Examples: Air Traffic Controller Monitoring Finance Trade Execution Moderate (once an hour) 4. Examples Daily Store Inventory Insurance Customer Analysis Marketing (targeting) 5. Examples Patient Capacity Dealer Management 6. Examples: Support Escalation Dashboard Finance Portfolio Dashboard Fraud Investigation Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI 2. Examples: Who’s Hot Sales Lead Tracking 3. Examples: Highway Web Traffic Dashboards Low (once a day) Moderate (once an hour) Always (Live) Data Refresh Frequency for Effective Business Decisions High to Moderate External Query Cache Use Low to Moderate Query Cache Use
  45. 45. High (every second) 7. Examples: WW Data Exploration Tableau Public (US Presidential Election) 30KViews/hour 8. Examples: Sales Quota Dashboard, Tableau on TV 9. Examples: Air Traffic Controller Monitoring Finance Trade Execution Moderate (once an hour) 4. Examples Daily Store Inventory Insurance Customer Analysis Marketing (targeting) 5. Examples Patient Capacity Dealer Management 6. Examples: Support Escalation Dashboard Finance Portfolio Dashboard Fraud Investigation Low (once a day) 1. Examples: Engineering - Ship Room Mortgage Inventory Traditional BI 2. Examples: Who’s Hot Sales Lead Tracking 3. Examples: Highway Web Traffic Dashboards Low (once a day) Moderate (once an hour) Always (Live) AnalyticsUseforEffectiveBusinessDecisions Data Refresh Frequency for Effective Business Decisions Add Backgrounders VizQL,DataServer(PublishedDS), DataEngine,CacheServers
  46. 46. PERFORMANCE
  47. 47. Improvements across the product Query Improvements Data Engine Improvements Server Improvements Parallel Query Vectorization All Query Improvements Query Fusion Parallel Plans Rendering Performance
  48. 48. Performance Comparison A test dashboard with a100 million rows of flight data took ~25 secs in 8.3 The same dashboard, takes ~12 secs in 9.0
  49. 49. Connection pool architecture Connection pool Connection group Connection Connection Connection group Connection Connection DBSession Session DBSession Session Connection pool Connection Connection DB Session DB Session
  50. 50. High Availability 9 35 days 9 4 days 9 8 hours 9 50 mins 9 5 mins %
  51. 51. CoordinationCoordination Active Repository HTTP(S)Server Gateway, etc. Cluster Controller VizPortal File Store CoordinationCoordination CoordinationCoordination Passive Repository HTTP(S)Server Worker1 Search & Browse Worker2 Data Engine Gateway, etc. Cluster Controller VizQL Server File Store Search & Browse Data Engine HTTP(S)Server Primary Cluster Controller Gateway Search & Browse Licensing Coordination Triggered by: Repository process dies Or… tabadmin failoverrepository [--target <host name or IPv4>|--preferred] Couple quick points… Cluster Controller has a leader Combining Coordination into ensemble to simplify demo Repository Failover
  52. 52. Coordination Active Repository HTTP(S)Server Gateway, etc. VizPortal File Store Passive RepositoryActive Repository HTTP(S)Server Worker1 Search & Browse Worker2 Data Engine Gateway, etc. Cluster Controller VizQL Server File Store Search & Browse Data Engine HTTP(S)Server Primary Gateway Search & Browse Licensing ! ! Cluster Controller Coordination Coordination Coordination Cluster Controller Passive Repository Almost done. Processes take a few minutes to bounce and update their configuration... …Vizportal …API Server …Vizql Server …Data Server …Backgrounder …API Search Index Down Repository recovers as Passive. Repository Failover You Tube Live Failover Demo
  53. 53. • JavaScript API: Integrate visualizations in web applications – Drive Mark Selections, Apply / Remove Filters – Two Way Events – Build your own custom tool bar • Extract API : Load any data into Tableau – Language support flexibility (Java/C/C++/Python) – Build data extracts on any machine • REST API : Extend server interaction in any language – Automate user onboarding – Move projects, workbooks across dev/test/production environments – Update permissions and more Extensibility with Tableau SDK
  54. 54. Enterprise Heterogeneous Connectivity Over 40 specialized connectors out of the box and ODBC Out of the box support for Big Data sources, Relational Databases, SAP HANA certified WebData Connector allows any web data to be brought into Tableau Data API via Tableau SDK allows you to bring any data you need into Tableau
  55. 55. Server Architecture Deep Dive
  56. 56. Gateway VizQL Server Data Server (Extracts) Postgres Data Engine Extracts Customer Data Source Published data source (live) Live Connection Permissions/MetaData/twb/twbx Request Flow – Web Visualization
  57. 57. Request Flow – Admin Management Gateway Application Server (JAVA) Search Service SOLR Postgres JSON -RPC
  58. 58. Gateway API Services (aka WGServer) SOLR Postgres Request Flow - REST API
  59. 59. Gateway Data Server (Extracts) Postgres Request Flow - Published Data Server Data Engine Extracts Customer Data Source Published data source (live) Live Connection Permissions/Metadata/tds/tdsx
  60. 60. Backgrounder Postgres Data Engine Same as Web Visualization Request Flow Refresh Extract Request Flow – Backgrounder
  61. 61. Tableau ServerData DataSourceDrivers Clients Gateway VizQL Server Data EngineFile Store Data Server Base Install Cache ServerBackgrounder Active Directory/SAML App ServerRepository Search & Browse Command Line Tools Browser/Mobile Tableau Desktop SQL
  62. 62. An Enterprise Deployment Architecture Database Untrusted Zone (Internet) Public DMZ App Zone Intranet ZoneDB Zone Maps Reverse Proxy Shadow Sync Policy ServerClient SSO Firewall
  63. 63. Does not mean simplistic Tableau architecture drives enterprises
  64. 64. Scaling Analytic Culture with Tableau Drive
  65. 65. Is this how you feel (now)?
  66. 66. So why do it?
  67. 67. Tableau is different. It can help you create better workplaces by building analytic culture.
  68. 68. Does the “report factory” model work for anyone? Requiremen ts Gathering Developme nt Planning User Acceptanc e Test Production … is an IT mission... Subject Matter Expert (ideas) Every idea….
  69. 69. Is your effort appreciated?
  70. 70. But can’t the process be “tweaked” using Agile? Should the business users move in with development? Planning Developme nt Production User Acceptanc e Test Subject Matter Expertise (ideas)
  71. 71. What happens when business users do the development? Self-service collapses phases of the agile process, allowing real-time iteration. Production Development Planning User Acceptance Test Subject Matter Expertise (requirements) Planning Developme nt Production User Acceptanc e Test Subject Matter Expertise (ideas) Self-service: a more agile Agile.
  72. 72. Self service model: IT = enabler IT Business Users ? ? ? ?? ? ? ? ? ?? ? ? Report factory model: IT = bottleneck
  73. 73. Tableau = Disrupter
  74. 74. But is this a BIG deal or a small deal? Our customers have been telling us for years that it’s a big deal, a really big deal (ie. you should care)
  75. 75. “Try it, you’ll like it”
  76. 76. Fun aside… what value is this bringing to my users and organization?
  77. 77. We work in a knowledge economy Intangibles (Human Capital contribution) as % of S&P 500 market cap. 1975 2015 17% 84%
  78. 78. Knowledge work cannot be forced
  79. 79. Analytic Culture Thinking Knowledge Participation Engagement
  80. 80. Analytic Culture • A shared, baseline understanding of the business: who, what, when, where, why, how.Knowledge • Empower those who know the business best to analyze data and share findings broadly with others. • Use data to build consensus, align initiatives, and win support. Participation • Leverage self-reliant analytics to strengthen commitment and job satisfaction by removing roadblocks, supporting learning, building community, and strengthening mission alignment. Engagement • Exercise, promote, and celebrate critical and creative thinking through analysis.Thinking
  81. 81. Curiosity: Wanting to know
  82. 82. Anxiety: Needing to know
  83. 83. Anxiety: Needing to know
  84. 84. Anxiety: Needing to know
  85. 85. Report Factory
  86. 86. - By technical specialists who often don’t have business context knowledge - Using specialized skills and complex tools - With exclusive access to enterprise data - As “Sole” source for reports “Report factory”
  87. 87. - Business-aligned subject matter experts with analytic skills - Run the “Center of Evangelism” - Participate in promotion to production workflow - Are hghly encouraged to become proficient (jedi-caliber) - Train, mentor, and work in real-time with others - Are sometimes paid to do analysis full time - Goal: - Everyone an analyst Tableau “Analyst” Community of Tableau Users Analyst Learner Consumer
  88. 88. Metcalfe’s Law value ∝ n2 Sharing information makes organizations smarter, exponentially.
  89. 89. Knowledge allows sense making “Core” Contextual Knowledge New Information Filtering Validation Synthesis
  90. 90. Capuchin monkey fairness experiment https://www.youtube.com/watch?v=-KSryJXDpZo
  91. 91. Fairness and workplace morale “Without data, opinion prevails. Where opinion prevails, whoever has power is king.”
  92. 92. Scientists seek the truth through data
  93. 93. Google decides with data
  94. 94. Systems Thinking
  95. 95. Simplistic isn’t sufficient By Nicolaus Copernicus The Earth revolves around the sun. (Applause) “All you need to know Is in this envelope!”
  96. 96. Execution is more than understanding
  97. 97. Analytic Culture • A shared, baseline understanding of the business: who, what, when, where, why, how.Knowledge • Empower those who know the business best to analyze data and share findings broadly with others. • Use data to build consensus, align initiatives, and win support. Participation • Leverage self-reliant analytics to strengthen commitment and job satisfaction by removing roadblocks, supporting learning, building community, and strengthening mission alignment. Engagement • Exercise, promote, and celebrate critical and creative thinking through analysis.Thinking
  98. 98. Feeling left out?
  99. 99. Democracy (vs. Monarchy)
  100. 100. Toyota Andon System
  101. 101. Toyota Way
  102. 102. Analytic Culture • A shared, baseline understanding of the business: who, what, when, where, why, how.Knowledge • Empower those who know the business best to analyze data and share findings broadly with others. • Use data to build consensus, align initiatives, and win support. Participation • Leverage self-reliant analytics to strengthen commitment and job satisfaction by removing roadblocks, supporting learning, building community, and strengthening mission alignment. Engagement • Exercise, promote, and celebrate critical and creative thinking through analysis.Thinking
  103. 103. Not Carrots, Not Sticks
  104. 104. Not about the Perks
  105. 105. Maslow
  106. 106. Eupsychian Management
  107. 107. Thinkers about thinking Abraham Maslow Mihaly Csikszentmihalyi Peter Drucker Martin Seligman (and many more)
  108. 108. • Autonomy • Mastery • Purpose • Community The Four Amigos of Engagement
  109. 109. Engaged Not Engaged • Autonomous • Challenged/Growing • Communal • Purposeful • Blocked • Stuck • Isolated • Meaningless
  110. 110. Engagement Pays
  111. 111. Most organizations aren’t doing so well… In developed countries, enagement hovers around 20% on average.
  112. 112. Employee Value Proposition
  113. 113. #1 Demotivator: Road Blocks “People are most satisfied with their jobs (and therefore most motivated) when those jobs give them the opportunity to experience achievement.”
  114. 114. #1 Demotivator: Road-blocks “[W]e discovered the progress principle: Of all the things that can boost emotions, motivation, and perceptions during a workday, the single most important is making progress in meaningful work.”
  115. 115. Flow matters
  116. 116. Our humanity is expressed in our choices
  117. 117. Our humanity is expressed in our choices
  118. 118. Which workplace reflects our humanity?
  119. 119. Zappos
  120. 120. Traditional BI is disengaging. It is inhumane.
  121. 121. Analytic Culture • A shared, baseline understanding of the business: who, what, when, where, why, how.Knowledge • Empower those who know the business best to analyze data and share findings broadly with others. • Use data to build consensus, align initiatives, and win support. Participation • Leverage self-reliant analytics to strengthen commitment and job satisfaction by removing roadblocks, supporting learning, building community, and strengthening mission alignment. Engagement • Exercise, promote, and celebrate critical and creative thinking through analysis.Thinking
  122. 122. Where will the next great idea come from?
  123. 123. Critical thinking Evaluate • Judge • Compare • Contrast • Critique • Choose • Rate • Select Synthesize • Compose • Originate • Design • Construct • Plan • Create • Invent • Organize • Combine • Predict • Revise Analyze • Compare • Classify • Point out • Distinguish • Infer • Select • Dissect • Specify • Distinguish • Categorize
  124. 124. Brain exercise
  125. 125. Foundational skill-set, “A Liberal Art” Cicero Socrates David S. Moore “Rich setting for problem solving and group work.”
  126. 126. Applied, experiential, active learning
  127. 127. Thinking Knowledge Participation Engagement
  128. 128. “Organic Growth”
  129. 129. Analytics 4 Fun != Analytics @ Scale Analytics for Fun Analytics at Scale Individual effort Community effort Self-starter, self-guided Shared resources/division of labor Private/rogue data Sanctioned, enterprise data Dashboard “oohs” and “ahs” Systematic skill building “Fend for yourself” Programmatic support & encouragement Narrow base of adoption Broad-based adoption
  130. 130. Deliberate, programmatic support
  131. 131. Why deliberate, programmatic support?
  132. 132. Why deliberate, programmatic support?
  133. 133. Why deliberate, programmatic support?
  134. 134. Drive is a roadmap to scale your analytic culture http://www.tableau.com/drive
  135. 135. Drive’s Big Ideas • Business owns the creative and analytical work. • IT is empowered to do what they do best, better. • Great visualizations are the beginning, not the end, of adoption. • Drive provides a concrete plan that expands the vision and reduces risk in deploying enterprise-wide analytics whether implemented in-house, with Tableau consulting, or partner consulting.
  136. 136. A partnership that works IT Role • Operations • Infrastructure • Systems • Security • Data • Production environment Business Role • Creative work • Data requirements • Community • Helpdesk • Evangelism • Sandbox environment ExecutionEnablement MORE responsibility NEW responsibilities
  137. 137. Drive best practices
  138. 138. Getting Started, Properly – Own the “getting started” experience and do it right.
  139. 139. Skills Pyramid – Develop champions throughout the organization and enable users.
  140. 140. Analysis Not Replication – Follow a repeatable process to translate business questions into data projects.
  141. 141. Balance Control with Agility – There is a difference between managed data discovery and traditional BI lockdown.
  142. 142. Teamwork or Bust – Bridge the gap between business and IT.
  143. 143. Make this strategic – Build-out the Centers of Evangelism and Operations.
  144. 144. Measure and Monitor – Create a feedback loop to quantify business impact.
  145. 145. • Discovery • Prototyping • Best Practices development • Custom training • Helpdesk • Scale-out • Assessment • Events Service Offerings
  146. 146. Tableau’s Mission Help people see and understand their data.
  147. 147. Ask a Tableau champion

×