SlideShare a Scribd company logo
Cache, cache, cache - Visionary’s Guide to
configuring Tableau Cache
Tableau Server & Cloud Admin TUG - June 2022
Mark Wu (Tableau Visionary)
enterprisetableau.com
markwu2000@gmail.com
2
What is Cache
Cache stores recently used query
results so that it can be quickly
rendered at a later time without
querying source data again
3
Cache always happens
•access the view
•run subscription
•API ….
4
Cache happens all layers
Data Query
Cache
Render Tile
Cache
Browser (client side)
Cache
Singlecachepolicy on server
Live Connection
Shorter cache
Extract
Longer cache
DefaultCachepolicyonTableauserver
Cache and reuse as long as
possible
DefaultCachepolicyonTableauCloud
12 hrs (not customizable)
(tsm data-access caching set -r low)
If mostly Live Connections
tsmdata-accesscachingset-ralways
Alwaysget the latest data : cache
refreshes each time a page is reloaded.
TableauServerCacheConfig
2
Server:MixedLive&Extract(mostcommon)
1
12+ Hrs
tsmdata-access
cachingset-r720
Live: Workbook level data refresh
policy by owner
3
(v2021.3)
Extract: View Acceleration by owner
4
(v2022.1)
Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
tsm data-access caching set -r <value>
◦ low or empty string (""). Default. Aways use cached data
◦ <value>. "<value>" maximum minutes data should be cached
◦ always or 0 (zero). Always get the latest data
1. Server (not site)
2. Tableau Cloud 720 mins (12 hrs)
3. Mark’s server recommendation : 720-1440 mins
4. All workbooks (live or extract) except …
a. Row Level Security
b. Data fresh policy on live workbooks
#1:
10
warm up cache only for frequently used
workbooks (not all workbooks)
#2: Pre-compute Query Cache After Extract
Workbook
Performance
CPU and
memory
# of views of the workbook last 7 days
—————————————————— >= 2
# of refresh in next 7 days
#2: Pre-compute Query Cache After Extract
By default,
# of views of the workbook last 7 days
—————————————————— >= 2
# of refresh in next 7 days
#2: Pre-compute Query Cache After Extract
By default,
backgrounder.externalquerycachewarmup.view_threshold 2
Implement Pre-compute Query Cache After Extract
1.Enabled by default: backgrounder.externalquerycachewarmup.enabled true
2.Config: backgrounder.externalquerycachewarmup.view_threshold 2
(# of views of the workbook last 7 days / # of refresh in next 7 days)
3.Observe &Adjust
Deciding Cachewarmup Threshold?
https://enterprisetableau.com/cachewarmup/
Download workbook
#3: Live - Set Data Refresh Policy by owner
(v2021.3)
#4: Extract - View Acceleration by owner
(v2022.1)
View Acceleration : Force pre-compute cache after extract refresh for
the specific queries used in the view only
View Acceleration vs Pre-compute
Query Cache After Extract
What Pre-compute
Query Cache
View Acceleration
Who controls Admin Content Owner
Trigger Extract refresh Extract refresh
Scope Extract workbooks only Extract workbooks only
Exclude RLS workbooks Yes No but only for thumbnail
generation user
All frequently used workbook Yes Selected workbook
Exclude fast workbooks No Yes
Exclude refresh often
workbooks
In context of usage Yes - 6 times or more refresh/
daily
Exclude inactive owner
workbook
No Yes
If embedded credentials
expired
No warm up No warm up
If extract failed No warm up No warm up
Site level control Enable or not site limit
Admin blacklist Not possible Yes
tsm data-access caching set -r <value>
Workbooks do not follow the above server setting
1.Row Level Security - no cache (live or
extract)
2.Data freshness policy (live only)
Quiz 1:
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS, if
•7
:
00am: view accessed by an user
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 2:
if pre-compute workbooks flag set or not
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked, if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 3:
if workbook is frequently used or not
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked and
workbook is the frequently used,
if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 4:
possible browser cache, refresh
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked, if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would get the changed data?
Cache & Extract
Frequently used
Less Frequently
used
Use Case:
Critical workbook, get
latest data always?
Answers:
1.Live: Data refresh policy
2021.3
2.Extract: View Acceleration
2022.1
Quiz 5:
24
Re-cap: Tableau Tools Available for Cache
Command Notes
1 tsm data-access caching set -r <value> Server level
2 Pre-compute recently viewed workbooks Server & Site level
3 backgrounder.subscription_image_caching Server level
4 No cache for Row Level Security workbook Workbook level
5 Data acceleration (for less frequently used workbook) Workbook level
6 Force a refresh
• ?:refresh=yes
• “Refresh”button
• Reload browser
Workbook level
7 Set workbook level policy (Live Connection only) Workbook level
ServerCPUandMemoryControlwith12-24hrsCache
Workbook
Performance
CPU and
memory
vizqlserver.querylimit-v180
3 mins
1 Shorter VizQL Session Timeout 3 mins (default 30 mins)
Avoidonebadlydesignedworkbook
bringingwholeservertoitsknees
Behavior of VizQL mem
80% limit : Reclamation
95-100% limit : Restart
2 VizQL Process Memory Recycle 60% system memory (default 80%)
Problem: Restart
happens too late
Question: How to let
VizQL restart earlier enough
(60% memory) before damage
made
native_api.memory_limit_enabled-vtrue-frc
native_api.memory_limit_per_process_gb-vxx -frc
Q: What is the value for host has 300GB with 4 VizQL w/o File Store?
2 VizQL Process Memory Recycle 60% system memory
1.No measurable impact on rolling
recycle VizQL process
2.Undocumented feature but
works perfect!
3.This setting applies to Hyper
process as well
A: 50 (50GB x 4 = 200GB is 66% of 300GB)
•HoSS to consume 210G
•Recommend 70% of system memory
(for example 300GB for HoSS only)
3 Hyper Process Memory ‘hyperd’ 70% system memory
Undocumented feature
but works perfect!
ProblemStatement:OnlyONE‘Hyperd’processon
eachHoSSnodesothememory_limit_per_process_gb
capHoSSnotabletoleveragesystemmemory
Solution:
hyper.srm_memory_limit_per_process_gb
-v210-frc
ServerManagementAdd-onfeature
•hyper.session_memory_limit-v10g-frc
10 GB
4 Hyper Session Memory Timeout 5-10G (default no limit)
Avoidonebadlydesignedworkbook
impactsmanyotherusers
•hyper.session_memory_limit-v10g-frc
10 GB
4 Hyper Session Memory Timeout 5-10G (default no limit)
2
Re-cap:MixedLive&Extract(mostcommon)
1
12+ Hrs
tsmdata-access
cachingset-r720
Live: Workbook level data refresh
policy by owner
3
(v2021.3)
Extract: View Acceleration by owner
4
(v2022.1)
Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
How Cache Works on Tableau Server/Cloud.pdf

More Related Content

Similar to How Cache Works on Tableau Server/Cloud.pdf

Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
Ligaya Turmelle
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
Wim Godden
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuning
OutsourceAX
 
Lonestar php scalingmagento
Lonestar php scalingmagentoLonestar php scalingmagento
Lonestar php scalingmagento
Mathew Beane
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
Karan Kukreja
 
11g R2
11g R211g R2
11g R2
afa reg
 
Managing and Monitoring TeamPage
Managing and Monitoring TeamPageManaging and Monitoring TeamPage
Managing and Monitoring TeamPage
Traction Software
 
Cloud computing 3702
Cloud computing 3702Cloud computing 3702
Cloud computing 3702
Jess Coburn
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
Wim Godden
 
AppFabric Velocity
AppFabric VelocityAppFabric Velocity
AppFabric Velocity
Dennis van der Stelt
 
Performance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra KumarPerformance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra Kumar
Swatantra Kumar
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems
Brent Ozar
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012
Wim Godden
 
Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012
eballisty
 
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
Amazon Web Services
 
Webinar: Best Practices for Upgrading to MongoDB 3.2
Webinar: Best Practices for Upgrading to MongoDB 3.2Webinar: Best Practices for Upgrading to MongoDB 3.2
Webinar: Best Practices for Upgrading to MongoDB 3.2
Dana Elisabeth Groce
 
Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011
Wim Godden
 
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTXHA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
ThinL389917
 
Performance Scenario: Diagnosing and resolving sudden slow down on two node RAC
Performance Scenario: Diagnosing and resolving sudden slow down on two node RACPerformance Scenario: Diagnosing and resolving sudden slow down on two node RAC
Performance Scenario: Diagnosing and resolving sudden slow down on two node RAC
Kristofferson A
 
Fastback Technical Enablementv1
Fastback Technical Enablementv1Fastback Technical Enablementv1
Fastback Technical Enablementv1
petchpaitoon
 

Similar to How Cache Works on Tableau Server/Cloud.pdf (20)

Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuning
 
Lonestar php scalingmagento
Lonestar php scalingmagentoLonestar php scalingmagento
Lonestar php scalingmagento
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
 
11g R2
11g R211g R2
11g R2
 
Managing and Monitoring TeamPage
Managing and Monitoring TeamPageManaging and Monitoring TeamPage
Managing and Monitoring TeamPage
 
Cloud computing 3702
Cloud computing 3702Cloud computing 3702
Cloud computing 3702
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
AppFabric Velocity
AppFabric VelocityAppFabric Velocity
AppFabric Velocity
 
Performance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra KumarPerformance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra Kumar
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012
 
Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012
 
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
AWS APAC Webinar Week - AWS MySQL Relational Database Services Best Practices...
 
Webinar: Best Practices for Upgrading to MongoDB 3.2
Webinar: Best Practices for Upgrading to MongoDB 3.2Webinar: Best Practices for Upgrading to MongoDB 3.2
Webinar: Best Practices for Upgrading to MongoDB 3.2
 
Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011Caching and tuning fun for high scalability @ FrOSCon 2011
Caching and tuning fun for high scalability @ FrOSCon 2011
 
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTXHA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
HA and DR Architecture for HANA on Power Deck - 2022-Nov-21.PPTX
 
Performance Scenario: Diagnosing and resolving sudden slow down on two node RAC
Performance Scenario: Diagnosing and resolving sudden slow down on two node RACPerformance Scenario: Diagnosing and resolving sudden slow down on two node RAC
Performance Scenario: Diagnosing and resolving sudden slow down on two node RAC
 
Fastback Technical Enablementv1
Fastback Technical Enablementv1Fastback Technical Enablementv1
Fastback Technical Enablementv1
 

More from Mark Wu

Tableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data GovernanceTableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data Governance
Mark Wu
 
Run IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BIRun IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BI
Mark Wu
 
Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016
Mark Wu
 
ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016
Mark Wu
 
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Mark Wu
 
NetApp Tableau Presentation Final
NetApp Tableau Presentation FinalNetApp Tableau Presentation Final
NetApp Tableau Presentation Final
Mark Wu
 

More from Mark Wu (6)

Tableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data GovernanceTableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data Governance
 
Run IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BIRun IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BI
 
Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016
 
ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016
 
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
 
NetApp Tableau Presentation Final
NetApp Tableau Presentation FinalNetApp Tableau Presentation Final
NetApp Tableau Presentation Final
 

Recently uploaded

Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
GDSC PJATK
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 

Recently uploaded (20)

Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 

How Cache Works on Tableau Server/Cloud.pdf

  • 1. Cache, cache, cache - Visionary’s Guide to configuring Tableau Cache Tableau Server & Cloud Admin TUG - June 2022 Mark Wu (Tableau Visionary) enterprisetableau.com markwu2000@gmail.com
  • 2. 2 What is Cache Cache stores recently used query results so that it can be quickly rendered at a later time without querying source data again
  • 3. 3 Cache always happens •access the view •run subscription •API ….
  • 4. 4 Cache happens all layers Data Query Cache Render Tile Cache Browser (client side) Cache
  • 5. Singlecachepolicy on server Live Connection Shorter cache Extract Longer cache
  • 6. DefaultCachepolicyonTableauserver Cache and reuse as long as possible DefaultCachepolicyonTableauCloud 12 hrs (not customizable) (tsm data-access caching set -r low)
  • 7. If mostly Live Connections tsmdata-accesscachingset-ralways Alwaysget the latest data : cache refreshes each time a page is reloaded. TableauServerCacheConfig
  • 8. 2 Server:MixedLive&Extract(mostcommon) 1 12+ Hrs tsmdata-access cachingset-r720 Live: Workbook level data refresh policy by owner 3 (v2021.3) Extract: View Acceleration by owner 4 (v2022.1) Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
  • 9. tsm data-access caching set -r <value> ◦ low or empty string (""). Default. Aways use cached data ◦ <value>. "<value>" maximum minutes data should be cached ◦ always or 0 (zero). Always get the latest data 1. Server (not site) 2. Tableau Cloud 720 mins (12 hrs) 3. Mark’s server recommendation : 720-1440 mins 4. All workbooks (live or extract) except … a. Row Level Security b. Data fresh policy on live workbooks #1:
  • 10. 10 warm up cache only for frequently used workbooks (not all workbooks) #2: Pre-compute Query Cache After Extract
  • 11. Workbook Performance CPU and memory # of views of the workbook last 7 days —————————————————— >= 2 # of refresh in next 7 days #2: Pre-compute Query Cache After Extract By default,
  • 12. # of views of the workbook last 7 days —————————————————— >= 2 # of refresh in next 7 days #2: Pre-compute Query Cache After Extract By default, backgrounder.externalquerycachewarmup.view_threshold 2
  • 13. Implement Pre-compute Query Cache After Extract 1.Enabled by default: backgrounder.externalquerycachewarmup.enabled true 2.Config: backgrounder.externalquerycachewarmup.view_threshold 2 (# of views of the workbook last 7 days / # of refresh in next 7 days) 3.Observe &Adjust
  • 15. #3: Live - Set Data Refresh Policy by owner (v2021.3)
  • 16. #4: Extract - View Acceleration by owner (v2022.1) View Acceleration : Force pre-compute cache after extract refresh for the specific queries used in the view only
  • 17. View Acceleration vs Pre-compute Query Cache After Extract What Pre-compute Query Cache View Acceleration Who controls Admin Content Owner Trigger Extract refresh Extract refresh Scope Extract workbooks only Extract workbooks only Exclude RLS workbooks Yes No but only for thumbnail generation user All frequently used workbook Yes Selected workbook Exclude fast workbooks No Yes Exclude refresh often workbooks In context of usage Yes - 6 times or more refresh/ daily Exclude inactive owner workbook No Yes If embedded credentials expired No warm up No warm up If extract failed No warm up No warm up Site level control Enable or not site limit Admin blacklist Not possible Yes
  • 18.
  • 19. tsm data-access caching set -r <value> Workbooks do not follow the above server setting 1.Row Level Security - no cache (live or extract) 2.Data freshness policy (live only) Quiz 1:
  • 20. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS, if •7 : 00am: view accessed by an user •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 2: if pre-compute workbooks flag set or not
  • 21. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 3: if workbook is frequently used or not
  • 22. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked and workbook is the frequently used, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 4: possible browser cache, refresh
  • 23. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would get the changed data? Cache & Extract Frequently used Less Frequently used Use Case: Critical workbook, get latest data always? Answers: 1.Live: Data refresh policy 2021.3 2.Extract: View Acceleration 2022.1 Quiz 5:
  • 24. 24 Re-cap: Tableau Tools Available for Cache Command Notes 1 tsm data-access caching set -r <value> Server level 2 Pre-compute recently viewed workbooks Server & Site level 3 backgrounder.subscription_image_caching Server level 4 No cache for Row Level Security workbook Workbook level 5 Data acceleration (for less frequently used workbook) Workbook level 6 Force a refresh • ?:refresh=yes • “Refresh”button • Reload browser Workbook level 7 Set workbook level policy (Live Connection only) Workbook level
  • 26. vizqlserver.querylimit-v180 3 mins 1 Shorter VizQL Session Timeout 3 mins (default 30 mins) Avoidonebadlydesignedworkbook bringingwholeservertoitsknees
  • 27. Behavior of VizQL mem 80% limit : Reclamation 95-100% limit : Restart 2 VizQL Process Memory Recycle 60% system memory (default 80%) Problem: Restart happens too late Question: How to let VizQL restart earlier enough (60% memory) before damage made
  • 28. native_api.memory_limit_enabled-vtrue-frc native_api.memory_limit_per_process_gb-vxx -frc Q: What is the value for host has 300GB with 4 VizQL w/o File Store? 2 VizQL Process Memory Recycle 60% system memory 1.No measurable impact on rolling recycle VizQL process 2.Undocumented feature but works perfect! 3.This setting applies to Hyper process as well A: 50 (50GB x 4 = 200GB is 66% of 300GB)
  • 29. •HoSS to consume 210G •Recommend 70% of system memory (for example 300GB for HoSS only) 3 Hyper Process Memory ‘hyperd’ 70% system memory Undocumented feature but works perfect! ProblemStatement:OnlyONE‘Hyperd’processon eachHoSSnodesothememory_limit_per_process_gb capHoSSnotabletoleveragesystemmemory Solution: hyper.srm_memory_limit_per_process_gb -v210-frc ServerManagementAdd-onfeature
  • 30. •hyper.session_memory_limit-v10g-frc 10 GB 4 Hyper Session Memory Timeout 5-10G (default no limit) Avoidonebadlydesignedworkbook impactsmanyotherusers
  • 31. •hyper.session_memory_limit-v10g-frc 10 GB 4 Hyper Session Memory Timeout 5-10G (default no limit)
  • 32. 2 Re-cap:MixedLive&Extract(mostcommon) 1 12+ Hrs tsmdata-access cachingset-r720 Live: Workbook level data refresh policy by owner 3 (v2021.3) Extract: View Acceleration by owner 4 (v2022.1) Recommend: Longer Cache Policy (12-24 hrs) with cache warmup