Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
©2018 Heimdall Data, inc. www.heimdalldata.com
SQL Database Proxy
Roland Lee
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 2
Agenda
1) Database Proxy Introduction
2) Demo
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 3
Web scale Challenges
More users and applications implies
• Higher data volumes
• More tables, changes to schema
• Higher latency responses
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
Feature ProxySQL
Automated Failover
SQL Read/Write Splitting
Database Vendor Neutral
Automated Cache
invalidation
Reduces network latency
Database Proxy Vendors
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 5
Heimdall Database Proxy
• No code changes
• Database vendor neutral
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 6
Heimdall Data Software Options
Application
Heimdall Data
Proxy
Vendor Database
Driver
Application Server
Runs as an
agent
Application
Heimdall Data
JDBC
Vendor JDBC
Driver
JDBC driver, .jar
file
Application Server
Any JDBC data source
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 7
Heimdall Distributed Database Proxy
Removes network
latency
Auto-caching
Auto-invalidation
Distributed, client
SQL Optimization
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 8
Heimdall Data Use Cases
Use case Customer Benefit
SQL Results Caching
• Auto-caching / Auto-invalidate
• NO code changes
Automated failover
• Faster failover for MySQL & SQL Server AlwaysOn
• PGPool-II Replacement
Horizontal Scale-out
• Better use of read replicas
• Lag detection to guarantee fresh data
• NO code changes
Auditing for Privacy
Compliance
• Logs data access: Who, what, when
• GDPR, SOX, PCI, HIPPA
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 9
Application Server
Application
Heimdall
Data
Heimdall Distributed Deployment
Application Servers
Application Server
Application
Heimdall
Data
Application Server
Application
Heimdall
Data
Application Server
Heimdall
Central
Console
• You choose the storage
• Heimdall auto-caches
• Heimdall auto-invalidates
Automated
DB Failover
Elasticache
SQL Analytics
Audit Logging
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
L2 Cache
Heimdall
DB Proxy
Removes network
latency
Application Server
Local
Cache
Application
SQL SQL
Application Server - SQL Caching
Elasticache
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 11
How it Works
1) Heimdall proxy intercepts and forwards SQL
traffic between app & database.
2) Heimdall caches SQL results only if there is a
performance benefit.
3) Heimdall acts as a look-aside cache
4) Not a write-through or read-through cache
5) Heimdall insures data consistency.
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 12
What to cache?
Uses real-time analysis and statistics on:
– Query frequency and variability
– Relative performance of Cache vs. Database
Provides:
– Auto-cache only if there is a performance benefit
– Cache recommendations and benefits
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 13
Heimdall Query Analytics
Very cacheable. 700 s per query
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 14
Heimdall Cache Rules Analytics
After: Cached response in 70 s per query. Reduced database load.
Before: Query response time 700 s per query. Significant repetitive database load.
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
©2018 Heimdall Data, inc. www.heimdalldata.com
Use Cases
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 16
Use case #1: Horizontal Scale-out
1. Automate use of read replicas (read/write splits)
1. Replication lag detection to ensure data freshness
2. Scale out the database with no application changes!
Write
Application Server
Application Heimdall
Data
Read 1
Read 2
Read 3
Application Server
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
Data
Consumers
Heimdall Data
SQL Traffic Manager
OLTP
Analytics
Data Federation
Intelligent SQL Routing
Amazon Redshift
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
Heimdall
DB Proxy
Application Server
Application
SQL
Heimdall
Central
Console
Use case #3: End-to-End SQL Analytics
Identifies performance bottleneck
• Application?
• Database?
• Network?
Click to edit Master title style
Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com
Thank You

Heimdall Data: "Increase Application Performance with SQL Auto-Caching; No Code Changes"

  • 1.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com ©2018 Heimdall Data, inc. www.heimdalldata.com SQL Database Proxy Roland Lee
  • 2.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 2 Agenda 1) Database Proxy Introduction 2) Demo
  • 3.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 3 Web scale Challenges More users and applications implies • Higher data volumes • More tables, changes to schema • Higher latency responses
  • 4.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com Feature ProxySQL Automated Failover SQL Read/Write Splitting Database Vendor Neutral Automated Cache invalidation Reduces network latency Database Proxy Vendors ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
  • 5.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 5 Heimdall Database Proxy • No code changes • Database vendor neutral
  • 6.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 6 Heimdall Data Software Options Application Heimdall Data Proxy Vendor Database Driver Application Server Runs as an agent Application Heimdall Data JDBC Vendor JDBC Driver JDBC driver, .jar file Application Server Any JDBC data source
  • 7.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 7 Heimdall Distributed Database Proxy Removes network latency Auto-caching Auto-invalidation Distributed, client SQL Optimization
  • 8.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 8 Heimdall Data Use Cases Use case Customer Benefit SQL Results Caching • Auto-caching / Auto-invalidate • NO code changes Automated failover • Faster failover for MySQL & SQL Server AlwaysOn • PGPool-II Replacement Horizontal Scale-out • Better use of read replicas • Lag detection to guarantee fresh data • NO code changes Auditing for Privacy Compliance • Logs data access: Who, what, when • GDPR, SOX, PCI, HIPPA
  • 9.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 9 Application Server Application Heimdall Data Heimdall Distributed Deployment Application Servers Application Server Application Heimdall Data Application Server Application Heimdall Data Application Server Heimdall Central Console • You choose the storage • Heimdall auto-caches • Heimdall auto-invalidates Automated DB Failover Elasticache SQL Analytics Audit Logging
  • 10.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com L2 Cache Heimdall DB Proxy Removes network latency Application Server Local Cache Application SQL SQL Application Server - SQL Caching Elasticache
  • 11.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 11 How it Works 1) Heimdall proxy intercepts and forwards SQL traffic between app & database. 2) Heimdall caches SQL results only if there is a performance benefit. 3) Heimdall acts as a look-aside cache 4) Not a write-through or read-through cache 5) Heimdall insures data consistency.
  • 12.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 12 What to cache? Uses real-time analysis and statistics on: – Query frequency and variability – Relative performance of Cache vs. Database Provides: – Auto-cache only if there is a performance benefit – Cache recommendations and benefits
  • 13.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 13 Heimdall Query Analytics Very cacheable. 700 s per query
  • 14.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 14 Heimdall Cache Rules Analytics After: Cached response in 70 s per query. Reduced database load. Before: Query response time 700 s per query. Significant repetitive database load.
  • 15.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com ©2018 Heimdall Data, inc. www.heimdalldata.com Use Cases
  • 16.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.comCopyright © 2018 Heimdall Data, Inc. www.heimdalldata.com 16 Use case #1: Horizontal Scale-out 1. Automate use of read replicas (read/write splits) 1. Replication lag detection to ensure data freshness 2. Scale out the database with no application changes! Write Application Server Application Heimdall Data Read 1 Read 2 Read 3 Application Server
  • 17.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com Data Consumers Heimdall Data SQL Traffic Manager OLTP Analytics Data Federation Intelligent SQL Routing Amazon Redshift
  • 18.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com Heimdall DB Proxy Application Server Application SQL Heimdall Central Console Use case #3: End-to-End SQL Analytics Identifies performance bottleneck • Application? • Database? • Network?
  • 19.
    Click to editMaster title style Copyright © 2016 Heimdall Data, Inc.| www.heimdalldata.com Thank You