• Like

Marketing Automation at Scale: How Marketo Solved Key Data Management Challenges with Continuent Tungsten

  • 377 views
Uploaded on

Marketo provides the leading cloud-based marketing software platform for companies of all sizes to build and sustain engaging customer relationships. Marketo's SaaS platform runs on MySQL and has …

Marketo provides the leading cloud-based marketing software platform for companies of all sizes to build and sustain engaging customer relationships. Marketo's SaaS platform runs on MySQL and has faced data management challenges common to all 24x7 SaaS businesses:

- Keeping data available regardless of DBMS failures or planned maintenance
- Utilizing hardware optimized for multi-terabyte MySQL servers
- Keeping replicas caught up and ready for instant failover despite high transaction loads

In this webinar, Nick Bonfiglio, VP of Operations at Marketo, describes how Marketo manages thousands of customers and processes a billion marketing analytics transactions a day using Continuent Tungsten and MySQL atop an innovative hardware architecture. He explains how Tungsten parallel replication paved the way to rapid growth by solving Marketo's biggest MySQL challenge: keeping DBMS replicas up to date despite massive transaction loads.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
377
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
14
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ©Continuent 2013 Marketing Automation at Scale: How Marketo Solved Key Data Management Challenges with Continuent Tungsten Nick Bonfiglio – VP Operations, Marketo Robert Hodges – CEO, Continuent
  • 2. About Us • Nick Bonfiglio - Marketo VP of Operations / Responsible for Cloud Services and Security at Marketo / Extensive background in scaling SaaS products and services • Robert Hodges - Continuent CEO / Tungsten Replicator designer / DBMS internals and applications for three decades / Author of Scale-Out Blog (http://scale-out- blog.blogspot.com)
  • 3. Marketo is the global leader in Revenue Performance Management. Marketo’s powerful, yet easy-to-use marketing automation and sales effectiveness solutions, transform how marketing and sales teams of all sizes work — and work together — to drive dramatically increased revenue performance and fuel business growth. The company’s proven technology, comprehensive services, and expert guidance are helping corporations around the world to turn marketing from a cost center to a business-building revenue driver. About Marketo
  • 4. / The leading provider of data replication and clustering for open source databases / Our Product: Continuent Tungsten • Clustering – Commercial-grade HA, performance scaling, and data management for MySQL • Replication – Flexible, high-performance data movement About Continuent
  • 5. 4 Customers from hot startups to Fortune 500 companies
  • 6. ©Continuent 2013 Marketo’s Data Management Challenge Nick Bonfiglio
  • 7. Marketo Architecture • Marketo uses a mixed stack architecture to support its customers’ subscriptions, leveraging the best technologies for the solution. • Marketo is a platform consisting of a collection of systems that provide user interface, landing pages/forms, fast secondary search (solr), big data (hadoop) and archiving (cassandra). • Our systems employ several custom queue- based background processing, which are key to a successful marketing automation platform. And, custom email assembly, scripting and tracking facilities.
  • 8. Systems Architecture Marketo SaaS Platform File Services Database Services Batch Processes User Interface Munchkin (tracking) APILanding Pages/Forms Email Services
  • 9. Infrastructure Architecture
  • 10. Marketo DB Design Goals • Deliver increased availability by providing redundancy and real-time resiliency at all layers. • Deliver an architecture that allows us to upgrade software and routine database schema changes without incurring downtime to customers. • Improve overall performance, scalability and management, including backups and disaster recovery of large 7TB+ DB servers.
  • 11. Marketo DB Architecture DB on FlexVol DB on FlexVol DB on FlexVol Replicator ReplicatorReplicator MasterSlave Slave Marketo Web Cluster Marketo Queue Cluster MySQL Client LibraryMySQL Client Library Manager ManagerManager Pod Data DBMS Vendor Client Library DBMS Vendor Client LibraryTungsten Connector Tungsten Connector Monitoringandcontrol Monitoringandcontrol
  • 12. Marketo DB Hardware Architecture • Leveraging NetApp FlexVols for snapshot, near-line and DR replication • Symmetric DB servers so failover yields full performance • Aggregates are 300GB drives for increased spindles • Each DB Server ~6-8T • Leverage flexcache for certain volumes
  • 13. Performance Metrics • Over 600 Million MySQL Transactions per day • Over 600TB of data • Processing 25 Million Campaigns per day • Sending over 500M Emails per month • Serving 15M Landing Page Visits per day • A few proof points: – Several failovers which would have caused an outage – Parallel replication allows is to keep lag to <300sec – Several roll rotations for maintenance (schema changes, dropping large dbs, db pruning, etc.)
  • 14. ©Continuent 2013 So… How Does Tungsten Do That? Robert Hodges
  • 15. Challenges to Hosting Critical Data on MySQL / Get back online quickly after DBMS failures / Perform DBMS maintenance and SQL upgrades without stopping applications / Load balance SQL across replicas to use hardware efficiently / Ensure slaves keep up with high master transaction loads / Address all of this without migrating or rewriting applications
  • 16. Tungsten Database-as-a-Service for MySQL ReplicatorReplicator • Cluster constructed from off-the-shelf MySQL servers • Automatic failover • Online maintenance • Query Load balancing for speed and h/w utilization • No migration of data or application changes Manager db2 Manager Replicator db1 Manager db3 Slave SlaveMaster Application Stack Tungsten Connector Application Stack Tungsten Connector ReplicatorReplicator
  • 17. Transparent connectivity enables illusion of single DBMS to applications • Locate appropriate server for transactions • Select most up-to-date server for reads • Shift connections from offline nodes • Hide offline databases • Supports SSL Send writes to master Load balance reads over slaves Application Stack Application StackApplication Stack Tungsten Connector db2 db1 db3
  • 18. ©Continuent 2013 Demo: Introducing a Tungsten Data Service
  • 19. What Causes Slave Lag? Application Application Application Master InnoDB Buffer Pool NetApp Storage Slave InnoDB Buffer Pool NetApp Storage Typical SaaS Application: 1.) Data size >> buffer pool 2.) Persistent stored on disk
  • 20. SaaS Applications Can Replicate Tenants in Parallel! • Sharded replication divides updates into independent streams (“shards”) tenant_1 tenant_2 tenant_3 tenant_4 Tenants independent from each other Tenant database updates applied in parallel tenant_1 tenant_2 tenant_3 tenant_4
  • 21. High-Speed Parallel Replication Lets Slaves Keep Pace with Busy Masters • Shard transactions by schema on master • Transfer log serially • Apply shard updates over “channels” • 5x or more speed boost in best cases • Works on all MySQL versions from 5.0 to 5.6 Replicator db1 Master db2 Slave Replicator Application Stack Application StackApplication Stack
  • 22. Problem: Uneven SaaS Tenant Size
  • 23. Problem: “Hot” Tables within Single Schemas that Block Other Tenants
  • 24. Tungsten’s Innovative Caching Minimizes Effects of Lumpy SaaS Workload Slave InnoDB Buffer Pool NetApp Storage Tungsten Replication Service NetApp Storage Fetch from Master Write to Log Apply to Slave Linux Page Cache Read log from page cache (No reads from storage) Write log to page cache Fsync to storage Shards apply Independently!
  • 25. Innovative Features of Tungsten Parallel Replication Support • Tungsten leans on OS page cache to keep long updates from blocking progress • Use any number of apply channels • Support for clean failover and backup • Completely crash-safe for InnoDB • Configuration checks to prevent accidental slave corruption • Sharding algorithm is configurable!
  • 26. ©Continuent 2013 Demo: Maintenance on MySQL master under load
  • 27. What Else Can Tungsten Do? • Cross-site data services – Primary/disaster recovery sites – Multi-master replication between sites • Fast data warehouse/NoSQL loading from MySQL/Oracle • Oracle-to-Oracle replication
  • 28. What’s Next? Push-Button Cloud Operation Configure DBMS type and resource levels for servers Define backup and recovery policies Confirm configuration and launch Choose Service Type Manage Deployment 1 2 3 4
  • 29. Making it Better – Visual Management
  • 30. Making it Better – Visual Management
  • 31. Failure detected and failover completed within 15 seconds Making it Better – Visual Management
  • 32. Failed db instance is isolated from the service Making it Better – Visual Management
  • 33. ©Continuent 2013 Conclusion and Q&A
  • 34. In Summary… 33 • Marketo performs 600M transactions daily on MySQL running over NetApp • Tungsten delivers fast failover, rolling maintenance, and parallel replication to make architecture work • Marketo and Continuent together have enabled an architecture that supports economical scaling to 1000s of customers
  • 35. Contact Us! Continuent, Inc. 560 S. Winchester Blvd Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com www.continuent.com Marketo, Inc. 901 Mariners Island Blvd Suite 200 San Mateo, CA 94404 Tel +1 (877) 260-MKTO (6586) Fax +1 (650) 376-2331 e-mail: sales@marketo.com www.marketo.com