Introduction to Database as a Service
using OpenStack Trove
February 26, 2015
Kamesh Pemmaraju, Mirantis
Kenneth Rugg, Tesora
Housekeeping
• Plan for 30 minutes plus Q&A
• Lines are muted
• Submit questions through chat
• Being recorded
• Slides and recording available
Welcome
Today’s Speakers
Kamesh Pemmaraju
Director of Partner Marketing
Mirantis
Ken Rugg
CEO
Tesora
An Intro to OpenStack Trove 2
Today’s Agenda
• OpenStack
• OpenStack Trove: Database as a Service
• OpenStack Trove Architecture
• Getting Started
• Database as a Service with Mirantis and Tesora
An Intro to OpenStack Trove 3
OpenStack
• Upstream OpenStack is very complicated
– Many components, configuration, choices
• New releases every 6 months
– How do I keep up?
• No enterprise support
– Who do I call if I have a problem?
• Not easy to deploy or manage
Why do you need an OpenStack Distribution?
An Intro to OpenStack Trove 5
Seven questions to ask about an OpenStack Distribution
1. Is it easy to deploy, operate, and upgrade?
2. Is it reliable, validated, and supported for enterprise-grade deployments?
3. Is it flexible enough to work with multiple operating systems, hypervisors,
databases, storage, and network technologies?
4. Is there premium 24x7 support and expertise available to support it?
5. Is the certification process open and transparent?
6. Does it work with my existing infrastructure?
7. Are there any proprietary hooks or other software or hardware bundling?
An Intro to OpenStack Trove 6
OpenStack Trove: Database as a Service
Traditional IT
• Provisioning by DBA’s
• Database management by specialists
• Waterfall development
• Few large machines/bare metal
• Oracle enterprise licenses
• Captive audience
Cloud
• Self-service provisioning
• Developers manage their own databases
• Agile development
• Many small machines/virtualization
• Many data management technologies
• Competition with AWS
Transformation of Cloud Data Management
An Intro to OpenStack Trove 8
What is OpenStack Trove?
• Database as a Service for OpenStack
• APIs for both development and operations
• Self service database provisioning
• Full database lifecycle management
• Multi-database support
• Both Relational and NoSQL
An Intro to OpenStack Trove 9
RDS for OpenStack …and DynamoDB …and RedShift
An Intro to OpenStack Trove
Amazon AWS OpenStack
EC2
SwiftS3
DynamoDB RedShift
CinderEBS
Nova
…
Trove
RDBMS
NoSQL DW
RDS
…
…and API’s for
DevOPs
OpenStack Trove Mission Statement
“To provide scalable and reliable Cloud Database as a Service
provisioning functionality for both relational and non-
relational database engines, and to continue to improve its
fully-featured and extensible open source framework.”
An Intro to OpenStack Trove 11
Tuning
• Automatically tune my.cnf
• Buffer Pool Size
• Log file size
• max_connections
• Sane defaults
• InnoDB only
• Disable load data infile
• Disable select into outfile
• API to programmatically set
configuration groups
Management
• Create database / schema
• Create users
• Grant permissions to a Schema
• Enable root user
• Resize flavor
• Resize volume
• Full and incremental backups
• Create replica
It’s More than Just Provisioning…
Security
• Security groups
• Turn off SSH
• Remove anonymous user
• Remove non-localhost users
• Remove local file access
• Mangle root user password
• Apply security patches
automatically
An Intro to OpenStack Trove 12
Trove In Production Today
• Public Cloud
– HP Cloud Relational Database
• Launched May 2012
– Rackspace Cloud Databases
• Launched August 2012
An Intro to OpenStack Trove
• Private Cloud: eBay
– Began mid 2013
– Supporting MySQL, MongoDB, Redis
Cassandra, Couchbase
– Multi-region + HA
– Working on Clustering
• Key Use Cases
– Development and test
– Web application hosting
– On-demand analytics
• Critical Capabilities
– Self-service provisioning and mgmt.
– Fleet wide configuration
– Multi-datastore architecture
13
OpenStack Trove Architecture
OpenStack Architecture
OpenStackTrove
An Intro to OpenStack Trove
What is Trove? Multi-Datastore Architecture
An Intro to OpenStack Trove
Datastore agnostic code in Trove
Controller and Dashboard
Datastore specific code isolated to Guest
Agents
Trove
Controller
Guest Agent
Guest Agent
Guest Agent
Guest Agent
Trove
Dashboard
(Horizon)
Guest Agent
Guest Agent
Guest AgentGuest Agent
Getting Started
• As a Trove user
– Download
• Mirantis OpenStack
• Tesora DBaaS platform
• As a Trove developer
$ git clone
http://github.com/openstack/trove
-integration
$ cd trove-integration/scripts
$ ./redstack install
$ ./redstack kick-start mysql
• On top of DevStack
– Add to localrc:
• ENABLED_SERVICES+=,trove,tr
-api,tr-tmgr,tr-cond
– Enable Swift for backup/restore
• ENABLED_SERVICES+=,s-
proxy,s-object,s-
container,s-account
• SWIFT_HASH=<swift-hash-
here>
Getting started with OpenStack Trove
An Intro to OpenStack Trove 18
Provisioning a Database Instance with OpenStack Trove
$ trove create <instance-name> <flavor-id>
--size <volume-size>
• Support for flavors
• Support for volumes using Cinder
• Optional parameters to create
– Image per ‘datastore type’ and ‘version’
– Support AZs using --availability_zone
– Support for Neutron using --nic
An Intro to OpenStack Trove 19
Managing a Database with OpenStack Trove
• Resize flavor
• Resize volume
• Datastore specific extensions:
– Create Database / Schema
– Create Users
– Grant Users Permissions on Databases
• Enable a Root User
An Intro to OpenStack Trove 20
$ trove backup-create
<backup-name> <instance-id>
• Optional params:
– Description through --description
– Incremental backups using –parent
• To Restore backup use create:
$ trove create
<instance-name> <flavor-id>
--size <volume-size>
--backup <backup-id>
• Fully managed
• Triggered and tracked via API
• Streamed to Swift (OpenStack Object
Storage)
• Incremental & full backups
• Multiple formats per datastore
supported via strategies:
– XtraBackup (Percona)
– mysqldump
Backup and Restore with OpenStack Trove
An Intro to OpenStack Trove 21
OpenStack Trove: Completed in Juno (Oct 2014)
• Async MySQL replication (master-slave)
• Clusters for MongoDB
• Neutron Support
• Support for PostgreSQL
• Config-groups enhancements
– Configuration groups per datastore / version
– Config-groups for MongoDB
• Backups for Cassandra and Couchbase
• Additional Tempest tests
An Intro to OpenStack Trove 22
Planned for Kilo (Apr 2015)
• Building out clusters
– Semi Synchronous MySQL clusters (Galera)
• Async Replication v2
– GTID based replication
– Manual failover support
• Associate flavors with datastores
• Access datastore logs via API
• Removing deprecated oslo-incubator code
• Upgrade testing through grenade
An Intro to OpenStack Trove 23
DBaaS with Mirantis and Tesora
Mirantis and Tesora: Pure Play OpenStack and
Enterprise Hardened Database as a Service
An Intro to OpenStack Trove
Easy to Deploy
and Use
Extensive
Testing
World Class
Support
From #1 Trove
Contributor
More Trove
Features, Earlier
25
Getting Started with Mirantis OpenStack
• Deploy a Mirantis OpenStack POC
– Download Mirantis OpenStack / Fuel ISO
• https://software.mirantis.com/
– Install the Fuel master to a VM or to bare metal
– Register for 30 days of complimentary support from within the Fuel GUI
• Documentation for planning, deployment and administration
– http://docs.mirantis.com/openstack/fuel/fuel-6.0/
• Deploy Mirantis OpenStack to VMs on a laptop using the ISO, the VirtualBox
hypervisor and the scripts
– https://software.mirantis.com/quick-start/
An Intro to OpenStack Trove 26
Thank you for joining us!
• OpenStack Trove resources:
– Trove Wiki
• https://wiki.openstack.org/wiki/Trove
– Trove Source
• https://git.openstack.org/cgit/openstack/trove
– Installable Trove packages from Tesora
• http://Tesora.com/software
– IRC
• #openstack-trove
– Trove Day
• http://Tesora.com/troveday
• http://Slideshare.net/Tesoracorp
– LinkedIn
• OpenStack Trove Group
Contact information
Tesora
Ken Rugg
@kenrugg
krugg@tesora.com
www.tesora.com
Mirantis
Kamesh Pemmaraju
@kpemmaraju
kpemmaraju@mirantis.com
www.mirantis.com
An Intro to OpenStack Trove 27

Webinar: Intro to Trove_Mirantis_26_feb2015

  • 1.
    Introduction to Databaseas a Service using OpenStack Trove February 26, 2015 Kamesh Pemmaraju, Mirantis Kenneth Rugg, Tesora
  • 2.
    Housekeeping • Plan for30 minutes plus Q&A • Lines are muted • Submit questions through chat • Being recorded • Slides and recording available Welcome Today’s Speakers Kamesh Pemmaraju Director of Partner Marketing Mirantis Ken Rugg CEO Tesora An Intro to OpenStack Trove 2
  • 3.
    Today’s Agenda • OpenStack •OpenStack Trove: Database as a Service • OpenStack Trove Architecture • Getting Started • Database as a Service with Mirantis and Tesora An Intro to OpenStack Trove 3
  • 4.
  • 5.
    • Upstream OpenStackis very complicated – Many components, configuration, choices • New releases every 6 months – How do I keep up? • No enterprise support – Who do I call if I have a problem? • Not easy to deploy or manage Why do you need an OpenStack Distribution? An Intro to OpenStack Trove 5
  • 6.
    Seven questions toask about an OpenStack Distribution 1. Is it easy to deploy, operate, and upgrade? 2. Is it reliable, validated, and supported for enterprise-grade deployments? 3. Is it flexible enough to work with multiple operating systems, hypervisors, databases, storage, and network technologies? 4. Is there premium 24x7 support and expertise available to support it? 5. Is the certification process open and transparent? 6. Does it work with my existing infrastructure? 7. Are there any proprietary hooks or other software or hardware bundling? An Intro to OpenStack Trove 6
  • 7.
  • 8.
    Traditional IT • Provisioningby DBA’s • Database management by specialists • Waterfall development • Few large machines/bare metal • Oracle enterprise licenses • Captive audience Cloud • Self-service provisioning • Developers manage their own databases • Agile development • Many small machines/virtualization • Many data management technologies • Competition with AWS Transformation of Cloud Data Management An Intro to OpenStack Trove 8
  • 9.
    What is OpenStackTrove? • Database as a Service for OpenStack • APIs for both development and operations • Self service database provisioning • Full database lifecycle management • Multi-database support • Both Relational and NoSQL An Intro to OpenStack Trove 9
  • 10.
    RDS for OpenStack…and DynamoDB …and RedShift An Intro to OpenStack Trove Amazon AWS OpenStack EC2 SwiftS3 DynamoDB RedShift CinderEBS Nova … Trove RDBMS NoSQL DW RDS … …and API’s for DevOPs
  • 11.
    OpenStack Trove MissionStatement “To provide scalable and reliable Cloud Database as a Service provisioning functionality for both relational and non- relational database engines, and to continue to improve its fully-featured and extensible open source framework.” An Intro to OpenStack Trove 11
  • 12.
    Tuning • Automatically tunemy.cnf • Buffer Pool Size • Log file size • max_connections • Sane defaults • InnoDB only • Disable load data infile • Disable select into outfile • API to programmatically set configuration groups Management • Create database / schema • Create users • Grant permissions to a Schema • Enable root user • Resize flavor • Resize volume • Full and incremental backups • Create replica It’s More than Just Provisioning… Security • Security groups • Turn off SSH • Remove anonymous user • Remove non-localhost users • Remove local file access • Mangle root user password • Apply security patches automatically An Intro to OpenStack Trove 12
  • 13.
    Trove In ProductionToday • Public Cloud – HP Cloud Relational Database • Launched May 2012 – Rackspace Cloud Databases • Launched August 2012 An Intro to OpenStack Trove • Private Cloud: eBay – Began mid 2013 – Supporting MySQL, MongoDB, Redis Cassandra, Couchbase – Multi-region + HA – Working on Clustering • Key Use Cases – Development and test – Web application hosting – On-demand analytics • Critical Capabilities – Self-service provisioning and mgmt. – Fleet wide configuration – Multi-datastore architecture 13
  • 14.
  • 15.
  • 16.
    What is Trove?Multi-Datastore Architecture An Intro to OpenStack Trove Datastore agnostic code in Trove Controller and Dashboard Datastore specific code isolated to Guest Agents Trove Controller Guest Agent Guest Agent Guest Agent Guest Agent Trove Dashboard (Horizon) Guest Agent Guest Agent Guest AgentGuest Agent
  • 17.
  • 18.
    • As aTrove user – Download • Mirantis OpenStack • Tesora DBaaS platform • As a Trove developer $ git clone http://github.com/openstack/trove -integration $ cd trove-integration/scripts $ ./redstack install $ ./redstack kick-start mysql • On top of DevStack – Add to localrc: • ENABLED_SERVICES+=,trove,tr -api,tr-tmgr,tr-cond – Enable Swift for backup/restore • ENABLED_SERVICES+=,s- proxy,s-object,s- container,s-account • SWIFT_HASH=<swift-hash- here> Getting started with OpenStack Trove An Intro to OpenStack Trove 18
  • 19.
    Provisioning a DatabaseInstance with OpenStack Trove $ trove create <instance-name> <flavor-id> --size <volume-size> • Support for flavors • Support for volumes using Cinder • Optional parameters to create – Image per ‘datastore type’ and ‘version’ – Support AZs using --availability_zone – Support for Neutron using --nic An Intro to OpenStack Trove 19
  • 20.
    Managing a Databasewith OpenStack Trove • Resize flavor • Resize volume • Datastore specific extensions: – Create Database / Schema – Create Users – Grant Users Permissions on Databases • Enable a Root User An Intro to OpenStack Trove 20
  • 21.
    $ trove backup-create <backup-name><instance-id> • Optional params: – Description through --description – Incremental backups using –parent • To Restore backup use create: $ trove create <instance-name> <flavor-id> --size <volume-size> --backup <backup-id> • Fully managed • Triggered and tracked via API • Streamed to Swift (OpenStack Object Storage) • Incremental & full backups • Multiple formats per datastore supported via strategies: – XtraBackup (Percona) – mysqldump Backup and Restore with OpenStack Trove An Intro to OpenStack Trove 21
  • 22.
    OpenStack Trove: Completedin Juno (Oct 2014) • Async MySQL replication (master-slave) • Clusters for MongoDB • Neutron Support • Support for PostgreSQL • Config-groups enhancements – Configuration groups per datastore / version – Config-groups for MongoDB • Backups for Cassandra and Couchbase • Additional Tempest tests An Intro to OpenStack Trove 22
  • 23.
    Planned for Kilo(Apr 2015) • Building out clusters – Semi Synchronous MySQL clusters (Galera) • Async Replication v2 – GTID based replication – Manual failover support • Associate flavors with datastores • Access datastore logs via API • Removing deprecated oslo-incubator code • Upgrade testing through grenade An Intro to OpenStack Trove 23
  • 24.
  • 25.
    Mirantis and Tesora:Pure Play OpenStack and Enterprise Hardened Database as a Service An Intro to OpenStack Trove Easy to Deploy and Use Extensive Testing World Class Support From #1 Trove Contributor More Trove Features, Earlier 25
  • 26.
    Getting Started withMirantis OpenStack • Deploy a Mirantis OpenStack POC – Download Mirantis OpenStack / Fuel ISO • https://software.mirantis.com/ – Install the Fuel master to a VM or to bare metal – Register for 30 days of complimentary support from within the Fuel GUI • Documentation for planning, deployment and administration – http://docs.mirantis.com/openstack/fuel/fuel-6.0/ • Deploy Mirantis OpenStack to VMs on a laptop using the ISO, the VirtualBox hypervisor and the scripts – https://software.mirantis.com/quick-start/ An Intro to OpenStack Trove 26
  • 27.
    Thank you forjoining us! • OpenStack Trove resources: – Trove Wiki • https://wiki.openstack.org/wiki/Trove – Trove Source • https://git.openstack.org/cgit/openstack/trove – Installable Trove packages from Tesora • http://Tesora.com/software – IRC • #openstack-trove – Trove Day • http://Tesora.com/troveday • http://Slideshare.net/Tesoracorp – LinkedIn • OpenStack Trove Group Contact information Tesora Ken Rugg @kenrugg krugg@tesora.com www.tesora.com Mirantis Kamesh Pemmaraju @kpemmaraju kpemmaraju@mirantis.com www.mirantis.com An Intro to OpenStack Trove 27

Editor's Notes

  • #20 { "instance": { "status": "BUILD", "updated": "2014-06-05T19:33:46", "name": "test", "links": [ { "href": "https://region-a.geo-1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb", "rel": "self" }, { "href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb", "rel": "bookmark" } ], "created": "2014-06-05T19:33:46", "id": "ce629494-a64d-41ed-b73c-04cb28bb33bb", "flavor": { "id": "1002", "links": [ { "href": "https://region-a.geo-1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002", "rel": "self" }, { "href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002", "rel": "bookmark" } ] }, "datastore": { "version": "5.5", "type": "mysql" } } }