SlideShare a Scribd company logo
1 of 35
Download to read offline
BUILDING COMPLETE PRIVATE
CLOUDS WITH APACHE
CLOUDSTACK + RIAK CS
John Burwell
(jburwell@basho.com | @john_burwell)

Wednesday, October 23, 13
Who Am I
•

Consulting Engineer @ Basho Technologies

•

Member, Apache CloudStack PMC

•

Ran operations and designed automated provisioning for hybrid
analytic/virtualization clouds

•

Led architectural design and server-side development of a SaaS
physical security platform

Wednesday, October 23, 13
... but I submitted a ticket.

Wednesday, October 23, 13
Shifting Role Of IT

From Cost Center
To Business Opportunity

Wednesday, October 23, 13
Wednesday, October 23, 13
Agility

Exploitation Requires Rapid Execution
Infrastructure management must be delegated

Wednesday, October 23, 13
Agility

Exploitation Requires Rapid Execution
Infrastructure management must be delegated

Infrastructure Must Become Boring

Wednesday, October 23, 13
Composable Infrastructure

•

Resource Pooling

•

Rapid Elasticity

•

Self Service

•

Metering/Charge Back

Wednesday, October 23, 13
Got It ... Why Go Private?

•

Leverage existing infrastructure investment

•

Customization and Control

•

Regulatory/Security Requirements

•

Cost

Wednesday, October 23, 13
Doesn’t vSphere/
XCP/XenServer
already do that?

Wednesday, October 23, 13
Traditional Workload
•
•

Fault tolerant

•

Management

Reliable

Specialized hardware

•

IT managed provisioning

•

Static scale out model

Wednesday, October 23, 13

Network

Hypervisor
Cluster

Hypervisor
Cluster

Shared
 Storage
Traditional Workload
•
•

Fault tolerant

•

Management

Reliable

Specialized hardware

•

IT managed provisioning

•

Static scale out model

Network

Hypervisor
Cluster

Hypervisor
Cluster

Shared
 Storage

Focused on resource consolidation not agility

Wednesday, October 23, 13
Software Is Eating Hardware
•

Moore’s Law has pulled fault tolerance up the stack

•

Commodity Hardware
•
•

•

Reduced cost
Services more workload types

Dynamic resource allocation

Wednesday, October 23, 13
Software Is Eating Hardware
•

Moore’s Law has pulled fault tolerance up the stack

•

Commodity Hardware
•
•

•

Reduced cost
Services more workload types

Dynamic resource allocation

Specialized Hardware Reduces Agility

Wednesday, October 23, 13
Cloud Workload
•

Reliable

•

Fault Tolerant

•

Self-Service provisioning

•

Control Plane - Dynamic
scale out model

Rack

Rack

App

App

Commodity Hardware

•

SDN

Wednesday, October 23, 13

Object
 Storage
Cloud Workload
•

Reliable

•

Fault Tolerant

•

Self-Service provisioning

•

Control Plane - Dynamic
scale out model

Rack

Rack

App

App

Commodity Hardware

•

SDN

Object
 Storage

Resource optimization increases agility

Wednesday, October 23, 13
Reality

Traditional
Workload

Wednesday, October 23, 13

Cloud
Workload
Complete Private Cloud

+

Compute Services

Wednesday, October 23, 13

Object Storage
Services
What Is Apache CloudStack?

§Open source software that deploys
and manages large networks of
virtual machines, as a highly
available, highly scalable
Infrastructure as a Service (IaaS)
cloud computing platform.
§CloudStack is a Top-Level Project at
the Apache Software Foundation.

Wednesday, October 23, 13
What Is Apache CloudStack?

§Open source software that deploys
and manages large networks of
virtual machines, as a highly
available, highly scalable
Infrastructure as a Service (IaaS)
cloud computing platform.
§CloudStack is a Top-Level Project at
the Apache Software Foundation.

Wednesday, October 23, 13
Apache CloudStack Users

Wednesday, October 23, 13
Anatomy Of CloudStack
•

Management Server Farm
•

•

VM

Network
Host

Collection of clusters in the same failure boundary

Primary
Storage

A grouping of hosts and their associated storage

Hosts
•

•

Collection of pods, network offerings and secondary storage

Cluster
•

•

Host

Pod
•

•

VM

Zone
•

•

Management and provisioning tasks

Cluster

Secondary
Storage

Cluster

Servers onto which services will be provisioned

Primary Storage
CloudStack Pod

•
•

VM disk storage

Network
•

CloudStack Pod

Logical network associated with service offerings
Zone

•

Secondary Storage
•

Template, snapshot and ISO storage

Wednesday, October 23, 13
Primary Vs Secondary Storage
•

Random access read/write

•

Write once/read many

•

Changes often

•

Append only

•

Zone Scoped

•

Zone or Region Scoped

•

Focused on throughput

•

Focused on reliability

Wednesday, October 23, 13
Primary Vs Secondary Storage
•

Random access read/write

•

Write once/read many

•

Changes often

•

Append only

•

Zone Scoped

•

Zone or Region Scoped

•

Focused on throughput

•

Focused on reliability

Secondary Storage is the lifeboat.

Wednesday, October 23, 13
What Is Riak CS?
•

Simple, reliable storage of large objects accessed using the S3 or Swift
API

•

Objects are stored in buckets with a unique key

•

Multi-tenant security model with per object ACLs

•

Usage Accounting and Reporting

•

Open Source (Apache2 Licensed)

Wednesday, October 23, 13
What Is Riak CS?
•

Simple, reliable storage of large objects accessed using the S3 or Swift
API

•

Objects are stored in buckets with a unique key

•

Multi-tenant security model with per object ACLs

•

Usage Accounting and Reporting

•

Open Source (Apache2 Licensed)

Your Own Private S3

Wednesday, October 23, 13
Anatomy Of Riak CS
•

S3/Swift API Proxy + Riak key-value store

•

Peer-to-Peer Design: Any node can service any read or accept
any write request

•

Maintains multiple replicas (default 3) spread across multiple
nodes

•

Ops Friendly

Wednesday, October 23, 13
Anatomy Of Riak CS
•

S3/Swift API Proxy + Riak key-value store

•

Peer-to-Peer Design: Any node can service any read or accept
any write request

•

Maintains multiple replicas (default 3) spread across multiple
nodes

•

Ops Friendly

Horizontally scaleable storage on
commodity hardware

Wednesday, October 23, 13
Riak CS Use Cases

•

Secondary Storage for VM snapshot and template data

•

Database Backup Storage

•

Serving Static Content

•

Application Log Storage

Wednesday, October 23, 13
Achieving Commodity Reliability

•

Create a zone per power source/backbone network connection

•

Establish independent primary storage in each zone

•

Utilize intra-datacenter block-level replication between zones

•

Distribute Riak CS nodes evenly across zones

Wednesday, October 23, 13
The Complete Private Cloud
Traditional

More Related Content

What's hot

Building a Bigdata Architecture on AWS
Building a Bigdata Architecture on AWSBuilding a Bigdata Architecture on AWS
Building a Bigdata Architecture on AWSArun Sirimalla
 
Protecting Your Big Data on the Cloud
Protecting Your Big Data on the CloudProtecting Your Big Data on the Cloud
Protecting Your Big Data on the CloudAlibaba Cloud
 
AWS Data migration services
AWS Data migration servicesAWS Data migration services
AWS Data migration servicesArun Sirimalla
 
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Amazon Web Services
 
Leveraging ApsaraDB to Deploy Business Data on the Cloud
Leveraging ApsaraDB to Deploy Business Data on the CloudLeveraging ApsaraDB to Deploy Business Data on the Cloud
Leveraging ApsaraDB to Deploy Business Data on the CloudOliver Theobald
 
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013RightScale
 
Riak at Engine Yard Cloud
Riak at Engine Yard CloudRiak at Engine Yard Cloud
Riak at Engine Yard CloudInes Sombra
 
Building Bulletproof Infrastructure on AWS
Building Bulletproof Infrastructure on AWSBuilding Bulletproof Infrastructure on AWS
Building Bulletproof Infrastructure on AWS2nd Watch
 
Responding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyResponding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyAlibaba Cloud
 
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]Rainforest QA
 
Getting Started with Elasticsearch
Getting Started with ElasticsearchGetting Started with Elasticsearch
Getting Started with ElasticsearchAlibaba Cloud
 
Backup on the cloud Webinar
Backup on the cloud WebinarBackup on the cloud Webinar
Backup on the cloud Webinar2nd Watch
 
Getting Started with EC2, S3 and EMR
Getting Started with EC2, S3 and EMRGetting Started with EC2, S3 and EMR
Getting Started with EC2, S3 and EMRArun Sirimalla
 
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...Amazon Web Services
 
Beyond Relational
Beyond RelationalBeyond Relational
Beyond RelationalLynn Langit
 
Launch and Scale Your E-commerce Website with Magento
Launch and Scale Your E-commerce Website with MagentoLaunch and Scale Your E-commerce Website with Magento
Launch and Scale Your E-commerce Website with MagentoAlibaba Cloud
 
AliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAlibaba Cloud
 

What's hot (20)

Building a Bigdata Architecture on AWS
Building a Bigdata Architecture on AWSBuilding a Bigdata Architecture on AWS
Building a Bigdata Architecture on AWS
 
Protecting Your Big Data on the Cloud
Protecting Your Big Data on the CloudProtecting Your Big Data on the Cloud
Protecting Your Big Data on the Cloud
 
AWS Data migration services
AWS Data migration servicesAWS Data migration services
AWS Data migration services
 
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...
Integrating Infrastructure as Code into a Continuous Delivery Pipeline | AWS ...
 
Leveraging ApsaraDB to Deploy Business Data on the Cloud
Leveraging ApsaraDB to Deploy Business Data on the CloudLeveraging ApsaraDB to Deploy Business Data on the Cloud
Leveraging ApsaraDB to Deploy Business Data on the Cloud
 
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013
Key Design Considerations Private and Hybrid Clouds - RightScale Compute 2013
 
IBM Cloud Object Storage
IBM Cloud Object StorageIBM Cloud Object Storage
IBM Cloud Object Storage
 
Riak at Engine Yard Cloud
Riak at Engine Yard CloudRiak at Engine Yard Cloud
Riak at Engine Yard Cloud
 
Building Bulletproof Infrastructure on AWS
Building Bulletproof Infrastructure on AWSBuilding Bulletproof Infrastructure on AWS
Building Bulletproof Infrastructure on AWS
 
Responding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database TechnologyResponding to Digital Transformation With RDS Database Technology
Responding to Digital Transformation With RDS Database Technology
 
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
How does Riak compare to Cassandra? [Cassandra London User Group July 2011]
 
Getting Started with Elasticsearch
Getting Started with ElasticsearchGetting Started with Elasticsearch
Getting Started with Elasticsearch
 
Backup on the cloud Webinar
Backup on the cloud WebinarBackup on the cloud Webinar
Backup on the cloud Webinar
 
Getting Started with EC2, S3 and EMR
Getting Started with EC2, S3 and EMRGetting Started with EC2, S3 and EMR
Getting Started with EC2, S3 and EMR
 
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...
(ENT201) New Generation Hybrid Architectures with Suncorp, NetApp, and AWS | ...
 
Beyond Relational
Beyond RelationalBeyond Relational
Beyond Relational
 
Launch and Scale Your E-commerce Website with Magento
Launch and Scale Your E-commerce Website with MagentoLaunch and Scale Your E-commerce Website with Magento
Launch and Scale Your E-commerce Website with Magento
 
AliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core Features
 
In Flux Limiting for a multi-tenant logging service
In Flux Limiting for a multi-tenant logging serviceIn Flux Limiting for a multi-tenant logging service
In Flux Limiting for a multi-tenant logging service
 
Zabbix at scale with Elasticsearch
Zabbix at scale with ElasticsearchZabbix at scale with Elasticsearch
Zabbix at scale with Elasticsearch
 

Similar to Building Complete Private Clouds with Apache CloudStack and Riak CS

Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CSBetter, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CSJohn Burwell
 
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CSMaking Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CSJohn Burwell
 
Database Modernization
Database ModernizationDatabase Modernization
Database ModernizationTrivadis
 
AWS Webcast - Library Systems on the AWS Cloud
AWS Webcast - Library Systems on the AWS CloudAWS Webcast - Library Systems on the AWS Cloud
AWS Webcast - Library Systems on the AWS CloudAmazon Web Services
 
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands On
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands OnCIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands On
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands OnCohesive Networks
 
Being Well Architected in the Cloud (Updated)
Being Well Architected in the Cloud (Updated)Being Well Architected in the Cloud (Updated)
Being Well Architected in the Cloud (Updated)Adrian Hornsby
 
Being Well-Architected in the Cloud
Being Well-Architected in the CloudBeing Well-Architected in the Cloud
Being Well-Architected in the CloudAmazon Web Services
 
AWS Architecture Fundamentals - Houston
AWS Architecture Fundamentals - HoustonAWS Architecture Fundamentals - Houston
AWS Architecture Fundamentals - HoustonNicole Maus
 
Cloud computing Azure
Cloud computing AzureCloud computing Azure
Cloud computing Azurevivek p s
 
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...Amazon Web Services
 
Azure Fundamentals Part 2
Azure Fundamentals Part 2Azure Fundamentals Part 2
Azure Fundamentals Part 2CCG
 
Scaling the Platform for Your Startup
Scaling the Platform for Your StartupScaling the Platform for Your Startup
Scaling the Platform for Your StartupAmazon Web Services
 
AWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the CloudAWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the CloudAmazon Web Services
 
Apache CloudStack 4.2: A First Look
Apache CloudStack 4.2: A First LookApache CloudStack 4.2: A First Look
Apache CloudStack 4.2: A First LookShanker Balan
 
What's New with Big Data Analytics
What's New with Big Data AnalyticsWhat's New with Big Data Analytics
What's New with Big Data AnalyticsAmazon Web Services
 

Similar to Building Complete Private Clouds with Apache CloudStack and Riak CS (20)

Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CSBetter, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CS
 
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CSMaking Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
Making Cloudy Peanut Butter Cups: Apache CloudStack + Riak CS
 
Database Modernization
Database ModernizationDatabase Modernization
Database Modernization
 
AWS Webcast - Library Systems on the AWS Cloud
AWS Webcast - Library Systems on the AWS CloudAWS Webcast - Library Systems on the AWS Cloud
AWS Webcast - Library Systems on the AWS Cloud
 
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands On
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands OnCIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands On
CIW Lab with CoheisveFT: Get started in public cloud - Part 2 Hands On
 
Being Well Architected in the Cloud (Updated)
Being Well Architected in the Cloud (Updated)Being Well Architected in the Cloud (Updated)
Being Well Architected in the Cloud (Updated)
 
Being Well-Architected in the Cloud
Being Well-Architected in the CloudBeing Well-Architected in the Cloud
Being Well-Architected in the Cloud
 
AWS Architecture Fundamentals - Houston
AWS Architecture Fundamentals - HoustonAWS Architecture Fundamentals - Houston
AWS Architecture Fundamentals - Houston
 
Cloud computing Azure
Cloud computing AzureCloud computing Azure
Cloud computing Azure
 
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...
 
Azure Fundamentals Part 2
Azure Fundamentals Part 2Azure Fundamentals Part 2
Azure Fundamentals Part 2
 
Intro to CloudStack
Intro to CloudStackIntro to CloudStack
Intro to CloudStack
 
Openstack: starter level
Openstack: starter levelOpenstack: starter level
Openstack: starter level
 
AWS Re Invent 2019 Recap
AWS Re Invent 2019 Recap AWS Re Invent 2019 Recap
AWS Re Invent 2019 Recap
 
Scaling the Platform for Your Startup
Scaling the Platform for Your StartupScaling the Platform for Your Startup
Scaling the Platform for Your Startup
 
Comenzando com la nube hibrida
Comenzando com la nube hibrida Comenzando com la nube hibrida
Comenzando com la nube hibrida
 
AWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the CloudAWS Webcast - Website Hosting in the Cloud
AWS Webcast - Website Hosting in the Cloud
 
Apache CloudStack 4.2: A First Look
Apache CloudStack 4.2: A First LookApache CloudStack 4.2: A First Look
Apache CloudStack 4.2: A First Look
 
Create cloud service on AWS
Create cloud service on AWSCreate cloud service on AWS
Create cloud service on AWS
 
What's New with Big Data Analytics
What's New with Big Data AnalyticsWhat's New with Big Data Analytics
What's New with Big Data Analytics
 

Recently uploaded

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 

Recently uploaded (20)

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 

Building Complete Private Clouds with Apache CloudStack and Riak CS

  • 1. BUILDING COMPLETE PRIVATE CLOUDS WITH APACHE CLOUDSTACK + RIAK CS John Burwell (jburwell@basho.com | @john_burwell) Wednesday, October 23, 13
  • 2. Who Am I • Consulting Engineer @ Basho Technologies • Member, Apache CloudStack PMC • Ran operations and designed automated provisioning for hybrid analytic/virtualization clouds • Led architectural design and server-side development of a SaaS physical security platform Wednesday, October 23, 13
  • 3. ... but I submitted a ticket. Wednesday, October 23, 13
  • 4. Shifting Role Of IT From Cost Center To Business Opportunity Wednesday, October 23, 13
  • 6. Agility Exploitation Requires Rapid Execution Infrastructure management must be delegated Wednesday, October 23, 13
  • 7. Agility Exploitation Requires Rapid Execution Infrastructure management must be delegated Infrastructure Must Become Boring Wednesday, October 23, 13
  • 8. Composable Infrastructure • Resource Pooling • Rapid Elasticity • Self Service • Metering/Charge Back Wednesday, October 23, 13
  • 9. Got It ... Why Go Private? • Leverage existing infrastructure investment • Customization and Control • Regulatory/Security Requirements • Cost Wednesday, October 23, 13
  • 10. Doesn’t vSphere/ XCP/XenServer already do that? Wednesday, October 23, 13
  • 11. Traditional Workload • • Fault tolerant • Management Reliable Specialized hardware • IT managed provisioning • Static scale out model Wednesday, October 23, 13 Network Hypervisor Cluster Hypervisor Cluster Shared
  • 13. Traditional Workload • • Fault tolerant • Management Reliable Specialized hardware • IT managed provisioning • Static scale out model Network Hypervisor Cluster Hypervisor Cluster Shared
  • 14.  Storage Focused on resource consolidation not agility Wednesday, October 23, 13
  • 15. Software Is Eating Hardware • Moore’s Law has pulled fault tolerance up the stack • Commodity Hardware • • • Reduced cost Services more workload types Dynamic resource allocation Wednesday, October 23, 13
  • 16. Software Is Eating Hardware • Moore’s Law has pulled fault tolerance up the stack • Commodity Hardware • • • Reduced cost Services more workload types Dynamic resource allocation Specialized Hardware Reduces Agility Wednesday, October 23, 13
  • 17. Cloud Workload • Reliable • Fault Tolerant • Self-Service provisioning • Control Plane - Dynamic scale out model Rack Rack App App Commodity Hardware • SDN Wednesday, October 23, 13 Object
  • 19. Cloud Workload • Reliable • Fault Tolerant • Self-Service provisioning • Control Plane - Dynamic scale out model Rack Rack App App Commodity Hardware • SDN Object
  • 20.  Storage Resource optimization increases agility Wednesday, October 23, 13
  • 22. Complete Private Cloud + Compute Services Wednesday, October 23, 13 Object Storage Services
  • 23. What Is Apache CloudStack? §Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. §CloudStack is a Top-Level Project at the Apache Software Foundation. Wednesday, October 23, 13
  • 24. What Is Apache CloudStack? §Open source software that deploys and manages large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. §CloudStack is a Top-Level Project at the Apache Software Foundation. Wednesday, October 23, 13
  • 26. Anatomy Of CloudStack • Management Server Farm • • VM Network Host Collection of clusters in the same failure boundary Primary Storage A grouping of hosts and their associated storage Hosts • • Collection of pods, network offerings and secondary storage Cluster • • Host Pod • • VM Zone • • Management and provisioning tasks Cluster Secondary Storage Cluster Servers onto which services will be provisioned Primary Storage CloudStack Pod • • VM disk storage Network • CloudStack Pod Logical network associated with service offerings Zone • Secondary Storage • Template, snapshot and ISO storage Wednesday, October 23, 13
  • 27. Primary Vs Secondary Storage • Random access read/write • Write once/read many • Changes often • Append only • Zone Scoped • Zone or Region Scoped • Focused on throughput • Focused on reliability Wednesday, October 23, 13
  • 28. Primary Vs Secondary Storage • Random access read/write • Write once/read many • Changes often • Append only • Zone Scoped • Zone or Region Scoped • Focused on throughput • Focused on reliability Secondary Storage is the lifeboat. Wednesday, October 23, 13
  • 29. What Is Riak CS? • Simple, reliable storage of large objects accessed using the S3 or Swift API • Objects are stored in buckets with a unique key • Multi-tenant security model with per object ACLs • Usage Accounting and Reporting • Open Source (Apache2 Licensed) Wednesday, October 23, 13
  • 30. What Is Riak CS? • Simple, reliable storage of large objects accessed using the S3 or Swift API • Objects are stored in buckets with a unique key • Multi-tenant security model with per object ACLs • Usage Accounting and Reporting • Open Source (Apache2 Licensed) Your Own Private S3 Wednesday, October 23, 13
  • 31. Anatomy Of Riak CS • S3/Swift API Proxy + Riak key-value store • Peer-to-Peer Design: Any node can service any read or accept any write request • Maintains multiple replicas (default 3) spread across multiple nodes • Ops Friendly Wednesday, October 23, 13
  • 32. Anatomy Of Riak CS • S3/Swift API Proxy + Riak key-value store • Peer-to-Peer Design: Any node can service any read or accept any write request • Maintains multiple replicas (default 3) spread across multiple nodes • Ops Friendly Horizontally scaleable storage on commodity hardware Wednesday, October 23, 13
  • 33. Riak CS Use Cases • Secondary Storage for VM snapshot and template data • Database Backup Storage • Serving Static Content • Application Log Storage Wednesday, October 23, 13
  • 34. Achieving Commodity Reliability • Create a zone per power source/backbone network connection • Establish independent primary storage in each zone • Utilize intra-datacenter block-level replication between zones • Distribute Riak CS nodes evenly across zones Wednesday, October 23, 13
  • 35. The Complete Private Cloud Traditional
  • 39.  CS Wednesday, October 23, 13 Rack App Hypervisor Cluster Rack
  • 40. Up And Running With Riak CS • Learn more @ http://basho.com/riak-cloud-storage • Grab it @ http://docs.basho.com/riakcs/latest/riakcsdownloads/ • Follow us @basho • Help from the riak-users@lists.basho.com mailing list or on FreeNode @ #riak • Contribute @ http://github.com/basho/riak_cs Wednesday, October 23, 13
  • 41. Join Our Community • Learn more @ http://cloudstack.apache.org • Grab it @ http://cloudstack.apache.org/downloads.html • Follow us @cloudstack • Help from the cloudstack-users@cloudstack.apache.org mailing list or on Freenode @ #cloudstack • Contribute on the cloudstack-dev@cloudstack.apache.org mailing list or on Freenode @ #cloudstack-dev Wednesday, October 23, 13
  • 43. Thank you! Deck http://slideshare.net/JohnBurwell1 Follow @ john_burwell Email jburwell@basho.com Wednesday, October 23, 13