OpenStack Swift In the Enterprise
An Introduction to Object Storage and Swift
1
• This webinar is being recorded and an on-
demand version will be available at the
same URL within 15 minutes after we
conclude
• Please submit questions via the button to
the upper left of the viewer
• On Twitter? Join the conversation:
#objectstorage, #swift, #openstack
@HOSTINGdotcom
2
Housekeeping
• Critical factors driving new storage types
• What is Object Storage?
• OpenStack Swift Architecture
• Key Considerations
3
Today’s Agenda
4
Critical factors driving new storage
types
• Object storage began when
applications started to store
and serve data to any device
regardless of where they are
connecting from
• Explosion in non-transactional
data (i.e. Machine generated,
rich media content, etc.)
• Cloud architectures
demanding alternatives to
storing and retrieving content
Critical factors driving new storage
types cont..
• Retention of data has
increased
• Performance
degrades as number
of files or storage
increases
• IT budgets are not
increasing
5
Data growth is exploding
A look at Traditional Storage Types
Block Based File Based
• Sits on top of Block Based Storage
• Stores FilesFolders that are
humanly readable
• Can increase and decrease size of
storage
• NIFSCIFS Protocols allow for a
single namespace
• Scaling limitations: Metadata is
held within the file, performance
suffers as number of files increase
• Is designed for LAN based access,
not Internet based devices
6
• Data is structured with a maximum
length (i.e. block size)
• Typically stored in SAN
environments where data is stored
in volumes
• Can increase storage, however, it
cannot decrease* (*without some
very nifty & complex tooling)
• Scaling the size of a block disk is
subject to vendor and partition
limitations
• Is designed for to be accessed by
low latency, high bandwidth
networking
7
Introduction to Object Storage
• Object storage, often called Cloud
Storage, is commonly used by cloud
providers due to it strength in
dynamically increasing or decreasing
storage demand dictates
• All objects (i.e. movies, pictures,
documents, etc.) are stored with
• A unique Identifier
• Meta Data
• Attributes
• Data
• Is accessed via HTTP restful API
allowing applications to access the data
from anywhere
• The most commonly used Object
Platforms are AWS’s S3, and
Object Storage: What
this Enables
• Consider an application where
patient records are inputted
• The application has form fields
where the care providers inputs
diagnosis, patient local,
symptoms, etc. (i.e. Metadata of
a patient record)
• Now imagine the care provider
or research facility wants to look
at this data from not just one
hospital or location but abroad
to learn about trends
• How can this data be accessed
independent of any one facility
while ensuring the security?
Source: RightScale 2014 State of the Cloud Report
• Object Storage allows application to retrieve
metadata of a file from billions objects
• The retrieval of a given object is
authenticated and authorized
• The data can be retrieved over the Internet
or over private networks instantly via
HTTP(s)
Swift based Object Storage
9
Feature Benefit
Swift is a core OpenStack Project OpenStack is the most successful open
source project ever that set out to solve
common challenges with cloud
technologies.
Deployment Flexibility Can be deployedconsumed from Public,
Private, Hybrid cloud designs
Unlimited Storage Huge & flat namespace, highly scalable
readwrite access. Ability to serve content
directly to from storage systems
Multi-dimensional Scalability (scale out
architecture)
Backup and archive large amounts of data
with linear performance
Easily* add capacity unlike RAID resize Elastic data scaling with ease
No central database Higher performance, no bottlenecks
Real-time visibility into client requests Know what exactly users are requesting
Numerous ways to access storage Users can access storage via CLI, HTTP,
file-system gateways
The Core Components of Swift
10
Swift Proxy
Storage Node
Identity Service
Understands where data lives &
routes requests
Runs account, container, &
object services
Authenticates and authorizes
storage requests
Swift Data Model
Three components to
unstructured data
• Account: A unique name
storage area that will
contain the metadata
• Container: User-defined
storage area containing
metadata of the
container and list of
objects
• Object: Is where the
data object and its
metadata is held
Account: Customer A
Container 1
Object
Orange
Object
Green
Swift Zones and
Regions
• Zones:
• Replicates data among nodes
within the environment
• Default replica is 3
• Ensure no networking  or
hardware single point of failure
• Regions:
• Conceptually designed to
distribute replicas among
geographically dispersed
areas
• Used for offsite DR purposes
• Used in ActiveActive Multi-
sharing environments
Region A Region B
Zone A
Node 1
Node 2
Node 3
Node 4
Node 5
Zone B
Node 1
Node 2
Node 3
Node 4
Node 5
• OpenStack Swift and
Amazon S3 are
eventually consistent
platforms
• CAP Theorem: These
platforms follow
availability and partition
tolerance dropping
consistency
Data Consistency Processes
• Swifts eventual consistency
is not a good fit for
applications needing ACID
guarantees such as:
• Baking systems that must
ensure two transactions don't
withdraw the same money
• Ticketing systems that must
ensure two people never buy
the same ticket
Not a good fit
Common Swift Use Cases
15
Scientific /
Research
Data
Rich Digital
Media
Archive and
Backups
• Manage rich
media
content’s
explosive
growth and file
size
• Consistent
performance
at scale
• Access to a
single system
that manages
billions and
petabytes of
recordsobjects
• Object metadata
makes it
possible to
conduct complex
searches
• Legal and
governance
requirements
• Monetization
of data
• Cheep and
deep
Key Considerations
Performance
• Separate Account and Container
databases to SSD backed storage
• Add and remove storage
inclemently as these operations
will incur ring rebalancing
Availability
• Use a load balancer in front of the
Proxy nodes for HA
• Consult your storage vendor when
using tradition NASSAN storage
as RAID technologies will overlap
with OpenStack’s replica
technology
Private | Public | Hybrid Swift
17
18
• Combining eplexity’s OpenStack
Swift expertise…
• OpenStack-focused, deployment
specialists
• Over 40PB+ systems management
• … with HOSTING’s World-Class
Infrastructure
• 6 privately-owned and managed
datacenters
• Specialists in business-critical
application availability
• Unique Support Experience
1
Our OpenStack Swift Solution
• No DIY required – Get started
and scale!
• Designed from the ground up for
Private Cloud
19
Q&A
John Clendennen | CTO, eplexity
Mike McCracken | Director, Professional Services, HOSTING
For more information about solutions by HOSTING, please contact our
team at 888.894.4678.

OpenStack Swift In the Enterprise

  • 1.
    OpenStack Swift Inthe Enterprise An Introduction to Object Storage and Swift 1
  • 2.
    • This webinaris being recorded and an on- demand version will be available at the same URL within 15 minutes after we conclude • Please submit questions via the button to the upper left of the viewer • On Twitter? Join the conversation: #objectstorage, #swift, #openstack @HOSTINGdotcom 2 Housekeeping
  • 3.
    • Critical factorsdriving new storage types • What is Object Storage? • OpenStack Swift Architecture • Key Considerations 3 Today’s Agenda
  • 4.
    4 Critical factors drivingnew storage types • Object storage began when applications started to store and serve data to any device regardless of where they are connecting from • Explosion in non-transactional data (i.e. Machine generated, rich media content, etc.) • Cloud architectures demanding alternatives to storing and retrieving content
  • 5.
    Critical factors drivingnew storage types cont.. • Retention of data has increased • Performance degrades as number of files or storage increases • IT budgets are not increasing 5 Data growth is exploding
  • 6.
    A look atTraditional Storage Types Block Based File Based • Sits on top of Block Based Storage • Stores FilesFolders that are humanly readable • Can increase and decrease size of storage • NIFSCIFS Protocols allow for a single namespace • Scaling limitations: Metadata is held within the file, performance suffers as number of files increase • Is designed for LAN based access, not Internet based devices 6 • Data is structured with a maximum length (i.e. block size) • Typically stored in SAN environments where data is stored in volumes • Can increase storage, however, it cannot decrease* (*without some very nifty & complex tooling) • Scaling the size of a block disk is subject to vendor and partition limitations • Is designed for to be accessed by low latency, high bandwidth networking
  • 7.
    7 Introduction to ObjectStorage • Object storage, often called Cloud Storage, is commonly used by cloud providers due to it strength in dynamically increasing or decreasing storage demand dictates • All objects (i.e. movies, pictures, documents, etc.) are stored with • A unique Identifier • Meta Data • Attributes • Data • Is accessed via HTTP restful API allowing applications to access the data from anywhere • The most commonly used Object Platforms are AWS’s S3, and
  • 8.
    Object Storage: What thisEnables • Consider an application where patient records are inputted • The application has form fields where the care providers inputs diagnosis, patient local, symptoms, etc. (i.e. Metadata of a patient record) • Now imagine the care provider or research facility wants to look at this data from not just one hospital or location but abroad to learn about trends • How can this data be accessed independent of any one facility while ensuring the security? Source: RightScale 2014 State of the Cloud Report • Object Storage allows application to retrieve metadata of a file from billions objects • The retrieval of a given object is authenticated and authorized • The data can be retrieved over the Internet or over private networks instantly via HTTP(s)
  • 9.
    Swift based ObjectStorage 9 Feature Benefit Swift is a core OpenStack Project OpenStack is the most successful open source project ever that set out to solve common challenges with cloud technologies. Deployment Flexibility Can be deployedconsumed from Public, Private, Hybrid cloud designs Unlimited Storage Huge & flat namespace, highly scalable readwrite access. Ability to serve content directly to from storage systems Multi-dimensional Scalability (scale out architecture) Backup and archive large amounts of data with linear performance Easily* add capacity unlike RAID resize Elastic data scaling with ease No central database Higher performance, no bottlenecks Real-time visibility into client requests Know what exactly users are requesting Numerous ways to access storage Users can access storage via CLI, HTTP, file-system gateways
  • 10.
    The Core Componentsof Swift 10 Swift Proxy Storage Node Identity Service Understands where data lives & routes requests Runs account, container, & object services Authenticates and authorizes storage requests
  • 11.
    Swift Data Model Threecomponents to unstructured data • Account: A unique name storage area that will contain the metadata • Container: User-defined storage area containing metadata of the container and list of objects • Object: Is where the data object and its metadata is held Account: Customer A Container 1 Object Orange Object Green
  • 12.
    Swift Zones and Regions •Zones: • Replicates data among nodes within the environment • Default replica is 3 • Ensure no networking or hardware single point of failure • Regions: • Conceptually designed to distribute replicas among geographically dispersed areas • Used for offsite DR purposes • Used in ActiveActive Multi- sharing environments Region A Region B Zone A Node 1 Node 2 Node 3 Node 4 Node 5 Zone B Node 1 Node 2 Node 3 Node 4 Node 5
  • 13.
    • OpenStack Swiftand Amazon S3 are eventually consistent platforms • CAP Theorem: These platforms follow availability and partition tolerance dropping consistency Data Consistency Processes
  • 14.
    • Swifts eventualconsistency is not a good fit for applications needing ACID guarantees such as: • Baking systems that must ensure two transactions don't withdraw the same money • Ticketing systems that must ensure two people never buy the same ticket Not a good fit
  • 15.
    Common Swift UseCases 15 Scientific / Research Data Rich Digital Media Archive and Backups • Manage rich media content’s explosive growth and file size • Consistent performance at scale • Access to a single system that manages billions and petabytes of recordsobjects • Object metadata makes it possible to conduct complex searches • Legal and governance requirements • Monetization of data • Cheep and deep
  • 16.
    Key Considerations Performance • SeparateAccount and Container databases to SSD backed storage • Add and remove storage inclemently as these operations will incur ring rebalancing Availability • Use a load balancer in front of the Proxy nodes for HA • Consult your storage vendor when using tradition NASSAN storage as RAID technologies will overlap with OpenStack’s replica technology
  • 17.
    Private | Public| Hybrid Swift 17
  • 18.
    18 • Combining eplexity’sOpenStack Swift expertise… • OpenStack-focused, deployment specialists • Over 40PB+ systems management • … with HOSTING’s World-Class Infrastructure • 6 privately-owned and managed datacenters • Specialists in business-critical application availability • Unique Support Experience 1 Our OpenStack Swift Solution • No DIY required – Get started and scale! • Designed from the ground up for Private Cloud
  • 19.
    19 Q&A John Clendennen |CTO, eplexity Mike McCracken | Director, Professional Services, HOSTING For more information about solutions by HOSTING, please contact our team at 888.894.4678.

Editor's Notes

  • #4 Critical factors for discussion: web\mobile applications and Cloud architectures Cloud Architectures Overview of Object storage and how it compares to traditional File Systems. Lastly - what are the use cases for Object storage? An overview of Swifts components, design examples and