• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
OpenStack Swift overview oscon2011
 

OpenStack Swift overview oscon2011

on

  • 4,474 views

 

Statistics

Views

Total Views
4,474
Views on SlideShare
4,474
Embed Views
0

Actions

Likes
12
Downloads
336
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    OpenStack Swift overview oscon2011 OpenStack Swift overview oscon2011 Presentation Transcript

    • Wednesday, July 27, 11
    • Swift Overview John Dickinson Swift Project Technical LeadWednesday, July 27, 11
    • What We’ll Cover ‣ Intro to Swift ‣ Architectural Overview ‣ Best Practices for Deploying ‣ Future workWednesday, July 27, 11
    • Swift is Scalable Storage ‣ Scales as the cluster grows ‣ Designed to run on commodity hardware ‣ No single point of failureWednesday, July 27, 11
    • Swift is not ‣ RAID ‣ Distributed Filesystem ‣ CDN ‣ SAN/NAS/DASWednesday, July 27, 11
    • Swift Use Cases ‣ Multi-tenant ‣ Backups ‣ Typically large ‣ Write-once, read-never ‣ Web Content ‣ Typically small ‣ Write-many, read-very-manyWednesday, July 27, 11
    • Swift Use Cases (cont.) ‣ Document management ‣ Medical imaging ‣ Disaster recovery ‣ Caching ‣ Storage appliances ‣ Scientific dataWednesday, July 27, 11
    • Example Usage ‣ Standard HTTP verbs ‣ Standard HTTP response codesWednesday, July 27, 11
    • Designing Swift ‣ At scale, everything breaks ‣ Design for failure ‣ The “Happy Path” is trivial ‣ At scale, even rare events happen a lot ‣ “Simple is better than complex.”Wednesday, July 27, 11
    • Swift History ‣ Rackspace Cloud Files (circa 2008) ‣ Distributed Storage ‣ Centralized MetadataWednesday, July 27, 11
    • Swift History (cont.) ‣ Rackspace Cloud Files (circa 2010) ‣ Redesigned from the ground up ‣ Distributed storage and metadata ‣ Worked closely with opsWednesday, July 27, 11
    • Modular Design of Swift ‣ Proxy Server ‣ The Ring ‣ Storage Servers ‣ Consistency ServersWednesday, July 27, 11
    • Proxy Server ‣ Public face of swift ‣ Determines appropriate storage nodes ‣ https://swift.example.com/v1/account/container/object ‣ Coordinates responsesWednesday, July 27, 11
    • The Ring ‣ Maps requests to storage nodes ‣ Availability zones ‣ Expandable without affecting most entitiesWednesday, July 27, 11
    • The Ring, in detail ‣ Started as a consistent-hashing ring ‣ Availability zones ‣ Became a static map of partitions to volumes ‣ What is a partition? ‣ What is a volume?Wednesday, July 27, 11
    • Storage Servers ‣ Account server ‣ Container server ‣ Object serverWednesday, July 27, 11
    • Storage Servers (Account and Container) ‣ SQLite databases ‣ Groups containers and objects into accounts ‣ Simple schema ‣ Table for listings ‣ Table for metadata ‣ Scale concerns?Wednesday, July 27, 11
    • Storage Servers (Object) ‣ Use the file system to store files ‣ Any file system that supports xattrs ‣ Files named by timestamp ‣ Tombstones ‣ Directory structure ‣ /mount/data_dir/partition/hash_suffix/hash/object.ts ‣ Directory metadata can easily exhaust RAMWednesday, July 27, 11
    • Consistency Servers ‣ Replicators ‣ Updaters ‣ AuditorsWednesday, July 27, 11
    • How to Deploy ‣ Swift All-In-One (SAIO) ‣ Small cluster ‣ Large clusterWednesday, July 27, 11
    • How to Deploy (SAIO) ‣ All services in one VM ‣ Use loopback devices for storage ‣ Great for dev work and validation testingWednesday, July 27, 11
    • How to Deploy (Small Cluster) ‣ Two load-balanced proxies ‣ Five storage nodes ‣ This area needs workWednesday, July 27, 11
    • How to Deploy (Large Cluster) ‣ Five zones ‣ Two proxies per zone ‣ Many storage nodes per zoneWednesday, July 27, 11
    • How to Deploy (Large Cluster, networking) ‣ 10g external network ‣ 1g (or 10g) internal network ‣ Use-case concerns ‣ Internal proxy network divided by replica countWednesday, July 27, 11
    • How to Deploy (Large Cluster, IO) ‣ Dedicated Container and Account servers ‣ Few, relative to object servers ‣ Many IOPS (i.e. SSDs) ‣ Cheap storage for Object serversWednesday, July 27, 11
    • Future Dev Work ‣ Better Ring ‣ Layered zones ‣ Unique-of-zones-as-possible ‣ Tiered storage ‣ More modular deployment ‣ Cabinets, not zones ‣ More featuresWednesday, July 27, 11
    • What Swift is “Missing” ‣ Auth ‣ ID management ‣ Stats ‣ Quotas ‣ Billing ‣ CDNWednesday, July 27, 11
    • Questions? ‣ #openstack on freenode ‣ http://launchpad.net/swift ‣ http://d.not.mn/swift_overview_oscon2011.pdfWednesday, July 27, 11
    • John Dickinson ‣ @notmyname ‣ http://gplus.to/notmyname ‣ me@not.mn ‣ http://not.mnWednesday, July 27, 11