Downloaded 273 times
Uploaded byPark YounSung
Openstack Swift Introduction
Swift is an object storage system that provides highly available, distributed, and eventually consistent storage. It features include high availability through replication, high scalability by adding additional servers and storage, and self-healing capabilities to recover from failures. It uses a REST API and supports features like large object storage, static web hosting, S3 compatibility, object expiration, temporary URLs, object versioning, global clusters, and storage policies.
More Related Content
Similar to Openstack Swift Introduction
Openstack Swift Introduction
- 1. Swift Highly available, distributed,eventually consistent object storage Youn Sung Park
- 2. March 16, 2015Presentation Title 2 1. Introduction 2. Features - Basic 3. Features - Advanced Contents
- 3.
- 4. Contents • What isSwift? Object Storage Swift component March 16, 2015 Swift 4 Introduction
- 5. Object Storage • ObjectStorage – What is Object Storage? March 16, 2015 Swift 5 Introduction Swift - Highly available, distributed, eventually consistent object storage
- 6. Object Storage • ObjectStorage – What is Object Storage? March 16, 2015 Swift 6 Introduction
- 7. Object Storage • ObjectStorage – What is Object Storage? March 16, 2015 Swift 7 Introduction Object storage vs File storage vs Volume storage
- 8. Object Storage • ObjectStorage – Legacy Storage March 16, 2015 Swift 8 Introduction WAS WAS WAS NAS / SAN NAS / SAN NAS / SAN NAS / SAN … …
- 9. Object Storage • ObjectStorage – Legacy Storage March 16, 2015 Swift 9 Introduction WAS WAS WAS NAS / SAN NAS / SAN NAS / SAN NAS / SAN … … Storage manager
- 10. Object Storage • ObjectStorage – Legacy Storage • Fault tolerance – RAID5, 6, 10, ... • Interface – NFS for file, iSCSI for volume • Multi tenancy - ?? • QoS - ?? • Authentication - ?? • Data Placement - ?? • Metadata management - ?? • Capacity management - ?? • Rebalancing - ?? March 16, 2015 Swift 10 Introduction
- 11. Object Storage • ObjectStorage March 16, 2015 Swift 11 Introduction WAS WAS WAS Object Storage …
- 12. Object Storage • ObjectStorage – Legacy Storage • Fault tolerance – Replication • Interface – REST API • Multi tenancy – Defaut. SWAuth, Keystone, Tempauth, LDAP(?) • QoS – Rate limit, Bandwidth limit, Software defined ... • Authentication - SWAuth, Keystone, Tempauth • Data Placement – The Ring algorithm • Metadata management – Ring and database • Capacity management – The Ring algorithm • Rebalancing – The Ring algorithm March 16, 2015 Swift 12 Introduction
- 13. Components • Swift component– Server application March 16, 2015 Swift 13 Introduction Proxy Server Account Server Container Server Object Server Sqlite Container List Object List Objects Sqlite
- 14. Components • Swift component– Daemon • Replicator – Replicate objects and make a system in a consistent state. • Recover disk failure, network outages situation. • Updater – Update metadata • Recover failure caused by container, account metadata high load. • Auditor – Delete problematic account, container or objects and replicate from other server • Recover dbs or files which have bit rot problem. March 16, 2015 Swift 14 Introduction
- 15. Components • Swift component– Key algorithm: Data placement, Metadata management March 16, 2015 Swift 15 Introduction RingContainer name/file name Location of a file ex) 3 copy Server A / sdc Server H / sdy Server D / sdp
- 16.
- 17. Contents • Swift features History– Releases Architecture High availability High scalability Eventually consistent & Self healing REST API March 16, 2015 Swift 17 Features - Basic
- 18. History - Releases March16, 2015 Swift 18 Features - Basic
- 19. Architecture • Architecture March 16, 2015 Swift 19 Features- Basic Proxy Server Proxy Server Proxy Server A/C/O Server A/C/O Server A/C/O Server Zone1 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone2 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone3 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone4 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone5 A/C/O Server … … … Sqlite Account server Container server Object server Sqlite Ring Ring Ring
- 20. Features • High availability,Fault tolerance March 16, 2015 Swift 20 Features - Basic Proxy Server Proxy Server Proxy Server A/C/O Server A/C/O Server A/C/O Server Zone1 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone2 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone3 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone4 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone5 A/C/O Server … … … Sqlite Account server Container server Object server Sqlite Ring Ring Ring
- 21. Features • High scalability March16, 2015 Swift 21 Features - Basic A/C/O Server HDDHDDHDDHDD A/C/O Server HDDHDDHDDHDD HDDHDDHDDHDD More disk Zone1 Zone1 Zone1 Zone1 Zone5 Zone1 Zone1 Zone1 Zone1 Zone1 Zone1 Zone7 More zones A/C/O Server A/C/O Server Zone A/C/O Server A/C/O Server Zone A/C/O Server A/C/O Server More servers
- 22. Features • Eventually consistent& self healing March 16, 2015 Swift 22 Features - Basic Proxy Server Proxy Server Proxy Server A/C/O Server A/C/O Server A/C/O Server Zone1 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone2 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone3 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone4 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone5 A/C/O Server … … … Sqlite Account server Container server Object server Sqlite Fail Eventual consistency Ring Ring Ring
- 23. Features • Eventually consistent& self healing March 16, 2015 Swift 23 Proxy Server Proxy Server A/C/O Server A/C/O Server Zone1 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone2 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone3 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone4 A/C/O Server … A/C/O Server A/C/O Server A/C/O Server Zone5 A/C/O Server … … … Sqlite Account server Container server Object server Sqlite Server failure Self healing - replication A/C/O Server Proxy Server Features - Basic Ring RingRing
- 24.
- 25. Contents • Swift features History– Features Features Large object support Static web hosting S3 compatible API Object expiration Temp url Object versioning Global cluster Storage policy March 16, 2015 Swift 25 Features - Advanced
- 26. History - Features March16, 2015 Swift 26 Features - Advanced Swift 2.2 Release Storage policy support Swift 1.10 Release Global clusters support Swift 1.8 Release Different replica policy for A/C/O StatsD update Swift 1.7 Release Object versioning support Statd integration Swift 1.4.8 Release Object expiration support Temp url support Swift 1.4.3 Release S3 compatible API support Swift 1.3 Release Static website support Swift 1.2 Release Large object support Swift 1.0 Release Swift 1.12 Release Account ACL support
- 27. Features • Large objectsupport • Swift limitation • Single object: 5GB • Split object & manage large object • Manage segmented objects by manifest file • Ref • http://docs.openstack.org/developer/swift/overview_large_objects.html March 16, 2015 Swift 27 Features - Advanced
- 28. Features • Static webhosting • Upload static web file and make web site • Upload web site file with index and error files • Use statiscweb middleware • Ref • http://docs.openstack.org/developer/swift/middleware.html#staticweb March 16, 2015 Swift 28 Features - Advanced
- 29. Features • S3 compatibleAPI • Support S3 API • Support limited API less than 40% • Use swift3 middleware • Ref • https://github.com/stackforge/swift3 March 16, 2015 Swift 29 Features - Advanced
- 30. Features • Object expiration •Schedule deletion of objects • Use X-Delete-At and X-Delete-After header while using an object PUT or POST •X-Delete-At: Delete object at specified time •X-Delete-After: Delete object after specified time • Ref • http://docs.openstack.org/developer/swift/overview_expiring_objects.html March 16, 2015 Swift 30 Features - Advanced
- 31. Features • Temp url •Provide url to access in limited time • Need temp_url_expires time in header • Use temporary URL middleware • Ref • http://docs.openstack.org/developer/swift/api/temporary_url_middleware.html March 16, 2015 Swift 31 Features - Advanced
- 32. Features • Global cluster •Make a single cluster in distant region • Read/Write affinity • Deferred replication • Ref • http://docs.openstack.org/developer/swift/admin_guide.html • https://swiftstack.com/blog/2012/09/16/globally-distributed-openstack-swift-cluster/ March 16, 2015 Swift 32 Features - Advanced
- 33. Features • Storage policy •Support various policy in sing storage cluster • Use multiple ring file • Ref • http://docs.openstack.org/developer/swift/admin_guide.html March 16, 2015 Swift 33 Features - Advanced
- 34.