Downloaded 38 times
Uploaded byGluster.org
Object Storage with Gluster
This document discusses using Gluster object storage with OpenStack Swift. Gluster-Swift mounts the Swift storage using FUSE and allows Swift to interface with Gluster backends. This avoids reimplementing the Swift object API. Gluster-Swift overrides Swift's distribution and replication to use the Gluster backend. The Swift API is implemented using FUSE operations on the Gluster volume. Future work includes upgrading Gluster-Swift, packaging, optimizations, and potentially developing a native Gluster object interface.
More Related Content
Similar to Object Storage with Gluster
Object Storage with Gluster
- 1. Object storage withGluster Prashanth Pai ppai@redhat.com
- 2. gluster-swift ➔ Just anotherapplication using fuse mount ➔ No libgfapi, yet ➔ Uses and “plugs into” OpenStack Swift ➔ Against having to write our own object front-end ➔ No catching-up to do ➔ Swift is largely decoupled from OpenStack ➔ Overrides vanilla Swift’s distribution and replication
- 3. Object access trivia ➔Object = {Data + Metadata + Identifier} ➔ Abstracted ➔ Stateless (HTTP) ➔ No partial PUTs ➔ Ranged GETs supported ➔ No renames, only COPY ➔ Fixed logical hierarchy ➔ Accounts/Container/Objects ➔ But you can’t mount it
- 4. Vanilla Swift Ring SQLite DB SQLiteDB XFS Account Server Container Server Object Server Proxy Server Client
- 5.
- 6. Proxy Server -pipeline /etc/swift/proxy-server.conf pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk tempurl ratelimit tempauth copy container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server ➔ Very similar to translators in gluster ➔ Middlewares are highly modular ➔ Many features are implemented as middlewares
- 7. Amazon S3 APISupport ➔ Implemented as a middleware (swift3) ➔ Converts incoming S3 requests into Swift requests S3 Request Swift Response Swift3 middleware Swift Request S3 Response Proxy Server S3 Client
- 8. Pluggability in Swift •Backends implement DiskFile • Object server is pluggable officially. • Account and container code overrides aren’t. • Makes cross-version compatibility tricky.
- 9. Swift Object Ops→ Fops PUT mkdir()* getxattr() open() write() fsetxattr() fsync() rename() close() GET open() fstat() fgetxattr() read() close() POST getxattr() stat() setxattr() DELETE getxattr() stat() unlink() rmdir()* *multiple calls - recursive HEAD getxattr() stat()
- 10. Swift Listing Ops→ Fops Container GET (plain text) – only names getxattr() opendir() readdir()* getxattr()1 Container GET (json/xml) – names + metadata getxattr() opendir() getdents()* getxattr()2 *multiple calls – recursive 1 only on dirs 2 on every inode
- 11. # getfattr -d/mnt/gluster-object/test/c1/animals/cat/kitten user.swift.metadata="{ "Content-Length":"4", "Etag":"4a4be40c96ac6314e91d93f38043a634", "X-Timestamp":"1475214280.33138", "X-Object-Type":"file", "X-Type":"Object", "Content-Type":"application/x-www-form-urlencoded"}" # curl -i http://host:8080/v1/AUTH_test/c1/animals/cat/kitten -X PUT -d'meow' HTTP/1.1 201 Created Last-Modified: Fri, 30 Sep 2016 05:44:41 GMT Content-Length: 0 Etag: 4a4be40c96ac6314e91d93f38043a634 Content-Type: text/html; charset=UTF-8 X-Trans-Id: txae858eb9169a4e1eb2a09-0057edfbc8 Date: Fri, 30 Sep 2016 05:44:40 GMT Object PUT
- 12. Swift API Violations #curl -i http://localhost:8080/v1/AUTH_test/c2 -X GET HTTP/1.1 200 OK Content-Length: 7 X-Container-Object-Count: 2 Accept-Ranges: bytes X-Storage-Policy: Policy-0 X-Container-Bytes-Used: 0 X-Timestamp: 0000000001.00000 Content-Type: text/plain; charset=utf-8 X-Trans-Id: tx18e21ccd03f04f4698a1a-0057ed1b84 Date: Thu, 29 Sep 2016 13:47:48 GMT HTTP Header needs to go first, then the response body.
- 13. Swift API Violations #curl http://host:8080/v1/AUTH_test/c1/animals/cat -X PUT Now, cat is a file # curl http://host:8080/v1/AUTH_test/c1/animals/cat/kitten -X PUT Here, cat is supposed to be directory but it’s already a file. Backend object path on filesystem gluster-swift: /mnt/gluster-object/test/c1/animals/cat Vanilla Swift: /mnt/sdb1/objects/778/69f/c2b307d78b6c419c0c1b76d91c08c69f/1412628708.01757.data
- 14. TODOs ➔ Bring upgluster-swift to ‘newton’ ➔ Centos SIG packages ➔ Zero copy and copy offload ➔ Use libgfapi python bindings ➔ Use hummingbird project ➔ Native object ? glfs_put(..), glfs_get(..) Ideas
- 15.