Your SlideShare is downloading. ×
0
Easy Access to OpenStack Object Storage
Juan J. Martínez
@reidrac
http://www.usebox.net/jjm/

Merry OpenStack, London
Dec ...
What is Swift?
Open Source software for creating redundant, scalable object storage using
clusters of standardized servers...
CUSTOMERS
What we have
What we sell
What customers want
Solution: Proxies
Back in 2011...
●
●
●

Building a product based on OpenStack Object Storage
Started working with Bexar
Went live with Diab...
Proxies: FTP
ftp-cloudfs
●
●
●
●

Personal project of Chmouel Boudjnah (now @ https://github.com/cloudfs/ftp-cloudfs)
Open...
Proxies: SFTP
sftp-cloudfs
●
●
●
●
●
●
●

Open Source sponsored by Memset Hosting Ltd
https://github.com/Memset/sftpcloudf...
Bigger than the API
●
●
●
●
●
●

+2 TB / mo traffic (in > out)
Small VMs running ftp-cloudfs, sftp-cloudfs and shared cach...
Easy Access to OpenStack Object Storage

Questions?
Juan J. Martínez
@reidrac
http://www.usebox.net/jjm/

Merry OpenStack,...
Upcoming SlideShare
Loading in...5
×

Easy access to open stack object storage

1,858

Published on

Slides for a talk that didn't happen because of time constraints (OpenStack London Meetup, Dec 2013).

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,858
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Easy access to open stack object storage"

  1. 1. Easy Access to OpenStack Object Storage Juan J. Martínez @reidrac http://www.usebox.net/jjm/ Merry OpenStack, London Dec 2013
  2. 2. What is Swift? Open Source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. Object Storage ● ● ● ● Not a real file system RESTful HTTP API Blob and metadata (name, content type, date-time, size, MD5) Support for pseudo-hierarchical directories Real File System ● ● ● ● ● open, read, write, seek, close chdir, mkdir, rmdir remove, rename chmod, chown, stat, lock etc
  3. 3. CUSTOMERS
  4. 4. What we have
  5. 5. What we sell
  6. 6. What customers want
  7. 7. Solution: Proxies
  8. 8. Back in 2011... ● ● ● Building a product based on OpenStack Object Storage Started working with Bexar Went live with Diablo ● Not many end-user tools using OpenStack Object Storage API (actually, it was Rackspace Cloud Files™) ● ● ● Customers won’t start building things straight away Internal tools WIP (snapshot VM, re-image from snapshot) It was hard to explain the product*: What is Amazon S3? I want Dropbox! * disclaimer: I’m a developer
  9. 9. Proxies: FTP ftp-cloudfs ● ● ● ● Personal project of Chmouel Boudjnah (now @ https://github.com/cloudfs/ftp-cloudfs) Open Source Very cool, “almost working” (Jan 2011) fs.py abstraction: Swift API -> FTP Memset started contributing to make it “production ready”: ● ● ● ● ● Support for OpenStack Object Storage (was Cloud Files only) Missing bits in fs.py, more tests, introduced some new bugs Fixed performance problems: memcache all the things! Finally migrated from python-cloudfiles to python-swiftclient (kept the name, it’s cool) Keystone auth, resume downloads, listing +10k files, large file support (>5GB)
  10. 10. Proxies: SFTP sftp-cloudfs ● ● ● ● ● ● ● Open Source sponsored by Memset Hosting Ltd https://github.com/Memset/sftpcloudfs ftp-cloudfs’ fs.py + paramiko (ssh2 protocol for Python) Secure: client -> SSH -> sftp-cloudfs -> SSL -> swift Not as performant as FTP (1.6 MB/s vs pyftpdlib 4.2 MB/s), CPU intensive Only password based authentication Partial SCP support contributed by Koert van der Veer + everything supported by ftp-cloudfs
  11. 11. Bigger than the API ● ● ● ● ● ● +2 TB / mo traffic (in > out) Small VMs running ftp-cloudfs, sftp-cloudfs and shared cache (memcache) Load-balanced with LVS Easy to scale (add more VMs!) Use cases: backups, CMS plugins (CDN support), remote storage, ??? FTP > SFTP > API Useful Open Source projects: ● ● ● Numbers from PyPI ftp-cloudfs: 1895 downloads last month sftp-cloudfs: 1107 downloads last month Patches from several contributors (including hosting companies) “Stable” and almost feature complete, a new release every now and then
  12. 12. Easy Access to OpenStack Object Storage Questions? Juan J. Martínez @reidrac http://www.usebox.net/jjm/ Merry OpenStack, London Dec 2013
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×