Presented by Giorgio Regni, CTO
Try Scality S3 Server Today!
https://s3.scality.com/
http://www.scality.com/scality-s3-server/
https://hub.docker.com/r/scality/s3server/
6. Customer & partner push for AWS S3 has swelled in last 18 months
AWS S3 has become de-facto interface standard
Growing demand
6
AWS S3 Adoption keeps rising
8. 8
Object vs File
Bucket 1
Object A
Object B
Object C
Bucket 2
Object A
Object B
Object C
Object/S3:
• Flat Data Model with collections called
“buckets”
• Objects are written and overwritten
not byte-wise modified
• Scales to billions of objects
File/Posix:
• Hierarchical Data Model
• Files are randomly writable in byte-
wise fashion
• Scales to hundreds of thousands
files
9. 9
Why object scales better
Separation of Metadata and Data
• Bucket listing and object locations are
stored separately from the Data
• Objects can be spread out anywhere
• Direct access to data - no need to
traverse a tree structure
Shared nothing:
• Consistency rules only apply to one
object or bucket at a time
• Clients are stateless - Last writer wins
• No relationships exist between
objects
Data Data Data
MD MD MD
Data…
…
SDS/Object Service
12. Steps:
1. Launch Kitematic UI to access Docker Hub
2. Pull the S3 Server container
3. Start the container
4. Use the Cyberduck UI to create Buckets, PUT, GET & DELETE Objects
Quick, Live AWS S3 Server Demo!
15. Logging is hard
• Challenges
• Logging is expensive as it taxes the Node.js process
• UDP datagrams have expensive DNS lookups
• Redundant transformations by bunyan and bunyan-logstash
• Solution: Werelogs
• Produces raw JSON logs with the least resistive path
• Forward logs to ELK using Filebeat for indexing
• Avoids expensive and redundant transformations
• Ability to track requests across the components with UIDs
• Dump log history on errors
Open source -> http://github.com/scality/werelogs
17. The performance cycle
Code, Benchmark, … Repeat
• Socket & Nagle algorithm on by default -> very high
latencies
• The event loop can get backed up quickly -> hunt for
all cpu intensive tasks in the main loop
• Buffers are much more efficient when writing server
response
• Micro optimizations: Date.now() > new Date()
• Beware of libraries doing way too many things for you
• ES6 support, Babel5 was killing performance -> Babel6
The Scality RING is software-defined storage. We run on standard x86 servers and create a giant pool of storage. We protect the data and provide 100% reliable, high performance access for any capacity-driven application. Why, what.
[On the same slide you can also talk about benefits]
Amazing capabilities and benefits:
100% reliability with no maintenance windows, for superior SLAs to your customers – case in point, our first customer hasn’t been down since they installed the RING five years ago!
The only storage platform with native file, object, and OpenStack support – legacy file, digital, and cloud workloads all in one platform for better utilization, holistic protection, and lower costs
Real hardware choice means the best hardware for you at any point in the lifecycle, and no data migration – better economics and lower risk
A little bit about Scality – founded in 2009 with our first customer in 2010.
Offices around the world - with local language support in US, France, and Japan
Managing over 800 billion objects in production at 100 customers around the world
Ranked #1 by IDC
Why we decided to refresh our S3 interface
Database have SQL
Files have NFS
Object has S3
De haut niveau = modèle de donnée.
So we thought, how can we help people try object?
Only object storage with scale-out access to any number of Buckets and objects
Fully redundant system design
Stateless S3 Servers integrate with standard load balancers
Highly-durable data protection through RING storage
Deluxe on Demand recognized it was not financially feasible to store content with the CDN provider. In this case, the CDN was Limelight. The hardware/software solution was a replacement for an Isilon system.
In addition:
Scality/HPE solution provided much lower TCO than EMC platform
Scality strong partnership with HP an advantage
Used to use Front Porch MAM, DIVA (data mover), small caching NAS (100TB) and LTO tape. With Scality, they only need a MAM and Scality. No need to move data between tiers. Front Porch charged annually per terabyte, so the cost was very high to maintain and scale.