Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hackathon scality holberton seagate 2016 v5

345 views

Published on

S3 Server, a Scality product, was born after a hackathon in Paris, France in 2015. What better way to continue with our philosophy of innovation than to host a hackathon of our own?

On October 21st, coders joined us for a weekend of coding, developing new solutions for storage, integrations for S3 and much more!

This event was sponsored by Seagate and hosted at Holberton School.

Published in: Technology

Hackathon scality holberton seagate 2016 v5

  1. 1. Welcome to the S3 Server Hackathon!
  2. 2. Cloud Field Days #1 © Scality 20163 Scality RING: Automatize storage for Digital Business The Scality RING is object-based software-defined storage for the cloud. We run on standard x86 servers and create a giant pool of storage. We protect the data and provide 100% reliable, high performance access for any capacity-driven application. FILE OBJECT OPENSTACK
  3. 3. Cloud Field Days #1 © Scality 20164 Agenda for today Presentation of the hackathon, judges and logistics S3 Server tech Overview - https://github.com/scality/s3 Seagate Kinetic drives Overview Teams and projects assignement!
  4. 4. The Judges!
  5. 5. The Prizes!
  6. 6. How are we judging? We know the code will not be beautiful! We’re looking for a functional prototype It’s OK not to handle quirky corner cases!
  7. 7. How are we judging? Innovation - We’re looking for creative solutions Market potential - It has to solve a real problem! Execution - A working demo ☺
  8. 8. Each team has: A name, A mentor, A clear defined project that’s achievable, Developers and a leader, Each team will do a 5 minutes pitch + demo on Sunday
  9. 9. Logistics!
  10. 10. – – –
  11. 11. Open source S3 Server!
  12. 12. S3 Adoption keeps rising
  13. 13. © Scality 201614 ▪ Open source object storage server https://github.com/scality/s3 ▪ Written in Node.js ▪ Single instance running in a docker container ▪ Uses docker volumes for persistent storage ▪ Same code as Scality’s RING S3 interface What is S3 Server?
  14. 14. © Scality 201615 ▪ Developers can install and develop S3-based apps locally ▪ Enterprises can host a local test/dev environment to learn about object storage ▪ Enterprises can host a small, local object storage system in production Scality S3 Server: From 0 to S3 in Under 5 Minutes S3 Serve r S3 Serve r S3 Serve r Backup Application S3
  15. 15. The team SF & Paris
  16. 16. © Scality 2016 Scality S3 Server achitecture Protocol Stack Buckets, Objects, MPU (REST API) AUTH BUCKETS DATA LevelDB* Scality RING v6 Kinetic IP drives Docker Volume * Persisted in Docker Volume Scality RING v6Vault JSON* S3 API S3 Server Public Cloud
  17. 17. © Scality 201618 S3 server released under a Apache-2.0 on Github 3.3K downloads on dockerhub! 18
  18. 18. Seagate Kinetic Kinetic development and the basics
  19. 19. Kinetic On SMR (Shingled Magnetic Recording) What is Kinetic Kinetic is a drive architecture in which the drive is a key/value service with Ethernet connectivity Put/Get/Delete API over TCP/SSL https://www.openkinetic.org/ SMR Data recording technology used in drives to increase storage density and overall storage capacity https://en.wikipedia.org/wiki/Shingled_magnetic_recording Why Kinetic on SMR Increase storage capacity Same Put/Get/Delete API (SMR is transparent to users/applications)
  20. 20. Kinetic Development Overview Kinetic Simulator Functionally equivalent to Kinetic Drives. https://github.com/Kinetic/kinetic-java Build git clone https://github.com/Kinetic/kinetic-java.git mvn clean package Run (one or more instances) java -jar kinetic-simulator/target/kinetic-simulator-"Version"-SNAPSHOT-jar-with-dependencies.jar or ./bin/startSimulator.sh More Information Related to Build https://github.com/Kinetic/kinetic-java/blob/master/README.md Preferred IDE: Eclipse (Mars and up) http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/neon1a
  21. 21. Notes On Kinetic K/V Keys Keys are sorted lexicographically. e.g. http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/primitives/ UnsignedBytes.html#lexicographicalComparator%28%29 https://github.com/Kinetic/kinetic-java/blob/master/kinetic-simulator/src/main/java/com/se agate/kinetic/simulator/persist/leveldb/KineticComparator.java Use sequential keys for better throughput (not required for later releases) An example to generate sequential keys https://github.com/Kinetic/kinetic-java/blob/master/kinetic-test/src/test/java/com/seagate/ki netic/KVGenerator.java
  22. 22. Hackathon Group Key Space Each group is recommended to use its own assigned key space (prefix) to avoid conflict for shared drives. For example, group 1 is assigned prefix of “00”, group 2 is assigned prefix of “01”., etc. Eg: All keys in group 1 would be in the form of “000”, “001”, “002” ..., etc. All keys in group 2 would be in the form of “010”, “011”, “012” ..., etc. All keys in group 3 would be in the form of “020”, “021”, “022” ..., etc.
  23. 23. Scality/Kinetic Hackathon Thank You!
  24. 24. Thank you!

×