Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ConcourseCI love Minio

1,065 views

Published on

Due to some design decision with ConcourseCI there is no sharing between JOB, so Object storage is a good way of achivig this. Minio is an OSS Object storage with a great coverage of S3 api.

Published in: Technology
  • Be the first to comment

ConcourseCI love Minio

  1. 1. 30/11/2016 Etourneau Gwenn @The_shinji62 ConcourseCI Minio
  2. 2. About me Etourneau Gwenn Sr Solution Architect Bef.: Platform Architect https://github.com/shinji62 https://twitter.com/the_shinji62
  3. 3. Pivotal • Spring Framework • RabbitMQ • Concourse • Cloud Foundry • Pivotal Tracker • Gemfire
  4. 4. Meetup
  5. 5. Agenda • Concourse • Minio • Minio for Concourse • Q&A
  6. 6. ConcourseCI
  7. 7. ConcourseCI • Do not allow snowflake • Isolation between job • Non share volume / storage / cache
  8. 8. Jobs 1 Jobs 2 file_out file_out X Concourse
  9. 9. Concourse • Concourse support S3 • S3 on premise is not easy • RiackCS, leoFS, Swift are hard • S3 compatibility so so…
  10. 10. Minio
  11. 11. Minio • Open Source • Object Storage written in Go • Distributed ? Not really more like replication • S3 compatible • Data protection using Bit rots , Erasure Code • Single go binary easy to start
  12. 12. Minio limitation • Do not scale-out or scale back • Not multi-tenant, user are the same for the same pool • number of nodes =>4 and <= 16 • Still early phase (>5 commit /day on master)
  13. 13. Minio 1 disk by node
  14. 14. Minio Many disk by nodes
  15. 15. Minio and Concourse • Support S3 native resource from concourse Jobs 1 Jobs 2 file_out file_out file_out
  16. 16. Minio bosh release
  17. 17. Minio bosh release • Deploy a pool of distributed minio in minutes • On any IaaS (supported by bosh)
  18. 18. Minio bosh release • Use bosh v2 links • Easy to spin-off • well, you need to learn bosh a little….
  19. 19. Minio 1 disk by node
  20. 20. Demo
  21. 21. Demo • Deploy Minio Cluster • Set Concourse Pipeline
  22. 22. Resources https://github.com/shinji62/minio-dist-boshrelease https://github.com/cloudfoundry-community/minio-boshrelease • Minio bosh release • Minio • https://github.com/minio/minio • https://minio.io/
  23. 23. Resources • ConcourseCI • https://concourse.ci
  24. 24. Slack https://concourseci.slack.com/ #general
  25. 25. Q & A
  26. 26. Thank You !!

×