Going public in 2014 presented Zalando's engineering team with some interesting new challenges, especially in the realms of auditing and compliance. To ensure that we remain in good standing with both our auditors and our customers, Zalando's cloud engineering team built STUPS: an open source set of tools and components that provides a convenient and audit-compliant Platform-as-a-Service (PaaS) for multiple autonomous teams on top of AWS. For those of us at Zalando who work with Cassandra, the creation and implementation of STUPS has added another set of challenges: How to orchestrate Cassandra in such an environment?
Zalando's teams use Cassandra because of its incredible performance, linear scalability and multi-datacenter support. In a dynamic cloud environment, you can't just bootstrap a Cassandra cluster in the usual way. In this talk, I will show you how my team and I got the job done--not only while working with AWS, but while facing the auditing and compliance standards that our STUPS PaaS aims to address. I'll describe our solution to orchestrate Cassandra on AWS's EC2 platform using STUPS, Docker, and etcd. I'll also summarize how we've been able to extend our Cassandra clusters from our datacenters to AWS without any downtime. By the end, we hope you'll have some useful takeaways for deploying and managing Cassandra on AWS if you have similar requirements.
2. Agenda
Who We Are and What We Do
How Do We Run a Tech Company
Cloud Infrastructure for Autonomous Teams
Cassandra in a Dynamic Cloud Environment
3. 15 countries
3 fulfilment centers
15+ million active customers
2.2+ billion € revenue 2014
130+ million visits per month
8.000+ employees
ONE OF EUROPE’S LARGEST ONLINE FASHION RETAILERS
Visit us: tech.zalando.com
Tech hubs in Berlin, Dublin, Dortmund
and Helsinki
12. #CassandraSummit
What’s That?
Radical Agility is Zalando Tech's approach
to running a technology organization -
an approach based on the three pillars of
autonomy, mastery and purpose
http://zln.do/ra-video
15. #CassandraSummit
What Is It?
“The STUPS platform is a set of tools
and components to provide a convenient
and audit-compliant Platform-as-a-Service
(PaaS) for multiple autonomous teams
on top of Amazon Web Services (AWS)”
Learn more at https://stups.io
31. #CassandraSummit
Custom SeedProvider
We created the EtcdSeedProvider
seed_provider:
- class_name: org.zalando.cassandra.locator.EtcdSeedProvider
parameters:
- url: http://example.org/v2/keys/cassandra/my-cluster-name/seeds
Dynamically updates list of seed nodes
from etcd
33. #CassandraSummit
Senza Template
We use the STUPS senza tool
to instrument AWS’ CloudFormation
Bootstraps a Cassandra Cluster
with one simple command
$ senza create https://goo.gl/qyTy7b
my-new-cluster
etcd.example.org
46. #CassandraSummit
We Want to Hear From You
https://github.com/zalando/stups-cassandra
https://github.com/zalando/stups-opscenter
https://github.com/zalando/cassandra-etcd-seed-provider
Try it out
Help us find issues
Send us pull requests
47. Thank you for listening
Check out our blog https://tech.zalando.com
Our many open source products https://github.com/zalando
The STUPS stack https://stups.io
Got more questions? You can reach us on twitter
@ZalandoTech
We’re hiring!
Special thanks to Jessie Dude.
No Continuum Transfunctioners were harmed during the production of these slides.