Fuchsia RFCs
Your SlideShare is downloading.
×
Check these out next
Recommended
More Related Content
More from Pascal-Louis Perez (15)
Recently uploaded (20)
Applied Lean Startup Ideas: Continuous Deployment at kaChing
- Applied Lean Startup Ideas:<br />Continuous Deployment at kaChing<br />&<br />#leanstartup<br />
- Twenty Deployments a Day Keeps the Nasty Bugs Away<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- “Connect investors with outstandinginvestment managers”<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Release is a Marketing ConcernReducing Code Inventory is what Engineering Focuses On<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Lean Startup at kaChing<br />“Human institution designed to create something new under conditions of extreme uncertainty” – Eric Ries<br />Methodical approach to eliminating waste<br />No Quality Assurance engineers, no Operations team<br />Empower everybody to drive their own projects<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- What is Continuous Deployment?<br />Continuous, successful and repeatable methodology to deploying code<br />Automates every steps of taking checked in code and making it run on production servers, in front of customers<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- True Story<br />Investment managers calls, comments about unintuitive trading flow.<br />Improvements are made, and deployed within the next 20 minutes.<br />We call him back “What do you think of the improvement?”<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Benefits of Continuous Deployment<br />It allows quick iterations<br />Obsoletes processes, e.g. “cutting a release”<br />Reduces risk<br />Everyone is aware of production<br />No one throws code over the wall<br />Exposes 24x7 operational requirements<br />Trunk stable<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- ContinuousDeployment<br />Immune System<br />Making Continuous Deployment a Reality<br />Pascal-Louis Perez, kaChing Group Inc.<br />Culture<br />Test Driven Development<br />
- #architecture<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Our architecture<br />Service oriented system<br />Vertical sharding<br />Everything uses the same platform kawala<br />Coordination using ZooKeeper<br />Data interchange using JSON and Protobufs<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Typical Stack<br />Clustered services, multiple instances<br />Replicated databases (e.g. MySQL or NoSQL)<br />Caching (e.g. memcached)<br />Denormalized data (e.g. Voldemort)<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- kawala<br />Command pattern. A command is a query<br />Dynamic Inversion of Control<br />Queries produce a value<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Sample Query<br />Pascal-Louis Perez, kaChing Group Inc.<br />class GetUser(Id<User> userId) { @Inject UserRepository repository; public User process() {<br /> return repository.load(userId); }}<br />
- What We do with Queries<br />Pascal-Louis Perez, kaChing Group Inc.<br />Persist Queries<br />RPC<br />Command Line Tools<br />Serve BLOBS<br />CPS<br />Render Pages<br />Push into MQ<br />
- Lifecycle<br />1. Commit<br />2. Testing<br />3. Deployment<br />4. Production Validation<br />5. Lifetime Monitoring<br />
- Development<br />Trunk stable<br />Small, frequent commits<br />Unreleased features live behind experiments<br />Forward & Backward compatibility<br />Responsive code review<br />Pair difficult problems<br />Trivial rollbacks<br />
- #testing<br />
- Testing Philosophy<br />Only automated testing matters<br />If it isn’t tested, it isn’t finished or correct<br />Write testable code<br />Embrace abstractions<br />Testing is cross functional, we all own quality<br />
- Benefits of TDD<br />It allows quick iterations<br />It empowers engineers to change anything, and as such helps in scaling the team<br />It is more cost effective than debugging<br />It obsoletes the need for functional QA<br />It facilitates continuous refactoring, allowing the code to get better with age<br />It attracts the right kind of engineers<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Types of Testing<br />Unit Testing - does the code work?<br />Integration Testing - does the code work together?<br />Regression Testing - learn from your mistakes<br />Frontend Testing - a whole different ballgame<br />
- Defensive Testing<br />Nightmare scenarios<br />A common conversation at lunch:<br />Alice: What would happen if X blew up?<br />Bob: uh... the site would go down.<br />Fix it, test for it<br />
- Implementation<br />Frameworks: junit, dbunit<br />CI: Hudson<br />Frontend: Selenium<br />
- Production<br />Process, Infrastructure, Deployment, Monitoring<br />
- Process<br />Everyone: writes code and tests, releases to production, adds monitoring<br />Specialists help out when needed<br />New hires push code the first day<br />Problems, issues, errors, bugs, oversights...<br />
- Stability<br />Cluster everything<br />As little global state as possible<br />Maintain global state through ZooKeeper<br />Monitor everything<br />
- ZooKeeper<br />Reliable Distributed Synchronization<br />For:<br />Service Discovery<br />Service Status<br />Coarse locking<br />
- Life of a Deployment<br />Pascal-Louis Perez, kaChing Group Inc.<br />DM<br />PM<br />PM<br />PM<br />ZK<br />
- Life of a Deployment<br />Pascal-Louis Perez, kaChing Group Inc.<br />DM<br />PM<br />PM<br />PM<br />ZK<br />
- Life of a Deployment<br />Pascal-Louis Perez, kaChing Group Inc.<br />DM<br />PM<br />PM<br />ZK<br />
- Life of a Deployment<br />Pascal-Louis Perez, kaChing Group Inc.<br />DM<br />PM<br />PM<br />PM<br />ZK<br />
- Life of a Deployment<br />Pascal-Louis Perez, kaChing Group Inc.<br />DM<br />PM<br />PM<br />PM<br />ZK<br />
- Deployment<br />Exponentially increasing group sizes<br />Increased monitoring during deployment<br />Self Test<br />Automated rollbacks<br />
- Implementation<br />Building: ant<br />Deploying: rpm, yum<br />Monitoring: nagios, jcollectd, RRDtool, jmx<br />Custom automation<br />
- #monitoring<br />
- Monitoring Philosophy<br />Prefer business metrics<br />Monitor statistical deviations not absolute values<br />Automatically annotate graphs<br />
- Monitoring Errors<br />False negatives - errors of omission<br />False positives - errors of implementation<br />
- End to End Monitoring<br />Run Selenium on production<br />Accessibility, Speed<br />Ad-hoc Keynote - customized for our flows<br />Must control data creation, analytics impact<br />
- #future-developments<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Quarantining<br />Isolate new releases<br />Flexible partitioning of requests<br />Gradually shift load to fresh services<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- Describing Infrastructure<br />Many moving parts<br />nagios, collectd, backups, services, databases, …<br />Consistency is key<br />Adding new tools should be easy and thorough<br />Standardize best practices<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- What we Covered Today<br />Lean Startup and Continuous Deployment<br />Anatomy of a Release<br />Commit, Testing, Deployment, Monitoring<br />Future Developments<br />Quarantining, DSL for Infrastructure<br />Pascal-Louis Perez, kaChing Group Inc.<br />
- References<br />We’re Recruiting jobs@kaching.com<br />kaChing’s blog http://eng.kaching.com<br />kawalahttp://bit.ly/kawala<br />Deployment Infrastructure http://eng.kaching.com/2010/05/deployment-infrastructure-for.html<br />Extreme Testing http://bit.ly/9bOFaA<br />Writing Testable Codehttp://googletesting.blogspot.com/2008/08/by-miko-hevery-so-you-decided-to.html<br />ZooKeeperhttp://bit.ly/kc-zookeeper<br />Eric Rieshttp://startuplessonslearned.com<br />Pascal-Louis Perez, kaChing Group Inc.<br />
