Deployment Driven Development and Performance Testing TEFCON2015
•Download as PPTX, PDF•
1 like•911 views
Talk Given in TEFCON2015 about how does deployment driven development integrates with performance testing.
Recommended
More Related Content
Similar to Deployment Driven Development and Performance Testing TEFCON2015
Similar to Deployment Driven Development and Performance Testing TEFCON2015 (20)
More from Almudena Vivanco
More from Almudena Vivanco (17)
Recently uploaded
Recently uploaded (20)
Deployment Driven Development and Performance Testing TEFCON2015
- 5. How long would it take to your organization to deploy a change that involves just one single line of code?
- 6. import random import time import requests class Speaker(object): def __init__(self): born = "Avilés - Asturias" studies = "Applied Maths and Computability" jobdescription = "Performance Jedi" company = "Telefónica I+D" team = "Product Engineering" project = "AWAZZA" talks = [VLC Testing, DevopsDays, WebPerfDays, Velocity] def talk(self): start_timer = time.time() r = requests.get('http://www.slideshare.net/almudenavivanco') r.raw.read() latency = time.time() - start_timer self.custom_timers['TEFCon 2015 - DDD'] = latency if __name__ == '__main__': speech = Speaker() speech.talk() print trans.custom_timers Almudena Vivanco (@MrsDaehin)
- 8. Performance It's not just about adding more servers
- 12. Continuous … wait … what?
- 13. Fear is the path to the dark side. Fear leads to Inertia. Inertia Leads to paralysis. Paralysis leads to going out of Business. Do not fear the Release!
- 14. Be ready for Failing!!
- 16. It’s not my machines! It’s your code
- 17. It’s not my code! It’s your machines!
- 21. How do you test performance? Different apps have different profiles Tests need to scale Tests should be shareable Tests should be straight forward.
- 23. THINK BIG
- 24. Dark Launching
- 26. Feature Toggling
- 28. A/B Testing
- 30. Canary Releasing
- 32. Do you still want to use Blue Green deployment?
Editor's Notes
- 15 years ago, seventeen people met to talk, ski, relax, and try to find common ground. What emerged was the Agile �Software Development� Manifesto. From that day on, our way of working has changed several times swinging between waterfall to agile back and forth trying to find a balance. Most of our teams are working now in Agile but many times it is only encapsulated to DEVS/QA or only to OPS. Finding it difficult to cover all the product life cycle with all the actors involved in agile methodology.
- We have been talking about continuous delivery for a while, we want our products to arrive fast to our customers and fullfill their needs. We have created deployment pipelines, automating the release, automating the infrastructure. But many times we are still fighting against our fear to release. We’ve found that the costs we incur–typically bugs or unpolished but functional features–are worthwhile in the name of getting feedback from our customers as quickly as possible about our product innovations. The sooner we have feedback from our customers, the sooner we know whether we guessed right about our product decisions and the sooner we can choose to either change course or double down on a winning idea.
- Integration of Performance Testing in Continuous delivery A continuous integration server to run and monitor our automated tests with every code commit cluster immune system to monitor and alert on statistically significant regressions, and automatically revert the commit if an error occurs) A source control commit check to stop new code being added to the codebase when something is broke Real-time monitoring and alerting to inform the team immediately when a bug makes in through our deployment process Root cause analysis (Five Whys) to drive incremental improvements in our deployment and product development processes ( HOSHIN ) Recognize this? single Line of change takes a week to deploy Slow feedback Code Freeze? what is that? Sleeples nights during deployments
- about me
- why Scrum teams usually require my skills? well usually when there is a question in the air… HOW MANY USERS can we handle? or how do the service scale? but these questions could be answered easily with some planification during development.
- We all know that performance is not just throwing money at more servers virtual or physical, but we are terrify by failing what are the problems we usually face during our last phase? To answer the question of how to scale the service we usually face the problems that we don’t have a server similar to production. we don’t know the behaviour of the users, we don’t have a system with the volume of production. we cannot have all services 3rd parties integrated and running at the same time.
- You must unlearn,what you have learned, you need to leave your confort zone, with no change there is no fail, but with no fail there is no way to improve.
- How many of us still deploy with a blue green? Pattern Deploy software to a non-production environment (call it blue) while production continues to run. Once it’s deployed and “warmed up”, switch production (green) to non-production and blue to green simultaneously. Anti-patterns Production is taken down while the new release is applied to production instance(s).
- So let me tell you a joke a really bad one but i bet some of us will feel represented by that. Borat the devop says!!!
- lets go back again we started talking about continuous deployment what benefits can we have? automate provisioning and deployment ensure devs, testers and ops collaborate throughout → devops incrementalism decoupling deployment and release Don’t be afraid of failing, inmune systems→ zero access policy ( phoenix or inmutable servers) Monitoring graphite, munin, nagios Anomaly Detection!!! Infraestructure as Code: Automate or Die! Ansible, Chef, Puppet… Containers: Docker, Joyent Triton
- don’t be afraid of failing we will fail! we are human but what can we do to minimize the impact of our failing? reducing release risk we need to build our Contingency plan to know how to react Significant Alarms Anomaly Build Environments mean time to restore service instead of mean time between failures
- Developers need to talk to ops about the impact of the code: What metrics will change and how? what are the risks? what are the signs that something is going wrong? what are the contingencies? Ops’ job is NOT to keep the site stable and fast Ops’ job is to enable the business (this is dev’s job too) The business requires change But change is the root cause of most outages! Discourage change in the interests of stability or Allow change to happen as often as it needs to
- Ops need to talk to devs what has changed? what features entered? how do we deploy the software where do you expect
- MTRS: mean time to restore service MTBF: mean time between failures Simian Army
- Don’t look at me after BDD TDD and the rest of xDD we can add one more acronyms to our salad. DDD is the tecnique that will help our collegues from Delivery to be sure about the products we create and develop. This tecniques will help to deploy ina more agile way
- real user monitoring because not always the users are what we expect and we need to give service to everyone of our target we need to be ready to find any anomaly detection and not only problems we need to check the server health Logs alarms significant and not raised all kind of alarms
- and how does this benefit our performance tests?
- why not testing a real environment real servers real traffic real users real workflow
- https://www.facebook.com/note.php?note_id=96390263919 "Dark Launching": During the two weeks prior to launch we began what we call a "dark launch" of all the functionality on the backend. Essentially a subset of user queries are routed to help us test, by making "silent" queries to the code that, on launch night, will have to absorb the traffic. Levers and the "Nuclear Option": We couldn't be sure of the exact level of traffic the launch would generate so we put together contingency plans to decrease the load on various parts of the site. This gave us overhead in core functionality at the expense of less essential services. Pattern Launch a new application or features when it affects the least amount of users. Anti-patterns Software is deployed regardless of number of active users.
- config file with the status of the feature disabled or enabled in this case there is just one front end that got the logic of the feature depending on its status RISK: huge chance of having a huge config file with obsolete features Diassociat deployment and feature activation Helps you to have a production ready code all the time Gives you the flexibility of trying out multiple features QA can test individual features or group of features Business can get fast feedback through feature previewing. we can do progressing activation of a feature → Canary Testing Measure impacts of a new version ( A/B testing) Diffeer feature activation on production Trunk Base Development Python - Waffle Golang - Flash Node.js - feature-flags C# Feature Switcher
- 2 or more versions of the same feature enabled most of the times without a segmentation of the users so we can try randomly which of the features is the best.
- An A/B test involves testing two versions of a web page — an A version (the control) and a B version (the variation) — with live traffic and measuring the effect each version has on your conversion rate. Start an A/B test by identifying a goal for your company then determine which pages on your site contribute to the successful completion of that goal.
- Pattern Release software to production for a small subset of users (e.g. , 10%) to get feedback prior to a complete rollout. Anti-patterns Software is released to all users at once.
- es to use a strategy with multiple canaries, the first one being visible only to their internal employees and having all theFeatureToggles turned on so they can detect problems with new features early. Canary releases can be used as a way to implement A/B testing due to similarities in the technical implementation. However, it is preferable to avoid conflating these two concerns: while canary releases are a good way to detect problems and regressions, A/B testing is a way to test a hypothesis using variant implementations. If you monitor business metrics to detect regressions with a canary [2], also using it for A/B testing could interfere with the results.
- How many of us still deploy with a blue green?