Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Stating the obvious
Giulio Vian
DevOps Lead
Glass Lewis Europe Ltd
@giulio_vian
Agenda
Why obvious?
Integration with Continuous Integration /
Continuous Deployment
Performance / load/ stress testing bas...
This Session
100/200 level
1.Performance is valuable
2.It’s easy to integrate Perf/load test in CI/CD
3.Reading the results is art
Three Takeaways
About me
giulio.dev@casavian.eu
@giulio_vian
http://blog.casavian.eu/
Common scenarios
Desktop / mobile / library Server / web
Database
server
Web serverClient
N1
N4
A1
A3 A2N3
N2
Common scenarios
Desktop / mobile / library Server / web
Database
server
Web serverClient
N1
N4
A1
A3 A2N3
N2
I. Why obvious?
Performance is
a Feature
«there are two kinds of
websites: the quick
and the dead»
https://blog.codinghorror.com/performan...
Response time
0.1 second is about the limit for having the user feel
that the system is reacting instantaneously, meaning
...
Hard facts
Google
Half a second delay
caused a 20% drop in
traffic.
Amazon.com
Even very small delays
(100 ms) would resul...
Why lubricant in
an engine?
Do you like the motor
to seize?
Always necessary?
Gartner Hype Cycle,
Source: Wikipedia
My perf test
of the year
2M users
40,000 RPS
2Gbps
Our tests around 1,000
There’s Nothing
Like Production
© 2016 IMG Unive...
II. CI / CD Integration
Tooling
Scripts & Runners
Load generators
Monitoring
Tools: Script & Runners
JMeter
Visual Studio (.webtest / .loadtest)
Selenium
×Unit family
Gatling
*Custom & More…
Tools: Load
VSTS Cloud-based load testing
CA BlazeMeter
SOASTA (Akamai) CloudTest
HPE LoadRunner
*Custom
& More…
Tools:
Monitoring
Loading tool
Basic
Performance Counters*
Application logs
Cloud
& More…
Analyzing
Build an aggregated
view
Find interesting events
High level architecture
You
Load
Generator
SUT
Monitoring
CI / CD
pipeline
Web Load Testing
Using Visual Studio, Visual Studio Team Services and Azure
Considerations
for CI/CD
Trunk or branch of
pipeline
Resources
KPIs
Time factor
Short-running Tests
Long-running Tests
Test Coverage
Frequency of execution
Value from test
Hidden
Gremlins
Cloud infrastructure
warm up
e.g. AWS ELB
Default configuration
nginx worker processes
Client resources
III. Perf / load/ stress testing
basics
(Technical) Goals
Benchmarking
Force defects to emerge
code, configuration, architecture
Capacity planning
Breaking point
...
Emerging Problems
Connection pool exhaustion
File locks
Database locks
(Thread) deadlocks
Memory exahustion
Domino Effect
...
Indicators
Latency
Throughput
Load
Error rate
Tool (client) errors
Compound
graph
+Error rates
+Tools errors
Relative Load
Latency
Required threshold
Max N seconds
100%60%
Throughput
Usage
How you read
this?
Relative Load
Latency
Required threshold
Max N seconds
100%60%
Throughput
Usage
How you read
this?
Resource Exhaustion!
Relative Load
Latency
Required threshold
Max N seconds
100%60%
Throughput
Usage
Analyze
Response curve
Inflection point
Bottlenecks
What to monitor
CPU
RAM
Disk I/O
Network I/O
Threads / processes
IV. Closing
Resources
Sisyphus
Choosing to invest
(Photo: Elya)
Bibliografy & References
http://www.slideshare.net/giuliov/presentations
https://github.com/giuliov/Stating-the-obvious/
h...
Bibliografy (2)
Writing High-Performance .NET Code
— Ben Watson (Ben Watson)
https://www.amazon.it/Writing-High-
Performan...
Bibliografy (3)
Software Performance and Scalability: A
Quantitative Approach — Henry H. Liu
(Wiley)
https://www.amazon.co...
Bibliografy (4)
Continuous Delivery: Reliable Software
Releases through Build, Test, and
Deployment Automation — J.Humble,...
1.Performance is valuable for business
2.It’s easy to integrate Perf/load test in CI/CD
3.Reading the results is art
Three...
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle
Session Title
Your Name
Your Title
Your Company
Your @TwitterHandle
bit.ly/addo-slack
Find me on slack, right now!
Upcoming SlideShare
Loading in …5
×

Stating the obvious - All Day DevOps 2017

Stating the obvious - All Day DevOps 2017
https://www.alldaydevops.com/

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Stating the obvious - All Day DevOps 2017

  1. 1. Stating the obvious Giulio Vian DevOps Lead Glass Lewis Europe Ltd @giulio_vian
  2. 2. Agenda Why obvious? Integration with Continuous Integration / Continuous Deployment Performance / load/ stress testing basics Closing
  3. 3. This Session 100/200 level
  4. 4. 1.Performance is valuable 2.It’s easy to integrate Perf/load test in CI/CD 3.Reading the results is art Three Takeaways
  5. 5. About me giulio.dev@casavian.eu @giulio_vian http://blog.casavian.eu/
  6. 6. Common scenarios Desktop / mobile / library Server / web Database server Web serverClient N1 N4 A1 A3 A2N3 N2
  7. 7. Common scenarios Desktop / mobile / library Server / web Database server Web serverClient N1 N4 A1 A3 A2N3 N2
  8. 8. I. Why obvious?
  9. 9. Performance is a Feature «there are two kinds of websites: the quick and the dead» https://blog.codinghorror.com/performance-is-a-feature/
  10. 10. Response time 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. 1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. 10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. Miller, R. B. (1968). Response time in man- computer conversational transactions. Proc. AFIPS Fall Joint Computer Conference Vol. 33, 267-277.
  11. 11. Hard facts Google Half a second delay caused a 20% drop in traffic. Amazon.com Even very small delays (100 ms) would result in substantial and costly drops in revenue.
  12. 12. Why lubricant in an engine? Do you like the motor to seize?
  13. 13. Always necessary? Gartner Hype Cycle, Source: Wikipedia
  14. 14. My perf test of the year 2M users 40,000 RPS 2Gbps Our tests around 1,000 There’s Nothing Like Production © 2016 IMG Universe, LLC. All Rights Reserved
  15. 15. II. CI / CD Integration
  16. 16. Tooling Scripts & Runners Load generators Monitoring
  17. 17. Tools: Script & Runners JMeter Visual Studio (.webtest / .loadtest) Selenium ×Unit family Gatling *Custom & More…
  18. 18. Tools: Load VSTS Cloud-based load testing CA BlazeMeter SOASTA (Akamai) CloudTest HPE LoadRunner *Custom & More…
  19. 19. Tools: Monitoring Loading tool Basic Performance Counters* Application logs Cloud & More…
  20. 20. Analyzing Build an aggregated view Find interesting events
  21. 21. High level architecture You Load Generator SUT Monitoring CI / CD pipeline
  22. 22. Web Load Testing Using Visual Studio, Visual Studio Team Services and Azure
  23. 23. Considerations for CI/CD Trunk or branch of pipeline Resources KPIs
  24. 24. Time factor Short-running Tests Long-running Tests Test Coverage Frequency of execution Value from test
  25. 25. Hidden Gremlins Cloud infrastructure warm up e.g. AWS ELB Default configuration nginx worker processes Client resources
  26. 26. III. Perf / load/ stress testing basics
  27. 27. (Technical) Goals Benchmarking Force defects to emerge code, configuration, architecture Capacity planning Breaking point Security leakage
  28. 28. Emerging Problems Connection pool exhaustion File locks Database locks (Thread) deadlocks Memory exahustion Domino Effect Lack of scalability
  29. 29. Indicators Latency Throughput Load Error rate Tool (client) errors
  30. 30. Compound graph +Error rates +Tools errors Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
  31. 31. How you read this? Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
  32. 32. How you read this? Resource Exhaustion! Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
  33. 33. Analyze Response curve Inflection point Bottlenecks
  34. 34. What to monitor CPU RAM Disk I/O Network I/O Threads / processes
  35. 35. IV. Closing
  36. 36. Resources Sisyphus Choosing to invest (Photo: Elya)
  37. 37. Bibliografy & References http://www.slideshare.net/giuliov/presentations https://github.com/giuliov/Stating-the-obvious/ https://docs.microsoft.com/en-us/vsts/build- release/tasks/test/cloud-based-load-test AWS Well-Architected Framework - Performance Efficiency Pillar https://www.amazon.com/dp/B01MSSLHBX Performance Testing Guidance for Web Applications https://msdn.microsoft.com/en-us/library/bb924375.aspx http://www.brendangregg.com/linuxperf.html
  38. 38. Bibliografy (2) Writing High-Performance .NET Code — Ben Watson (Ben Watson) https://www.amazon.it/Writing-High- Performance-NET-Code- Watson/dp/0990583430/ Time Is Money: The Business Value of Web Performance — Tammy Everts (O'Reilly Media) https://www.amazon.com/Time-Money- Business-Value- Performance/dp/1491928743
  39. 39. Bibliografy (3) Software Performance and Scalability: A Quantitative Approach — Henry H. Liu (Wiley) https://www.amazon.com/Software- Performance-Scalability-Quantitative- Approach/dp/0470462531 Continuous Delivery with Windows and .NET — Matthew Skelton and Chris O'Dell (O'Reilly) http://www.oreilly.com/webops- perf/free/continuous-delivery-with-windows- and-net.csp
  40. 40. Bibliografy (4) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation — J.Humble, D.Farley (Addison-Wesley) https://www.amazon.com/Continuous- Delivery/dp/0321601912/ The DevOps Handbook — G.Kim, P.Debois, J.Willis, J.Humble (IT Revolution Press) https://www.amazon.com/DevOps-Handbook- World-Class-Reliability- Organizations/dp/1942788002/
  41. 41. 1.Performance is valuable for business 2.It’s easy to integrate Perf/load test in CI/CD 3.Reading the results is art Three Takeaways
  42. 42. Session Title Your Name Your Title Your Company Your @TwitterHandle
  43. 43. Session Title Your Name Your Title Your Company Your @TwitterHandle
  44. 44. Session Title Your Name Your Title Your Company Your @TwitterHandle
  45. 45. bit.ly/addo-slack Find me on slack, right now!

×