Your SlideShare is downloading. ×
0
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Apache Hadoop India Summit 2011 talk "Profiling Application Performance" by Umesh Bellur

1,697

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,697
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
43
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Profiling Application Performance <br />and Resource Utilization <br />in Virtualized Environments<br />UmeshBellur<br />PuruKulkarni<br />http://www.cse.iitb.ac.in/synerg<br />Systems and Networks Research Group<br />Department of Computer Science and Engineering<br />Indian Institute of Technology Bombay<br />
  • 2. Feb 16th, 2011<br />2<br />Virtualization based Provisioning<br /><ul><li>Abstract physical resources as virtual objects
  • 3. Virtual machines execute on virtual systems
  • 4. Virtual machine monitor multiplexes physical resources across virtual resources</li></ul>Benefits<br />Direct: Rapid provisioning, Interoperability, Performance Isolation, Increased resource utilization<br />Derived: Server Consolidation, Flexible resource provisioning, Efficient replication, backup and migration<br />
  • 5. Feb 16th, 2011<br />3<br />Virtualisation-based Provisioning Issues<br />Flexible Resource Provisioning<br /> SLA-based migration<br /> Power-aware consolidation<br /> Cloud bursting<br />Physical to Virtual <br /> Resource requirements mapping and modelling<br /> Capacity planning<br /> VM placement policies<br />Deployment & Manageability <br /> Resource accounting in Clouds<br /> Prototype Testbed setups<br />VM Optimisations<br /> Shared memory based I/O for co-located VMs<br /> Network I/O guarantees <br />VMs for resource-constrained platforms<br />
  • 6. Feb 16th, 2011<br />4<br />Provisioning & Capacity Planning in Virtual Environments<br />Some questions of interest<br /><ul><li>Application behaviour and resource usage under different workloads?
  • 7. Throughput, response time, network, CPU, memory ...
  • 8. With finite resources which SLAs can be satisfied?
  • 9. How will the application behave with varying resources?
  • 10. What is the impact of VM movement on application performance and resource usage levels?</li></li></ul><li>Dom0<br />DomU<br />Xen Hypervisor<br />Hardware<br />Why separate VM-specific tool?<br /><ul><li>Existing tools not virtualisation aware
  • 11. All I/O through privileged VM
  • 12. Impact of VM migration
  • 13. Workload generation and </li></ul> app. performance separate <br /> from resource monitoring<br />5<br />01/12/10<br />
  • 14. Feb 16th, 2011<br />6<br />State of the art for virtual environments<br /><ul><li>Resource monitoring tools
  • 15. xenmon, xentop, sysstat, Hyperic etc.
  • 16. Load generation benchmarks
  • 17. SPEC, TPCW ...
  • 18. Isolated from profiling framework
  • 19. Load generation and control
  • 20. Resource configuration control
  • 21. User-level performance metrics</li></ul>Require an integrated tool (for VM worlds)<br /><ul><li>Load generation + Resource configuration + Measurement and monitoring</li></li></ul><li>Feb 16th, 2011<br />7<br />Problem Definition<br /><ul><li>Profile performance and resource utilization levels of applications in virtual environments
  • 22. Multi-tier, multi-VM based (web-based) application
  • 23. Implications on
  • 24. Capacity planning
  • 25. Migration and Consolidation Strategies
  • 26. virtPerf*
  • 27. Application profiling tool for virtualized environments
  • 28. Built on top of AutoPerf** – a similar tool for physical environments
  • 29. Current scope: Xen and kvm</li></ul>* The tool forms the thesis of a current MTech student - PrajaktaPatil. <br />** S. Shirodkar, “Autoperf: An automated load generator and performance measurement tool for multi-tier software systems,”, Master’s thesis, IIT Bombay, Mumbai, India, June 2007 <br />
  • 30. Feb 16th, 2011<br />8<br />virtPerf slave<br />virtPerf master<br />profiling<br />agent<br />requests<br />DomU<br />Load<br />generator<br />responses<br />DomU<br />Physical machine 1<br />Controller<br />DomU<br />Physical machine 2<br /> Dom0<br />virtPerf Architecture<br /><ul><li>Workload generator + Measurement tool</li></li></ul><li>Feb 16th, 2011<br />9<br />DomU<br /> Dom0<br />Load Generator and VM Initialisation<br /> - Multi-client load <br /> - Warm-up detection<br />requests<br />Load<br />generator<br />responses<br />load<br />level<br />ready<br />Initialize<br />Controller<br />Load characteristics<br />Transaction details<br /> - Set resource configurations<br /> - IP to VM id mapping<br /><ul><li>Warmup => Stable response time
  • 31. Thread startup, caching etc.
  • 32. Agents initialize VMs according to resource profile</li></li></ul><li>Feb 16th, 2011<br />10<br />DomU<br />Dom0<br />Run Termination and Measurements<br />requests<br />Load<br />generator<br />responses<br />DONE<br />STOP<br />profiling<br />Controller<br />Resource usage summary<br />Load characteristics<br />Transaction details<br /><network, disk, cpu, <br />memory, service demand><br />...<br /> - Collate performance metrics<br /> - Select next load level or VM resource configuration<br /><ul><li>xentop at Host (Dom0)
  • 33. sysstat and ps at Guest (DomU)</li></li></ul><li>Feb 16th, 2011<br />MSD of a Profiling Run<br />
  • 34. Feb 16th, 2011<br />12<br />Metrics Collected<br /><ul><li>Resource usage metrics (averaged over duration of run)
  • 35. CPU utilization
  • 36. Network & disk I/O
  • 37. Memory usage
  • 38. Performance metrics
  • 39. Average response time
  • 40. Throughput
  • 41. Service demand</li></li></ul><li>Feb 16th, 2011<br />13<br />virtPerf Features<br /><ul><li>Load control
  • 42. #clients, think time distributions, total number of requests issued
  • 43. AutoPerf provides automatic load level section, warmup detection
  • 44. VM resource configuration
  • 45. #cores, CPU allocation (min, max), CPU pinning
  • 46. To both Dom0 and each of the DomUs individually
  • 47. Impact of Migration
  • 48. Resource requirements, Performance hit (drop in throughput, increase in response time)
  • 49. VM Collocation
  • 50. Resource contention impact & VM Collocation benefits</li></li></ul><li>Feb 16th, 2011<br />virtPerf Validation<br /><ul><li>Metrics reported by virtPerf for resource consumptions and service performance known apriori.
  • 51. CPU Utilization:
  • 52. A service that can consume a specific amount of CPU with a knob to vary this from 10-90%.
  • 53. virtPerf reports utilization within 3% of this at any level.
  • 54. Network I/O:
  • 55. A service will read/write a known amount of I/O at a specified rate.
  • 56. virtPerf reports back this number with a 400 byte overhead per request caused by HTTP and the virtPerf control overhead
  • 57. Disk I/O:
  • 58. Similar to Network I/O and results are within 2% of expected values</li></li></ul><li>Feb 16th, 2011<br />15<br />Experimental Setup<br /><ul><li>PMs: Quad Core Intel i5 Intel(R) Core(TM) i5 CPU, 760 @ 2.80GHz 3 GB RAM
  • 59. Virtual platforms: Xen</li></ul>WebCalendar & RUBiS - 2 tier applications with a MySQL backend and a HTTP-based front-end <br />
  • 60. Feb 16th, 2011<br />16<br />Throughput Saturation<br /><ul><li>webcal saturates at 64 req/s
  • 61. virtPerf can automatically detect saturation
  • 62. CPU utilization of VM (DomU) ~400% (4 core m/c)</li></li></ul><li>Feb 16th, 2011<br />17<br />Close to service demand =>high arrival rate<br /> Throughput saturates with ~12 users<br />Think time >> service demand =>Low arrival rate<br /> Throughput saturates with~72 users<br />Sensitivity to Think time distribution<br />Think time distribution: Poisson<br />Mean: <80ms, 150 ms, 800 ms><br />
  • 63. Feb 16th, 2011<br />18<br />1 core vs 4 cores<br />Maximum Throughput 16 req/s : 65 req/s<br />Response time 900 ms : 3000 ms<br /> Performance metrics ratio 1 : 4 <br />VM CPU Utilization<br />~80 % : 375 %<br />VM Configuration (#cores)<br />
  • 64. Feb 16th, 2011<br />19<br /><ul><li>Throughput
  • 65. c1 >> c2 >> c3
  • 66. Response time
  • 67. c1 << c2 << c3</li></ul>VM Configuration (placement)<br /><ul><li> 3 configurations
  • 68. Single VM app (c1)
  • 69. 2 VMs on 1 PM (c2)
  • 70. 2 VMs on 2 PMs(c3)</li></li></ul><li>Feb 16th, 2011<br />20<br />Response time approximately<br />half with 300% CPU<br />Achieved Throughput doubles<br />(25 req/s to 50 req/s)<br />WebCalendar<br />VM Resource Configuration (CPU availability)<br /><ul><li>CPU for DomU (guest VM): 150% and 300%</li></li></ul><li>Feb 16th, 2011<br />VM Resource Configuration for RUBiS<br /><ul><li> Impact of increasing CPU for the Web Tier felt ONLY AFTER DB</li></ul> Tier has sufficient CPU (260%). Else DB Tier is the bottleneck<br /><ul><li> Throughput goes from 90 to 156 requests/sec at the DB tier </li></ul> receiving upto 340% and the web tier going from 10 to 30%<br />
  • 71. Feb 16th, 2011<br />VM Migration while under Load<br />
  • 72. Feb 16th, 2011<br />Effects of Migration<br />
  • 73. Feb 16th, 2011<br />Core Pinning<br />Peak throughput almost doubles when the DB<br />Tier is given it’s own core compared to when<br />all the cores are shared amongst the VMs – Dom0<br />and 2 app VMs sharing 2 cores.<br /><ul><li> VirtPerf can pin VMs on specific</li></ul> cores of a machine.<br /><ul><li> Makes a difference to application</li></ul> performance.<br />
  • 74. Feb 16th, 2011<br />25<br />virtPerf Summary<br /><ul><li>Workload generator + Application profiler
  • 75. Feature set (profiling with ...)
  • 76. Load distribution control
  • 77. VM resource configuration and control
  • 78. Performance and resource metrics with migration
  • 79. Profiling with VM interference (to do)
  • 80. Demonstration of virtperf capabilities and correctness via lab set up with a couple of multi tier applications</li></li></ul><li>Feb 16th, 2011<br />26<br />Future Work<br /><ul><li>virtperf features addition
  • 81. Session, probabilistic request workloads
  • 82. Profiling with interfering VMs
  • 83. Evaluation and testing with kvm
  • 84. Experimentation
  • 85. Testing at scale
  • 86. Black-box vs. Gray-box profiling - Remove profiling within VM
  • 87. Bottleneck identification with multi-tier applications
  • 88. Open-source release</li></ul>http://www.cse.iitb.ac.in/synerg<br />
  • 89. Feb 16th, 2011<br />27<br />Other Virtualisation Related Projects@synerg<br /><ul><li>Flexible resource provisioning
  • 90. Affinity-aware resource profiling and prediction
  • 91. Benchmarking, estimating and tuning cost of migration
  • 92. p2v placement tool
  • 93. Design and evaluation of virtualisation-aware placement techniques
  • 94. vSim
  • 95. A simulator for virtualisation-based systems
  • 96. Network I/O optimisations for collocated VMs</li></li></ul><li>Feb 16th, 2011<br />Questions?umesh, puru@cse.iitb.ac.in<br />Acknowledgements:<br />Prof. VarshaApte who seeded the idea with AutoPerf<br />Past MTech students ShrirangShirodkar, VaibhavSelot<br />

×