Реалистичное нагрузочное тестирование
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Реалистичное нагрузочное тестирование

on

  • 1,392 views

Доклад Валерия Куваева на конференции SQA Days-12, 30 ноября-1 декабря, Минск

Доклад Валерия Куваева на конференции SQA Days-12, 30 ноября-1 декабря, Минск

Statistics

Views

Total Views
1,392
Views on SlideShare
1,214
Embed Views
178

Actions

Likes
0
Downloads
34
Comments
0

3 Embeds 178

http://www.sqadays.com 101
http://sqadays.com 75
http://sqadays.byte-force.yar.ru 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • This slide gives an overview of the key areas (core Application Lifecycle) that need to be addressedThe integrations and traceability between this key areas is key to ensuring management, visibility and quality into the application lifecycle.Addressing the key personas involved in the application lifecycle on one single, collaborative solution:Business AnalystsProject TeamsDevelopment TeamsQA Teams
  • This slide gives an overview of the key areas (core Application Lifecycle) that need to be addressedThe integrations and traceability between this key areas is key to ensuring management, visibility and quality into the application lifecycle.Addressing the key personas involved in the application lifecycle on one single, collaborative solution:Business AnalystsProject TeamsDevelopment TeamsQA Teams
  • Тестовое покрытие не полноRUMскриптыПрофиль нагрузочного теста не похож на продуктивныйRUMтранзакцииЗаглушки «нереально» производительныSVВоздействие слабых сетей не изученоShunra
  • HP Service Virtualization is software for simulating functional and performance behavior of otherwise unavailable application components.Instead of applications hitting these systems, SV allows IT Dev and Test to eliminate dependencies on unavailable or inaccessible components and services by creating virtual versions for testing. It allows dev teams to speed up application development as it can simulate service components during the build phaseAnd it allows teams to use SV to begin to learn and simulate application behavior and begin testing against these simulated virtual services earlier, find problems earlier, and save orgs time and money.
  • Todd to JohnThis is a screenshot from LoadRunner with a test scenario running.The scenario has 500 VUsers running as local (way most customers are falsely testing today) and then begins to introduce 50 emulated mobile VUsers.From the results we can see that with only 9 mobile VUsers (or 1.8% of the overall VUser population) transaction response time (TRT) goes from ~1 second to ~5 seconds for ALL VUsers.This illustrates the significant impact that mobile users have on performance (and Functionality), both from an ‘end user’ and ‘backend infrastructure’ perspective.Without Shunra, this would have gone un-notice until production, where it very easily could have taken down the production site / app for ALL users.
  • John
  • John to ToddIMPORTANT: ~70% of the end-to-end performance of a mobile app is a result of the network. As a result, without using Shunra, you will get ‘false positives’ resulting in significant risk of failure in production. This risk is both ‘Front End’ and ‘Back End’ and will impact ALL of your users.Mobile applications and the mobile network create unique performance challenges on several levels:First – The performance of a mobile device will dramatically vary depending on the quality of it’s network signal. In most cases, on a mobile network, the majority of transaction time will be consumed in the mobile network and will not be on the device or back end servers. Second – Mobile devices and mobile connections require more back end resources to support them. They open connections to servers and take much longer to complete transactions. The addition of a few mobile clients to an existing system can have a dramatic impact on the performance of the whole system, not just the mobile users.In order to minimize the risks associated with mobile performance, HP’s industry leading LoadRunner and Performance Center tools are able to record and execute mobile performance tests. Supporting both browser applications and native applications, customers are able to script and test their mobile applications. The network is a very real challenge, and ensuring that changing and realistic network conditions are incorporated in the testing process is where our partner Shunra enters the equation. With their integrated Shunra for HP product, load tests can support dynamic and realistic network conditions to ensure that the performance test is accurate and reliable.
  • Information about window’s size, position, docking, state (opened or hidden) is saved under the current layout name
  • Introduces a new way to get a visual context of the script while reviewing its code. It provides a new way to navigate to a location in the code using a visual representation of the business process.
  • Key FeaturesCommon look and feel of all the snapshots within the Snapshot View.Split view synchronization.Brand new snapshot content displayed for most step types (i.e. Xml Viewer, Buffer Viewer, Image Viewer, revised HTTP Viewer, etc…).
  • Key FeaturesSearch per Script or ActionStep context menuThumbnail viewSync with other panes
  • Major enhancements:Code completion list Syntax highlight for the C codeHighlighting of the Step parametersCollapsing of user-defined functionsTooltips display information for user-defined functions, parameters, etc.
  • Breakpoint pane - displays the list of breakpoints in all opened actionsCallback - contains the list of stack frames according to the current debugger source positionWatch - manages user defined watch expressions
  • APIsLoadRunner 11.50:web_reg_save_param_ex()Enhanced web_reg_save_param()Smart scoping for formatted data (DFE)web_reg_save_param_XPath()Correlate by XPathweb_reg_save_param_regexp()Correlate by Regular expression (Perl syntax standards)Record based correlationsReplay based correlationsCorrelation rulesCorrelation from snapshot

Реалистичное нагрузочное тестирование Presentation Transcript

  • 1. РеалистичноенагрузочноетестированиеВалерий КуваевТехнический консультант по ALM,HP Software© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 2. Управление жизненным циклом приложений Управление жизненным циклом приложений Управление Управление Управление требованиями разработкой качеством Основа Связи артефактов Стандартизация Составление Интеграции (Открытые API) процессов отчетов2 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 3. Управление жизненным циклом приложений (развертывание, APM, ITSM)и планирование (PPM, гос.) Управление жизненным циклом приложений Интеграция в стратегию Планирование и отслеживание проекта Управление корпоративными версиями Интеграция в работу Управление Управление Управление требованиями разработкой качеством IDE, SCM, Управле-ние Безопасная Функцио- Определение Управление Производи- Безопас- построение дефектами разработка нальный требований требованиями тельность ность интеграций уровень Основа Связи артефактов Стандартизация Составление Интеграции (Открытые API) процессов отчетов Объединено для базовых этапов жизненного цикла приложений, интегрировано3 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information containedциклом to change without notice. с полным herein is subject приложений
  • 4. Что сложного в созданииреалистичного нагрузочноготестатеста?
  • 5. Некоторые из сложностей 1. Тестовое покрытие не полно 2. Профиль нагрузочного теста не похож на продуктивный 3. Заглушки «нереально» производительны 4. Воздействие слабых сетей не изучено5 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 6. Сложности реалистичного тестированияТестовое покрытие не полноПрофиль нагрузочного теста непохож на продуктивныйНеобходимо взаимодействие сэксплуатацией6 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 7. Интеграция с подразделениями ЭксплуатацииRUM, SiteScope, BPM Мониторы Мониторы SLA SLО Нагрузочна Продуктив я среда Скрипты Транзакционная Сценарии нагрузка7 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 8. Взаимодействие в HP Performance centerПостоянное совершенствование Сбор и анализИспользование продуктивных данных: продуктивных данныхИсточники• RUM, WebTrends, логи, итд НагрузкаСкрипты Переопределение• Синтетические Business Process Monitoring (BPM) тестового сценария• Из реальных сессий - Real User Monitoring (RUM) Анализ данных нагрузкиИнформация о мониторинге Сравнение данных• Профили SiteScope продуктива и нагрузки8 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 9. Сравнение теста и продуктиваДоведение тестового сценария до совершенстваСравнение тестовые результаты VS продуктивные результаты9 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 10. Распределение нагрузки10 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 11. Производительность11 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 12. Сложности реалистичного тестированияЗаглушки «нереально»производительныВиртуализация сервисов12 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 13. HP Service Virtualization – Существующиеснятие органичений сервисы 3я сторона Тестирование на виртуальных сервисах Плата-за-транзакцию Обучение и симуляция Service Тестируемое приложение Virtualization 14:00-16:00 • Доступно Ответ 3 с 13 • Не надо спрашивать других © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. • Снижение затрат
  • 14. Service Virtualization v 2.2 - поддерживаемыетехнологии (User Guide)14 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 15. Сложности реалистичного тестирования«Воздействие слабых сетей не Изучено»15 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 16. Нужно ли это?16 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 17. Проблема века: доступ с мобильныхустройствАудитория мобильного интернета растѐт в два раза быстрее, чемаудитория интернета в целом. В январе 2012 года мобильнымиустройствами для выхода в сеть воспользовались более 22% всехжителей российских городов с населением свыше 100 тыс. человек*На конец первого полугодия 2011 года в России насчитывалось около 40млн. активных пользователей передачи данных с мобильных устройств** * Аналитический отчет Яндекс, весна 2012 ** J’son & Partners Consulting 201117 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 18. Влияние скорости сети на восприятие18 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 19. Приложения стали сложнее19 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 20. Влияние мобильных клиентов на производительность Clean Lab Virtualized 1.8% мобильны Results Mobile х Results пользоват елей20 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 21. Производительность системы Реальные условия Быстрая сеть сети Capacity > 500+ Users Response < 10 или Capacity < 200 Users Response > 40 Результаты теста без эмуляции скорости сети излишне оптимистичны21 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 22. HP Load Runner+ Shunra Network Virtualization Полная картинка производительности ~70% потерь времени в сети• Shunra воссоздает реальные условия сети• Динамически меняющиеся параметры связи• Анализ и рекомендации по уличшению 22 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 23. LoadRunner 11.5© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 24. Introducing HP LoadRunner 11.50Focus on agility for you, your IT Organization and the your Business Application Agility Usability & Efficiency Flexibility Extensibility• New VUGen • Mobile Protocols • Extension on the functionality of the• New Correlation Studio • Ajax TruClient for Internet Explorer (IE 9) all-new VUGen IDE itself by • Enhancements on: developing add-ins that is integrated• Enhancements to Analysis into the user interface.• Collaboration – TruClient for Firefox • Additional Data Format Extensions – Web Services (DFE) – Flex • Added new languages: Italian, • Now supporting: Russian, Brazilian Portuguese and – .NET4 Support Korean. – Recording 64bit Applications – IPv6 – Web Protocol Asynchronous24 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 25. Usability & EfficiencyNew and Innovative VUGen (Virtual User Generator)Modern Interface redesigned withIDE and Design Studio• Intuitive look and feel• Customizable Layout• Improved Editor, now context-sensitive• True C language debugger included• Solution Explorer: faster comparison• Step Navigator: single view for script• Help: context-sensitive25 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 26. Usability & Efficiency VUGen: LayoutsAbility to customize the appearance of the interface in VuGen. 26 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 27. Usability & Efficiency VUGen: Thumbnail ExplorerA combined view of the visual context of the script and its codewith easy navigation. 27 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 28. Usability & Efficiency VUGen: Improved Snapshot viewEnables you to view the snapshots captured duringrecording or replay of the script. 28 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 29. Usability & Efficiency VUGen: Step Navigator Enables quick navigation between steps in the script as well as filtering and searching according to its properties29 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 30. Usability & Efficiency VUGen: Script Editor• Code completion list• Syntax highlight for the C code• Highlighting of the Step parameters• Collapsing of user-defined functions• Tooltips display information for user-defined functions, parameters, etc. 30 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 31. Usability & Efficiency VUGen: Debug Panes Easy way to help you to find issues within your script31 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 32. Usability & Efficiency VUGen: New Correlation Design Studio and Additional APIsMakes script easier, faster and more intuitiveGrouping correlation capabilities in one place 32 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 33. Usability & Efficiency CollaborationCommunity Sharing Integration into HP online community Easily access to forums, blogs and twitter. Include your favorite search engine 33 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
  • 34. Bundle Name Protocols Bundle Name ProtocolsПротоколы ® .NET Record/ Replay Microsoft ADO.N ET Rich Internet Action M essage Format (includes RTMP AMF) / Microsoft .N ET 2.0, 3.0, and 3.5 Applications AJAX Click and Script W indows® Communication Foundation (W CF) AJAX TruClient Flex Virtual User (for Adobe® Flash)и Среды Database Microsoft SQ L Server Silverlight Vuser O DBC O racle (2-Tier) SAP SAP Click and Script SAP GUI DCOM Microsoft COM/ DCO M SAP-W eb GUI Virtual Users HP Functional Testing (Q uickTest Professional) SOA MQSeries-Client MQSeries-Server Java Record/ Replay CO RBA-Java Service Test Vuser Jacada W eb ServicesHP LoadRunner Java over HTTP Vuser JMS RMI-Java (includes O RMI) Templates* C Vuser C#.N ET Vuser (Visual Studio Add-in) C++.N ET Vuser (Visual Studio Add-in) Network Domain N ame Resolution (DN S) Enterprise Java Beans (EJB) File Transfer Protocol (FTP) Java Vuser Internet M essaging (IM AP) Javascript Vuser Lightweight Directory Access Protocol (LDAP) VB Script Vuser Microsoft Exchange (M API) VB VuserHP Performance Post O ffice Protocol (PO P3) Simple M ail Transfer Protocol (SMTP) Tuxedo W indows Sockets Web 2.0 VB.N ET Vuser (Visual Studio Add-in) VBN et Vuser W eb and Multimedia, RIA and SOA (combined)Center Mobile TruClient, Mobile Applications protocol Oracle E-Business O racle N CA O racle W eb Applications 11i (Click and Script) Web and Multimedia Media Player (MMS) PeopleSoft Enterprise (Click and Script) Real (RealPlayer) PeopleSoft-Tuxedo W eb (Click and Script) Siebel-W eb W eb (HTTP HTML) / Remote Access Citrix Virtual User (ICA) Wireless i-mode Terminal Emulation (RTE) Multimedia M essaging Service (MMS) Palm Remote Desktop Microsoft Remote Desktop (RDP) W AP * Template virtual users are manually developed and do not include record capability Enterprise Suite—(Performance Center Only) Includes the following protocol bundles: .N et Record/ Replay, Database, Java, N etwork, O racle Enterprise Business, Remote Access, Remote Desktop, Rich Internet Application,34 © Copyright 2012 Hewlett-Packard Development Company, L.P., SDK, information M ultimedia, hereinireless. (The enterprise suite EXCLUDES the following bundles: DCOM, GUI Virtual Users and Templates) SAP The SOA, W eb & contained and W is subject to change without notice.
  • 35. Спасибо!