Staying on top of product quality is no easy task under normal circumstances. It’s even harder when your company is rapidly expanding and developers are adding features to your product 24/7 from all over the globe. Jan Swaelens from Sofico shows us how they integrated Dynatrace UEM-based test automation with Bamboo-driven Selenium-based test automation. Learn from Sofico's experience to create an automation engine step-by-step. Then you can start today—not tomorrow—and increase customer satisfaction and organizational efficiency.

1. DevOps at Sofico
Shift-Left Performance to scale Distributed Team Output

2. Software solutions for automotive finance, leasing, fleet & mobility management
DevOps at Sofico
Shift-Left Performance to scale Distributed Team Output
February 7, 2017 – Dynatrace PERFORM
Jan Swaelens
Solution Management @ Sofico
@JanSwaelens

3. confidential
Who is Sofico?

5. Global Presence

6. Our Customers

7. confidential
What and How we Engineer!

8. People

9. Turnover

10. One Solution
Many Devices
Mobile
Backoffice
Web
Lots of Challenges

11. Architecture Overview
Database
Client Layer
Web Layer
Business Layer

12. Transaction Flow

13. Figures

14. Figures

15. Figures

16. Figures

17. confidential
Shift-Left?
Why and How with Dynatrace

18. A Delivery Pipeline DOES NOT PREVENT your teams from
pushing bad code through the pipeline more frequently!
Dev&Test: Trial & Error
Check-In Mentality!
Performance: Too many code commits
leads to skipping performance testing to
save time
Ops/Biz: Bad Code Changes make it
to Production. Impact End Users and
Resource Usage
Test / CI: Basic Tests that only
test high-level functionality!

19. “To Deliver High Quality Working Software Faster“
„We have to Shift-Left Performance to Optimize Pipelines“
http://apmblog.dynatrace.com/2016/10/04/scaling-continuous-delivery-shift-left-performance-to-improve-lead-time-pipeline-flow/

20. = Unit-, Integration-, WebAPI- & Functional Test Result (passed/failed)
+ Web Performance Metrics (# of Images, # of JavaScript, Page Load Time, ...)
+ App Performance Metrics (# of SQL, # of Logs, # of API Calls, # of Exceptions ...)
Fail the build early!

21. Reduce Lead Time: Stop 80% of Performance Issues
in your Integration Phase
CI/CD: Test Automation (Selenium, Appium,
Cucumber, Silk, ...) to detect functional and
architectural (performance, scalabilty) regressions
Perf: Performance Test (JMeter,
LoadRunner, Neotys, Silk, ...) to detect
tough performance issues

22. Shift-Left Performance results in Reduced Lead Time
powered by Dynatrace Test Automation
http://apmblog.dynatrace.com/2016/10/04/scaling-continuous-delivery-shift-left-performance-to-improve-lead-time-pipeline-flow/

23. confidential
Shift-Left at Sofico!
for Functional Tests

24. RIA Test Automation

25. confidential
Browse to URL
Run Test
Close Browser
Browse to URL instrumented with Dynatrace UEM
Tag Session with Test Name.
(optional) Test Run ID
Run Test
End Visit
(optional) Functional Status
Close Browser
sessionStorage.DT_TESTNAME=“Tests.MyApp.HomepageTest“;
sessionStorage.DT_TESTRUNID = “<TEST RUN ID FOR BUILD>“;
sessionStorage.DT_TESTRUN_SUCCESS = “Functional Success??“;
dynaTrace.endVisit();
Level-Up with Dynatrace
For UI-Driven Tests

26. Test Automation Overview
#4: Regressions
Detected
#1: Every Browser
Driven Test
#3: Metrics are
baselined per Test

27. Test Overview in Dynatrace across Builds

28. Drill into Visits for each Test Execution

29. confidential
Shift-Left at Sofico!
for Web API Tests

30. WS Test Automation

31. confidential
Deploy your App
Run Tests
Analyze Results
Deploy your app instrumented with Dynatrace AppMon
Register WebAPI Test Run: Retreive <DT Test Run ID>
Pass Test Run ID to Test to add as HTTP Header: NA=Test Name;TR=<DT Test Run ID>
Dynatrace provides Results
Level-Up with Dynatrace
Run Tests
For Web API Tests

32. WS Response Time

33. Test Automation Overview
#4: Regressions
Detected
#1: Every Web API
Test is detected
#3: Metrics are
baselined per Test

34. confidential
Shift-Left at Sofico!
Integrated into our Pipeline

35. Integrating it to our Build Server

36. Automation Overview
Version Control System
Dynatrace
Server(s)
Developer Commit
Trigger
build
Build and
run tests
Publish performance
metrics
Drilldown
for further
analysis
Inform
about build
status

37. Caution

38. Advice

39. Tuned Pipeline

40. confidential
The Success of Shift-Left!

41. Response Time

42. Memory Issues
Out of
Memory

43. Wait Time

44. Measure
= Knowledge

45. Ping - Pong
Dismissed!

46. Product
Mgmt
IT Sales
Release
Mgmt
Development

47. confidential
Code Lab

48. confidential
Test Automation Data is managed around Dynatrace Test Runs
 Test Runs are registered & managed
 Via REST -> or via existing Plugins for Jenkins, Bamboo, TFS
 For a System Profile
 For a particular Test Category: Unit, Performance (Unit), UI-Driven, Web API
 For a particular Build (Major, Minor, Revision, Build #, additional name / value pairs)
 Explore the REST API: https://YOURDTSERVER:8021/api-docs/current/index.html#/Test_Automation

49. confidential
Here is what we built for Bamboo
 https://marketplace.atlassian.com/plugins/be.sofico.bamboo.plugins.bamboo-dynatrace-plugin/server/overview

50. Code Lab
#1
#2
#3

51. Code Lab
#1
#1

52. Code Lab
#1
#2
#3
#4

53. Code Lab: SauceLabs

54. Code Lab: Output in Dynatrace

55. Code Lab: Output in Bamboo

56. confidential
Advanced REST API

57. confidential
External Result REST API: Use Dynatrace as Test Result Repository!
 Feeding any external Test Data into Dynatrace
External Test Type
Value + Corridor

58. confidential
External Test Data REST API: Register your External Test
System Profile
Build Info
“External” Test
Category
Test Run ID

59. confidential
Feeding External Test Data to Test Run
Profile + Test
Run ID
POST Test Results
Test Name: A.B.C.xxx
List of Measures:
Value, Min, Max, Color …

60. Resources
Find more information Online
 YouTube Performance Clinic: http://bit.ly/dttutorials
 Shift-Left Performance with Dynatrace AppMon & UEM
 Blog Posts: http://blog.dynatrace.com
 How to Automate Performance Testing for GWT and SmartGWT
 Podcast: https://www.spreaker.com/user/pureperformance
 Shift-Left Performance at Sofico
 Bamboo Integration
 https://marketplace.atlassian.com/plugins/be.sofico.bamboo.plugins.b
amboo-dynatrace-plugin/server/overview

61. Software solutions for automotive finance, leasing, fleet & mobility management
www.sofico.be
www.soficoservices.com
www.sofico.be
www.soficoservices.com

62. Thank you.