Sebastian Kruk - Compuware - Ostatnia faza produktu: co się dzieję kiedy programista zakończył swoje zadanie

1. ‹#›
COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
…
or
when
does
the
job
for
the
developer
end?
!
Dr
inż.
Sebastian
Ryszard
Kruk
Product
Manager
Compuware
Late
phases
of
the
application
lifecycle
1

2. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
You
build
it,
you
test
it,
you
deploy
it
…
2

3. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
How
often
do
they
deploy?
3

4. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
How
often
do
they
deploy?
3
300
Deployments
/
Year

5. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
How
often
do
they
deploy?
3
300
Deployments
/
Year
10+
Deployments
/
Day

6. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
How
often
do
they
deploy?
3
300
Deployments
/
Year
50-­‐60
Deployments
/
Day
10+
Deployments
/
Day

7. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
How
often
do
they
deploy?
3
300
Deployments
/
Year
50-­‐60
Deployments
/
Day
10+
Deployments
/
Day
Every
11.6
seconds

8. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
More
on
Amazons
Story
4
Deploying
every
11.6s

9. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
More
on
Amazons
Story
4
75%
fewer
outages
since
2006
Deploying
every
11.6s

10. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
More
on
Amazons
Story
4
75%
fewer
outages
since
2006
90%
fewer
outage
minutes
Deploying
every
11.6s

11. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
More
on
Amazons
Story
4
75%
fewer
outages
since
2006
90%
fewer
outage
minutes
~0.001%
of
deployments
cause
a
problem
Deploying
every
11.6s

12. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
More
on
Amazons
Story
4
75%
fewer
outages
since
2006
90%
fewer
outage
minutes
~0.001%
of
deployments
cause
a
problem
Instantaneous
automatic
rollback
Deploying
every
11.6s

13. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
…
and
it
fails
5

14. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Status
Quo:
Unreliable
Software
Impacts
Business
6

15. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Status
Quo:
Unreliable
Software
Impacts
Business
6

16. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive7
War rooms

17. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive7
NASA
Mission
Control
Center,
Apollo
13,
1970
War rooms

18. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive7
NASA
Mission
Control
Center,
Apollo
13,
1970 Facebook,
December
2012
War rooms

19. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive7
NASA
Mission
Control
Center,
Apollo
13,
1970 Facebook,
December
2012
Mean
Time
to
Innocence
War rooms

20. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
It
works
on
your
machine
8

21. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Status
Quo:
Bugfixing
in
Production
happens
9
YES
we
know
this
!
80%
Dev
Time
in
Bug
Fixing
!
$60B
Defect
Costs
!
BUT

22. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Status
Quo:
Bugfixing
in
Production
happens
9
~80%
of
problems
caused
by
~20%
patterns
YES
we
know
this
!
80%
Dev
Time
in
Bug
Fixing
!
$60B
Defect
Costs
!
BUT

23. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Architectural
Decisions
10
Metrics:
#
Visitors,
#
Requests
/
User

24. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Deployment
Considerations
11
Metrics:
#
Log
Messages

25. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Implementation
Flaws
12
Still
crashes
Problem
fixed!Fixed
Version
Deployed
Metrics:
Heap
Size,
#
Objects
Allocated

26. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Not
Optimized
3rd
party
Software
13
Metrics:
#
SQL
Executions
/
Request
#
of
“same”
SQL
Executions

27. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
CDN
Configuration
Mistakes
14
46!
HTTP
403
Requests
for
images
on
the
landing
page
Probably
a
configuration
problem
with
their
Amazon
AWS
CDN
Lots
of
time
“wasted”
due
to
roundtrips
that
just
result
in
a
403
Metrics:
HTTP
4xx
&
5xx
Total
Number
of
Resources

28. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Disconnect
with
Business
15
Metrics:
Expected
vs
Real
Visits
by
Region
Expected
vs
Real
User
Experience

29. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
Network
16
Metrics:
End-­‐to-­‐End
RTT,
loss
rate,
network
performance

30. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
End-­‐User’s
Hardware
17
Metric:
Zero
windows
size
events

31. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Root
Cause:
End-­‐User
18
Reason:
Inefficient
use
of
custom
SAP
T-­‐Codes
in
custom
report

32. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Where
is
the
problem?
19
Shopzilla
CIO
(in
2010):
“…
when
they
get
in
the
war
room
-­‐
the
developers
and
ops
teams
describe
the
problem
as
the
enemy,
not
each
other”
Image
taken
from
https://www.scriptrock.com/blog/devops-­‐whats-­‐hype-­‐about/

33. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Where
is
the
problem?
19
Shopzilla
CIO
(in
2010):
“…
when
they
get
in
the
war
room
-­‐
the
developers
and
ops
teams
describe
the
problem
as
the
enemy,
not
each
other”
Image
taken
from
https://www.scriptrock.com/blog/devops-­‐whats-­‐hype-­‐about/

34. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Where
is
the
problem?
19
Shopzilla
CIO
(in
2010):
“…
when
they
get
in
the
war
room
-­‐
the
developers
and
ops
teams
describe
the
problem
as
the
enemy,
not
each
other”
Image
taken
from
https://www.scriptrock.com/blog/devops-­‐whats-­‐hype-­‐about/
Missing
Collaboration

35. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Where
is
the
problem?
19
Shopzilla
CIO
(in
2010):
“…
when
they
get
in
the
war
room
-­‐
the
developers
and
ops
teams
describe
the
problem
as
the
enemy,
not
each
other”
Image
taken
from
https://www.scriptrock.com/blog/devops-­‐whats-­‐hype-­‐about/
Missing
Collaboration
Disconnected
Teams

36. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Special
Ops
20
?

37. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
From
Waterfall
to
Continues
Operations
21
Waterfall Agile
Continuous
Integration
Continuous
Delivery
Continuous
Deployment

38. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
From
Waterfall
to
Continues
Operations
21
Waterfall Agile
Continuous
Integration
Continuous
Delivery
Continuous
Deployment
DevOps
Continuous
Operations

39. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Solution:
DevOps
+
Performance
Focus
22

40. ‹#› COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Culture Become ONE Team

41. ‹#› COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Culture Become ONE Team
Performance Scalability Deployment

42. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Automate
24
#
of
SQL
Executions
#
of
Log
Lines
MBs
/
Uses
Time
for
Deployment
Time
for
Rollback
Response
TimesPerf
Test
Code
Coverage
Measurability

43. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Automate
24
#
of
SQL
Executions
#
of
Log
Lines
MBs
/
Uses
Time
for
Deployment
Time
for
Rollback
Response
TimesPerf
Test
Code
Coverage
Measurability
Tools Results

44. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build # Test Case Status
Test Framework Results

45. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build # Test Case Status
Test Framework Results
We identified a regresesion

46. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status
Test Framework Results
Problem solved

47. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status # SQL # Excep CPU
Test Framework Results Architectural Data
Lets look behind the scenes

48. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
Test Framework Results Architectural Data
Lets look behind the scenes

49. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
Test Framework Results Architectural Data
Lets look behind the scenes
Exceptions probably reason for
failed tests

50. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
Lets look behind the scenes
Problem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regression

51. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Continuous
Performance
in
Test
Automation
25
12 0 120ms
3 1 68ms
Build testSearch OK
testPurchase OK
Build testSearch OK
testPurchase OK
Build testSearch FAILED
testPurchase OK
Build testSearch OK
testPurchase OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test Framework Results Architectural Data
Lets look behind the scenes
Now we have the functional and
architectural confidence

52. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive26
What about war rooms?

53. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive26
What about war rooms?
CANCELLED

54. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Learn
More
about
DevOps
27

55. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Learn
More
about
DevOps
27

56. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Learn
More
about
DevOps
27

57. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Let’s
Play
Murder
…
• Fonterra,
our
customer,
noticed
that
the
more
milk
churns
they
scan
the
slower
the
response
from
SAP
server
is.
28

58. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Let’s
Play
Murder
…
• Fonterra,
our
customer,
noticed
that
the
more
milk
churns
they
scan
the
slower
the
response
from
SAP
server
is.
28

59. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE#APMLive
Let’s
Play
Murder
…
• Fonterra,
our
customer,
noticed
that
the
more
milk
churns
they
scan
the
slower
the
response
from
SAP
server
is.
28
Why??
http://bit.ly/apmblog-fonterra

60. compuware.com/APM
COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Participate
in
Compuware
APM
Discussion
Forums
apmcommunity.compuware.com
Join
our
LinkedIn
group
Compuware
APM
User
Group
Follow
us
on
Twitter
twitter.com/CompuwareAPM
Read
our
Blog
apmblog.compuware.com
Like
us
on
Facebook
facebook.com/CompuwareAPM
Watch
our
Videos
&
product
Demos
youtube.com/Compuware
29

61. © 2011 Compuware Corporation — All Rights Reserved© 2011 Compuware Corporation — All Rights Reserved
COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE30