0
Hard & Soft Skills to
Avoid Outages
@pascallouis from @SquareNY
Code

Bless

Ship

Maintain

Profit!

git rm
Code

Bless

Ship

Maintain

Profit!

git rm
Tactics

• Fighting mixing ids
• Entity bound ids (e.g. Id<T>)
• Textual ids MWDN-YP89-OLVL-USER
• Testable configurations
...
TDD

• Not controversial (anymore)
• Living code documentation
• Enables collaboration
• Technique to encode invariants
Co...
Gold Tests

• Tests which can be changed by a (small)
subset of engineering

• Enforced via policy or technology
Code

Ble...
Expressive Tests

• “Change your language and you change
your thoughts” — Karl Albrecht

• Can be implementation agnostic
...
...
Given feed PaymentEventFeedListener receives:
"""
{
"payment_id": "EPT-300",
"isTivoReplay": false,
"merchant": {
"tok...
Automated

Oups!

Manual
Quality

or

or

Time

Code

Bless

Ship

Maintain

Profit!

git rm
Code Analysis

• In theory: static vs dynamic
• In practice: pre vs post-production
Code

Bless

Ship

Maintain

Profit!

g...
Pre Analysis

• Type Checking
• Testing, CI
• Linters
• Forbidden Call Analysis
Code

Bless

Ship

Maintain

Profit!

git r...
Post Analysis

• Logging
• Metrics
• Invariant Checking
Code

Bless

Ship

Maintain

Profit!

git rm
Speaking of Alerts: Metrics vs Checks
OK

?

WARNING

1
0

200ms

0ms

Code

Bless

Ship

Maintain

Profit!

git rm
Alerting & Reporting
Sign

Precise

Imprecise

Immediate

Alert

Oups!

Deferred

Report

Report

Res
pon
se

Code

Bless
...
Fix It Weeks

• Time set aside, monthly or quarterly
• No top-down mandate except “fix it”
Code

Bless

Ship

Maintain

Pro...
Code

Bless

Ship

Maintain

Profit!

git rm
Post-Mortem

• When Anytime there are issues!
• Why Learn and avoid mistakes of the past
• How Blameless
Code

Bless

Ship...
Post-Mortem

• Go through the timeline
• The Good, The Bad and the Ugly
• Action Items
Code

Bless

Ship

Maintain

Profit!...
Root Cause Analysis

Code

Bless

Ship

Maintain

Profit!

git rm
Code

Bless

Ship

Maintain

Profit!

git rm
Proportional Investing

• When you lose N hours to maintenance, you
spend an equivalent N hours on improving
things.

Code...
Safety drives productivity; and
unleashes creativity.
Technology, sure. But, it’s mostly about
culture and people.
Many la...
Hard & Soft Skills to Avoid Outages by Pascal-Louis Perez
Hard & Soft Skills to Avoid Outages by Pascal-Louis Perez
Upcoming SlideShare
Loading in...5
×

Hard & Soft Skills to Avoid Outages by Pascal-Louis Perez

724

Published on

In just a few years, Square has achieved ubiquitous recognition for mobile card processing grossing over $15B a year in credit card transactions. At the heart of Square's technology are many financial systems which must operate safely, correctly, and sustain rapid growth.

This presentation will cover non-controversial topics such ad TDD, but from new angles. It'll also cover emerging practices like continuous deployment, and softer areas such as engineering management practices geared towards safety. You'll come out of this session with a fresh perspective on how to build software.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
724
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Hard & Soft Skills to Avoid Outages by Pascal-Louis Perez"

  1. 1. Hard & Soft Skills to Avoid Outages @pascallouis from @SquareNY
  2. 2. Code Bless Ship Maintain Profit! git rm
  3. 3. Code Bless Ship Maintain Profit! git rm
  4. 4. Tactics • Fighting mixing ids • Entity bound ids (e.g. Id<T>) • Textual ids MWDN-YP89-OLVL-USER • Testable configurations • etc. Code Bless Ship Maintain Profit! git rm
  5. 5. TDD • Not controversial (anymore) • Living code documentation • Enables collaboration • Technique to encode invariants Code Bless Ship Maintain Profit! git rm
  6. 6. Gold Tests • Tests which can be changed by a (small) subset of engineering • Enforced via policy or technology Code Bless Ship Maintain Profit! git rm
  7. 7. Expressive Tests • “Change your language and you change your thoughts” — Karl Albrecht • Can be implementation agnostic Code Bless Ship Maintain Profit! git rm
  8. 8. ... Given feed PaymentEventFeedListener receives: """ { "payment_id": "EPT-300", "isTivoReplay": false, "merchant": { "token": "m-1" }, ... } """ Then expect table balance_changing_events order by id: | event_type | status | process_attempts | | HOLD | UNPROCESSED | 1 | | CAPTURE | UNPROCESSED | 0 | When then the time is 2012-01-06 17:10:00 And balance changing event queue processes items Then expect table balance_changing_events order by id: | event_type | status | process_attempts | | HOLD | UNPROCESSED | 2 | | CAPTURE | PROCESSED | 1 | Code Bless Ship Maintain Profit! git rm
  9. 9. Automated Oups! Manual Quality or or Time Code Bless Ship Maintain Profit! git rm
  10. 10. Code Analysis • In theory: static vs dynamic • In practice: pre vs post-production Code Bless Ship Maintain Profit! git rm
  11. 11. Pre Analysis • Type Checking • Testing, CI • Linters • Forbidden Call Analysis Code Bless Ship Maintain Profit! git rm
  12. 12. Post Analysis • Logging • Metrics • Invariant Checking Code Bless Ship Maintain Profit! git rm
  13. 13. Speaking of Alerts: Metrics vs Checks OK ? WARNING 1 0 200ms 0ms Code Bless Ship Maintain Profit! git rm
  14. 14. Alerting & Reporting Sign Precise Imprecise Immediate Alert Oups! Deferred Report Report Res pon se Code Bless al Ship Maintain Profit! git rm
  15. 15. Fix It Weeks • Time set aside, monthly or quarterly • No top-down mandate except “fix it” Code Bless Ship Maintain Profit! git rm
  16. 16. Code Bless Ship Maintain Profit! git rm
  17. 17. Post-Mortem • When Anytime there are issues! • Why Learn and avoid mistakes of the past • How Blameless Code Bless Ship Maintain Profit! git rm
  18. 18. Post-Mortem • Go through the timeline • The Good, The Bad and the Ugly • Action Items Code Bless Ship Maintain Profit! git rm
  19. 19. Root Cause Analysis Code Bless Ship Maintain Profit! git rm
  20. 20. Code Bless Ship Maintain Profit! git rm
  21. 21. Proportional Investing • When you lose N hours to maintenance, you spend an equivalent N hours on improving things. Code Bless Ship Maintain Profit! git rm
  22. 22. Safety drives productivity; and unleashes creativity. Technology, sure. But, it’s mostly about culture and people. Many layers of defense, lots of ways to do it — find what’s right for your team.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×