Don’t get blamed
for your choices
Hannes Lowette
Introduction
We need to talk …
What we do
The magpie
• Bird (black & white)
• Long tail
• Up to ~45cm long
• Wingspan up to ~60cm
• Omnivores
• Intelligent
(Rumoured to be…)
• Attracted to shiny things
The developer
• Human being (black/white/…)
• No tail
• Up to ~220cm tall
• No wings
• Omnivores
• Intelligent
(libraries, frameworks, architectures, …)
• Attracted to shiny things
Is that behavior so surprising?
• Technology constantly evolves
• We chose this profession regardless
 we like change
 we like new challenges
• Our profession is still young:
• We are still finding better ways
• No recipe for success
• Education is limited
Our education – an analogy
If you teach someone to:
• Pour concrete foundations
• Lay bricks
• Tile a roof
• Do an electrical wiring diagram
• Connect pipes for plumbing
Can they design and build a house?
So we look for guidance…
• Conferences (like this one)
• Blog posts
• The big guys:
• StackOverflow
• Facebook
• Google
• …
• The most dedicated developer in our team
And we do what they do…
… until we run into a wall.
But why do we run
into that wall?
For that, we are going to need CHAD
Who is CHAD*?
• A developer
• Ran into that wall
• We will learn from his mistakes!
* Any resemblance to actual persons, living or dead,
or actual events is purely coincidental.
Join me for an adventure with CHAD …
The case
• We processed text files
• Used about 70% of the data
• The other 30% might be useful in the future
Task:
• Store them somewhere for future use
The solution
• Set up RavenDB
• Parse the contents of the text files
• Upload to RavenDB
What went wrong?
• RavenDB:
• € Per GB: pretty high
• Maintain the OS
• Maintain the DB engine
• The solution:
• Took longer to write
• Breaks on file structure changes
What did CHAD forget?
Technical fit
Is this technology suitable to solve my problem?
Technical fit
Ask yourself questions such as:
• Does this help with our problem?
• How does it compare against competitors?
• Is there something that will help us more?
• Is this the simplest thing we can do?
• Does it bring unnecessary costs?
• Is there extra maintenance?
Why did CHAD go wrong?
RavenDB is a database:
• Meant to be queried
• With indexes, projections, etc.
• Optimization for query
= concessions for inserts
 We were not using most of RavenDB.
If you only use it for storage, it is expensive
What could CHAD have done?
Azure BLOB storage:
• Easy to set up
• No parsing required
• € per GB: very cheap
• Very robust (and easy to distribute)
• No maintenance needed
The case
• 2014
• ASP.NET MVC Project
• No separation between UI/Logic/Data
• Issues due to N+1 selects from the views
• Inexperienced remote developers
Task:
• Make sure EF & logic are shielded
The solution
• Agatha = Request/Response Layer
• CHAD had used Agatha before
• Doing Agatha from MVC Controllers
• Solved our problem technically
• Didn’t introduce real overhead
• Made us think about Includes etc.
• Ran in process,
but could move to ‘over WCF’
 Technical fit was OK
What went wrong?
Agatha:
• Mainly maintained by 1 person
• Maintenance stopped in 2013
• No async/await support
• Dependencies didn’t get updated
• PRs didn’t get approved
 Agatha had become a liability, not an asset
What did CHAD forget?
Check for
warning signs
Can this technology become a risk?
Warning signs
Ask yourself questions such as:
• Who maintains this library?
• Are there multiple maintainers?
• Are they still actively working on it?
• Commits
• Blog posts
• Is the source available?
• readable and well structured?
• Can we legally use it?
• What does the licensing model look like?
• How likely is that model to change?
• Has the product recently been acquired by another party?
What could CHAD have done?
MediatR*:
• Almost the same functionality
• Source on GitHub
• Apache license
• Jimmy Bogard still blogs about it
• Actively maintained (by multiple people)
This seems like a safe bet!
*https://github.com/jbogard/MediatR
The case
• ASP.NET MVC Project
• Decent SOLID
• Some messaging to other products
• CHAD had recently become a fan of Greg Young & CQRS
Task:
• Build a screen with:
• an audit trail
• Resulting state
The solution
• CHAD decided on using EventStore
• Wrapper around the actual EventStore stuff
 For other devs to easily do the same
• Implemented the screen using his wrapper
Technical fit was OK
No warning signs
What went wrong?
Team:
• Lots of juniors
• Some remote contractors (abroad)
• No prior EventStore experience
The wrapper:
• Meant to solve this knowledge gap
• Only worked for a pretty specific case
• No access to the wrapper’s source
 The team never touched any EventStore screens.
CHAD was the only one who worked on those.
What did CHAD forget?
Team skills
Can we get everyone up to speed on this?
Team skills
Ask yourself questions such as:
• Do we have the knowledge?
• What it does
• How it is meant to be used
• What are the pitfalls?
• If not, is the knowledge available?
• Documentation
• Tutorials
• Book
• Code samples
• Can we teach the team?
• Give a presentation
• Build a POC together (hackathon?)
• Does everyone have the background to
understand?
• Can everyone work on it?
• For the implementation
• For maintenance
• What documentation do we need?
What could CHAD have done?
Spread the knowledge:
• Research the pitfalls and best practices
• Give a presentation to the team
• Organize a hackathon to try it out together
• Pair with another dev (preferably a junior) for the implementation
• Document the wrapper:
• Usage
• Effects
• Inner workings
• Make his wrapper code public
• Assist the team when they first touch that code
What happened
CHAD took me aside to talk to me:
• We should stop using Entity Framework
• We should write our own SQL instead
• We could use a micro-ORM like Dapper
(his latest library-crush)
Context:
• Our application was mainly CRUD
• Not a lot of high-load code either
What were the frustrations?
The reasons he gave me were:
• EF is very slow
• EF does too many things
• We have no control over the SQL it outputs
• The SQL is very messy
 All of these have truth to them, in the right context
What went wrong?
We looked at some problematic code:
• Used SQL Profiler
• Analyzed query plans
We saw:
• DbContext initialization delays  just started in Debug mode
• An N+1 select  Include to fix
• A missing index in the DB  Query was OK
• A crazy amount of fields were fetched  projection FTW
So until that day…
Entity Framework was MAGIC to CHAD
He didn’t know how
• A LINQ statement becomes a query
• How Lazy Loading actually worked
• What ModelCreating does
• Etc.
It was a black box that took away the need to understand/write SQL
What did CHAD forget?
Take away the magic
If you understand how it was built,
you’re a lot less likely to use it wrongly
Take away the magic
Ask yourself questions such as:
• Do we know how it works?
• Do we understand how it was built?
• What did the developers use?
• What concepts & technologies are applied?
• Could we (theoretically) build our own?
• Do we know what the effect of our usage is?
What could CHAD have done?
Understand the inner workings of EF:
• Look at EF source Code
• Use a profiler
• Try to build a simple ORM yourself
The case
• Long running project (8+ years)
• CHAD was the lead developer
• The whole team admired his skill
• The only one who learned in his free time
• Liked ‘doing things right’
• Took care of the ‘team skills’ part really well
The situation
• When CHAD learned
• A better way
• A new framework
• An awesome library
• He explained it to the team:
• Presentation
• Pitfalls
• Tips & tricks
• And from then on, they would do it like that
What went wrong?
Never re-wrote the old code:
• Business didn’t want to invest in old features
• The team was already under pressure
So old code stayed ‘the old way’
Why was that so bad?
The result of this was:
• 4 ways of writing frontend
• 3 ways of doing backend
• 2 API frameworks
• 2 database engines
• 3 ways of structuring backend code
• …
 Training a new developer in this codebase was impossible
What did CHAD forget?
Trim your stack
When new tech goes in, old tech must go out
Trim your stack
Ask yourself questions such as:
• What problem does this new tech solve?
• Do we really need that?
• Which old, similar, thing do we have?
• Can that be replaced by the new one?
• How much effort would that be?
• What effect does having both have?
• On maintenance
• On onboarding
What could CHAD have done?
“Hollow out the trunk”:
• Replace old tech as new tech arrives
• Add the estimate of the removal to the
‘cost’ of the introduction
• Make the pain of this technical debt
visible
• Limit introductions when previous
transitions aren’t complete
The case
• Small LOB application
• Little complexity
• Greenfield
• Team:
• Experience in ASP.NET MVC
• 1 senior
• 3 devs < 3y experience
• No real ops experience
The solution
• Angular
• IdentityServer
• Docker & Kubernetes
• Microservices
• MongoDB
 All of this was new to them
What went wrong?
The team got stuck on:
• Learning the new stack
• Deployments
• Debugging
• MongoDB
So the project got a huge delay
 Massive ‘team skills’ violation
 Over-engineered
Team skill - revisited
New skills in real projects:
• Don’t combine learning & delivery goals
• Budget extra time
• 1 at a time
• Explain the risk to business
“This new way of doing things will make sure
we can deliver more, with higher quality.”
What did CHAD forget?
Evolving architecture
Your architecture should follow your problems
Not the other way around
Evolving Architecture
KISS
• Start with the simplest solution
(Monolith?)
• Discover your needs from PROD
• Adapt to those needs, extract services
 You’ll have a working product WAY faster
Microservices
When:
• Scale independently
 Balance cost with speed
• Multiple clients
 Clients can adopt features at will
• Different technology stacks
• Conflicting dependencies
What could CHAD have done?
Let the architecture follow the needs:
• Start simple:
• ASP.NET & Angular (1 new thing)
• No microservices
• Structure code to allow decoupling:
• SOLID
• Vertical slices
• Solve architectural problems when they arise
 Deliver your project on time!
Summary
1. Technical fit
2. Check for warning signs
3. Team skill
4. Take away the magic
5. Trim your stack
6. Evolving architecture
No, really. Who is CHAD?
• Me
• The whole team
• Someone else
• Totally made up
 But he did what he did because he cared!
No, really. Who is CHAD?
• Me
• The whole team
• Someone else
• Totally made up
 But he did what he did because he cared!
Carelessly or Helplessly Acting Developer
Wouldn’t it be better to be a …
RAD
?
Wouldn’t it be better to be a …
RAD
Responsibly Acting Developer?
Are you on board with this mission?
1. Spread the word!
2. Follow us on Twitter: @RadCert
3. Go to https://rad-cert.com
4. Take the quiz
5. Go the Axxes booth
6. Go pick up your SWAG
7. Snail mail option coming soon!
About
Hannes Lowette
@hannes_lowette
.NET Consultant & Competence Coach
At @Axxes_IT

Don't get blamed for your choices - Techorama 2019

  • 2.
    Don’t get blamed foryour choices Hannes Lowette
  • 3.
  • 4.
  • 5.
    The magpie • Bird(black & white) • Long tail • Up to ~45cm long • Wingspan up to ~60cm • Omnivores • Intelligent (Rumoured to be…) • Attracted to shiny things
  • 6.
    The developer • Humanbeing (black/white/…) • No tail • Up to ~220cm tall • No wings • Omnivores • Intelligent (libraries, frameworks, architectures, …) • Attracted to shiny things
  • 7.
    Is that behaviorso surprising? • Technology constantly evolves • We chose this profession regardless  we like change  we like new challenges • Our profession is still young: • We are still finding better ways • No recipe for success • Education is limited
  • 8.
    Our education –an analogy If you teach someone to: • Pour concrete foundations • Lay bricks • Tile a roof • Do an electrical wiring diagram • Connect pipes for plumbing Can they design and build a house?
  • 9.
    So we lookfor guidance… • Conferences (like this one) • Blog posts • The big guys: • StackOverflow • Facebook • Google • … • The most dedicated developer in our team And we do what they do…
  • 10.
    … until werun into a wall.
  • 11.
    But why dowe run into that wall?
  • 12.
    For that, weare going to need CHAD Who is CHAD*? • A developer • Ran into that wall • We will learn from his mistakes! * Any resemblance to actual persons, living or dead, or actual events is purely coincidental.
  • 13.
    Join me foran adventure with CHAD …
  • 15.
    The case • Weprocessed text files • Used about 70% of the data • The other 30% might be useful in the future Task: • Store them somewhere for future use
  • 16.
    The solution • Setup RavenDB • Parse the contents of the text files • Upload to RavenDB
  • 17.
    What went wrong? •RavenDB: • € Per GB: pretty high • Maintain the OS • Maintain the DB engine • The solution: • Took longer to write • Breaks on file structure changes
  • 18.
    What did CHADforget? Technical fit Is this technology suitable to solve my problem?
  • 19.
    Technical fit Ask yourselfquestions such as: • Does this help with our problem? • How does it compare against competitors? • Is there something that will help us more? • Is this the simplest thing we can do? • Does it bring unnecessary costs? • Is there extra maintenance?
  • 20.
    Why did CHADgo wrong? RavenDB is a database: • Meant to be queried • With indexes, projections, etc. • Optimization for query = concessions for inserts  We were not using most of RavenDB. If you only use it for storage, it is expensive
  • 21.
    What could CHADhave done? Azure BLOB storage: • Easy to set up • No parsing required • € per GB: very cheap • Very robust (and easy to distribute) • No maintenance needed
  • 23.
    The case • 2014 •ASP.NET MVC Project • No separation between UI/Logic/Data • Issues due to N+1 selects from the views • Inexperienced remote developers Task: • Make sure EF & logic are shielded
  • 24.
    The solution • Agatha= Request/Response Layer • CHAD had used Agatha before • Doing Agatha from MVC Controllers • Solved our problem technically • Didn’t introduce real overhead • Made us think about Includes etc. • Ran in process, but could move to ‘over WCF’  Technical fit was OK
  • 25.
    What went wrong? Agatha: •Mainly maintained by 1 person • Maintenance stopped in 2013 • No async/await support • Dependencies didn’t get updated • PRs didn’t get approved  Agatha had become a liability, not an asset
  • 26.
    What did CHADforget? Check for warning signs Can this technology become a risk?
  • 27.
    Warning signs Ask yourselfquestions such as: • Who maintains this library? • Are there multiple maintainers? • Are they still actively working on it? • Commits • Blog posts • Is the source available? • readable and well structured? • Can we legally use it? • What does the licensing model look like? • How likely is that model to change? • Has the product recently been acquired by another party?
  • 28.
    What could CHADhave done? MediatR*: • Almost the same functionality • Source on GitHub • Apache license • Jimmy Bogard still blogs about it • Actively maintained (by multiple people) This seems like a safe bet! *https://github.com/jbogard/MediatR
  • 30.
    The case • ASP.NETMVC Project • Decent SOLID • Some messaging to other products • CHAD had recently become a fan of Greg Young & CQRS Task: • Build a screen with: • an audit trail • Resulting state
  • 31.
    The solution • CHADdecided on using EventStore • Wrapper around the actual EventStore stuff  For other devs to easily do the same • Implemented the screen using his wrapper Technical fit was OK No warning signs
  • 32.
    What went wrong? Team: •Lots of juniors • Some remote contractors (abroad) • No prior EventStore experience The wrapper: • Meant to solve this knowledge gap • Only worked for a pretty specific case • No access to the wrapper’s source  The team never touched any EventStore screens. CHAD was the only one who worked on those.
  • 33.
    What did CHADforget? Team skills Can we get everyone up to speed on this?
  • 34.
    Team skills Ask yourselfquestions such as: • Do we have the knowledge? • What it does • How it is meant to be used • What are the pitfalls? • If not, is the knowledge available? • Documentation • Tutorials • Book • Code samples • Can we teach the team? • Give a presentation • Build a POC together (hackathon?) • Does everyone have the background to understand? • Can everyone work on it? • For the implementation • For maintenance • What documentation do we need?
  • 35.
    What could CHADhave done? Spread the knowledge: • Research the pitfalls and best practices • Give a presentation to the team • Organize a hackathon to try it out together • Pair with another dev (preferably a junior) for the implementation • Document the wrapper: • Usage • Effects • Inner workings • Make his wrapper code public • Assist the team when they first touch that code
  • 37.
    What happened CHAD tookme aside to talk to me: • We should stop using Entity Framework • We should write our own SQL instead • We could use a micro-ORM like Dapper (his latest library-crush) Context: • Our application was mainly CRUD • Not a lot of high-load code either
  • 38.
    What were thefrustrations? The reasons he gave me were: • EF is very slow • EF does too many things • We have no control over the SQL it outputs • The SQL is very messy  All of these have truth to them, in the right context
  • 39.
    What went wrong? Welooked at some problematic code: • Used SQL Profiler • Analyzed query plans We saw: • DbContext initialization delays  just started in Debug mode • An N+1 select  Include to fix • A missing index in the DB  Query was OK • A crazy amount of fields were fetched  projection FTW
  • 40.
    So until thatday… Entity Framework was MAGIC to CHAD He didn’t know how • A LINQ statement becomes a query • How Lazy Loading actually worked • What ModelCreating does • Etc. It was a black box that took away the need to understand/write SQL
  • 41.
    What did CHADforget? Take away the magic If you understand how it was built, you’re a lot less likely to use it wrongly
  • 42.
    Take away themagic Ask yourself questions such as: • Do we know how it works? • Do we understand how it was built? • What did the developers use? • What concepts & technologies are applied? • Could we (theoretically) build our own? • Do we know what the effect of our usage is?
  • 43.
    What could CHADhave done? Understand the inner workings of EF: • Look at EF source Code • Use a profiler • Try to build a simple ORM yourself
  • 45.
    The case • Longrunning project (8+ years) • CHAD was the lead developer • The whole team admired his skill • The only one who learned in his free time • Liked ‘doing things right’ • Took care of the ‘team skills’ part really well
  • 46.
    The situation • WhenCHAD learned • A better way • A new framework • An awesome library • He explained it to the team: • Presentation • Pitfalls • Tips & tricks • And from then on, they would do it like that
  • 47.
    What went wrong? Neverre-wrote the old code: • Business didn’t want to invest in old features • The team was already under pressure So old code stayed ‘the old way’
  • 48.
    Why was thatso bad? The result of this was: • 4 ways of writing frontend • 3 ways of doing backend • 2 API frameworks • 2 database engines • 3 ways of structuring backend code • …  Training a new developer in this codebase was impossible
  • 49.
    What did CHADforget? Trim your stack When new tech goes in, old tech must go out
  • 50.
    Trim your stack Askyourself questions such as: • What problem does this new tech solve? • Do we really need that? • Which old, similar, thing do we have? • Can that be replaced by the new one? • How much effort would that be? • What effect does having both have? • On maintenance • On onboarding
  • 51.
    What could CHADhave done? “Hollow out the trunk”: • Replace old tech as new tech arrives • Add the estimate of the removal to the ‘cost’ of the introduction • Make the pain of this technical debt visible • Limit introductions when previous transitions aren’t complete
  • 53.
    The case • SmallLOB application • Little complexity • Greenfield • Team: • Experience in ASP.NET MVC • 1 senior • 3 devs < 3y experience • No real ops experience
  • 54.
    The solution • Angular •IdentityServer • Docker & Kubernetes • Microservices • MongoDB  All of this was new to them
  • 55.
    What went wrong? Theteam got stuck on: • Learning the new stack • Deployments • Debugging • MongoDB So the project got a huge delay  Massive ‘team skills’ violation  Over-engineered
  • 56.
    Team skill -revisited New skills in real projects: • Don’t combine learning & delivery goals • Budget extra time • 1 at a time • Explain the risk to business “This new way of doing things will make sure we can deliver more, with higher quality.”
  • 57.
    What did CHADforget? Evolving architecture Your architecture should follow your problems Not the other way around
  • 58.
    Evolving Architecture KISS • Startwith the simplest solution (Monolith?) • Discover your needs from PROD • Adapt to those needs, extract services  You’ll have a working product WAY faster
  • 59.
    Microservices When: • Scale independently Balance cost with speed • Multiple clients  Clients can adopt features at will • Different technology stacks • Conflicting dependencies
  • 60.
    What could CHADhave done? Let the architecture follow the needs: • Start simple: • ASP.NET & Angular (1 new thing) • No microservices • Structure code to allow decoupling: • SOLID • Vertical slices • Solve architectural problems when they arise  Deliver your project on time!
  • 61.
    Summary 1. Technical fit 2.Check for warning signs 3. Team skill 4. Take away the magic 5. Trim your stack 6. Evolving architecture
  • 62.
    No, really. Whois CHAD? • Me • The whole team • Someone else • Totally made up  But he did what he did because he cared!
  • 63.
    No, really. Whois CHAD? • Me • The whole team • Someone else • Totally made up  But he did what he did because he cared! Carelessly or Helplessly Acting Developer
  • 64.
    Wouldn’t it bebetter to be a … RAD ?
  • 65.
    Wouldn’t it bebetter to be a … RAD Responsibly Acting Developer?
  • 66.
    Are you onboard with this mission? 1. Spread the word! 2. Follow us on Twitter: @RadCert 3. Go to https://rad-cert.com 4. Take the quiz 5. Go the Axxes booth 6. Go pick up your SWAG 7. Snail mail option coming soon!
  • 67.