Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Technical Debt
Is your codebase approaching the fiscal cliff?
Andy Clarke
clarkie@concrete.cc
http://clarkie.io
@clarkiecla...
This is me
• Software Engineer for 8 years
• Never worked on a greenfield
project (at work)
• Technical Architect @ Concrete
Outline
• What is Technical Debt?
• Debt Accumulation
• Debt Reduction
• Q + A
– Ward Cunningham, OOPSLA 1992
“Shipping first time code is like going into debt;
!
a little debt speeds development so lon...
– Intel, 2009
“Technical Debt is loosely defined as the volume
of poorly written lines of code, poorly refactored,
not foll...
– Michael (Doc) Norton, 2009
“Messy Code is not Technical Debt”
Technical Debt
Is your code base project a ‘bag of spanners’?
Andy Clarke
clarkie@concrete.cc
http://clarkie.io
@clarkiecl...
Where does it come from?
• No vision
• No design
• Operational
• Experimentation
Where to keep your debt
• Code
• Persistent storage
• Architecture
• Infrastructure
Debt Accumulation
Case Study: how Concrete got to where we are today
• A single platform to manage projects, plans,
people, content, all from one place in the cloud.
• Enterprise collaboratio...
• First client
• Need a picture library
In the beginning
• How do you create two new client
sites?
• Easy
• Ctrl+C, Ctrl+V
In the beginning
• A client needs feature X
• A new client signs and requires
feature X
Adding Features
• 2010 - 10 clients sites
• New client = new developer
• Cross selling features
• Bug fixes?
• 12 million lines of code!
Sc...
What we have today
• 30 production sites & 30 test sites
• Monthly releases
• Full LNSR HA stack
• Code deployed in sub 1 ...
What we have today
• 9 new applications
• A vision
• A product owner
• An architect
What we have today
0
10
20
30
40
50
60
70
80
90
2005-04
2005-07
2005-10
2006-01
2006-04
2006-07
2006-10
2007-01
2007-04
20...
Debt Reduction
what we did to enable us to grow
How did we change
• Dev sites
• SubVersion
• Continuous Delivery
Continuous Delivery
Continuous Delivery
Continuous Delivery
• SVN post commit hooks
• Windows batch file & scheduled task
• Jenkins
Continuous Delivery
Consolidated Codebase
• 10 clients
• 6 applications
• 12,000,000 lines of code
Consolidated Codebase
• 30 clients
• 12 applications
• 72,000,000 million lines of code (estimate)
• 800,000 (actual)
Consolidated Codebase
• New framework
• Moved ‘core’ applications to a new repository
• ‘Core’ deployed independently
• In...
Consolidated Codebase
• 3 new clients
• Usual copy and paste or…
• ConcretePlatform v2
Consolidated Codebase
• Merged ‘core’ back into a site
• Manually diff databases
• Merged everything on deployment
• Still...
Consolidated Codebase
‘CP2’
New Clients
‘Coring’
Consolidated Codebase
• True single codebase (so close!)
• Deployed 10 times
• Each instance can run any client site
• Con...
404 Handlers
• Tidy up static assets
• Enabled migration to Railo
404 Handlers
• <cfdirectory>
• QoQ
• Redirect or fail
Keeping dev sites up to date
• Code
• Database
• Configuration
Release Process
• Monthly releases
• Run pre_core.sql
• Run pre_client.sql for each client db
• Deploy code (anything up t...
Database Rebuilds
• Take back up from live
• Copy backup to dev server
• Drop the dev db
• Restore the backup from live
• ...
Database Rebuilds
Local Development
• Self installed + Configured
• Virtualbox Image shared
• Vagrant + Puppet
Monitoring and Metrics
• Statsd & Graphite
• Make it easy
• <cfset statsd.increment(“app.sessions.expired”) />
• Think abo...
Monitoring and Metrics
• Logstash and Kibana
• Make it easy
Monitoring and Metrics
file {

type => "statsD"

path => "/var/www/*/WEB-INF/railo/logs/statsd.log"

}
grok { 

type => “s...
Monitoring and Metrics
So what’s next?
More to do
• Decouple our backend services from the
databases
• Decouple the front end applications from the
asset storage...
Codebase VCS Deployment Monitoring Logging
Crawl Individual _old FTP Nagios <cfmail
Walk ‘Core’ SubVersion Batch files
Ping...
Conclusion
• Technical debt isn’t bad
• Debt can give you a quicker time to market
• There isn’t a silver bullet
• You hav...
Thanks
• clarkie@concrete.cc
• http://clarkie.io
• @clarkieclarkie
Upcoming SlideShare
Loading in …5
×

Technical Debt - SOTR14 - Clarkie

907 views

Published on

Technical Debt - Is your codebase approaching the fiscal cliff?

Published in: Technology
  • Be the first to comment

Technical Debt - SOTR14 - Clarkie

  1. 1. Technical Debt Is your codebase approaching the fiscal cliff? Andy Clarke clarkie@concrete.cc http://clarkie.io @clarkieclarkie
  2. 2. This is me • Software Engineer for 8 years • Never worked on a greenfield project (at work) • Technical Architect @ Concrete
  3. 3. Outline • What is Technical Debt? • Debt Accumulation • Debt Reduction • Q + A
  4. 4. – Ward Cunningham, OOPSLA 1992 “Shipping first time code is like going into debt; ! a little debt speeds development so long as it is paid back promptly with a rewrite.”
  5. 5. – Intel, 2009 “Technical Debt is loosely defined as the volume of poorly written lines of code, poorly refactored, not following coding standards, not supported with sufficient unit tests, and the amount of code duplication.”
  6. 6. – Michael (Doc) Norton, 2009 “Messy Code is not Technical Debt”
  7. 7. Technical Debt Is your code base project a ‘bag of spanners’? Andy Clarke clarkie@concrete.cc http://clarkie.io @clarkieclarkie
  8. 8. Where does it come from? • No vision • No design • Operational • Experimentation
  9. 9. Where to keep your debt • Code • Persistent storage • Architecture • Infrastructure
  10. 10. Debt Accumulation Case Study: how Concrete got to where we are today
  11. 11. • A single platform to manage projects, plans, people, content, all from one place in the cloud. • Enterprise collaboration What is Concrete today?
  12. 12. • First client • Need a picture library In the beginning
  13. 13. • How do you create two new client sites? • Easy • Ctrl+C, Ctrl+V In the beginning
  14. 14. • A client needs feature X • A new client signs and requires feature X Adding Features
  15. 15. • 2010 - 10 clients sites • New client = new developer • Cross selling features • Bug fixes? • 12 million lines of code! Scaling
  16. 16. What we have today • 30 production sites & 30 test sites • Monthly releases • Full LNSR HA stack • Code deployed in sub 1 minute • An understanding of our debt
  17. 17. What we have today • 9 new applications • A vision • A product owner • An architect
  18. 18. What we have today 0 10 20 30 40 50 60 70 80 90 2005-04 2005-07 2005-10 2006-01 2006-04 2006-07 2006-10 2007-01 2007-04 2007-07 2007-10 2008-01 2008-04 2008-07 2008-10 2009-01 2009-04 2009-07 2009-10 2010-01 2010-04 2010-07 2010-10 2011-01 2011-04 2011-07 2011-10 2012-01 2012-04 2012-07 2012-10 2013-01 2013-04 2013-07 2013-10 2014-01 2014-04 Thousands Total
  19. 19. Debt Reduction what we did to enable us to grow
  20. 20. How did we change • Dev sites • SubVersion • Continuous Delivery
  21. 21. Continuous Delivery
  22. 22. Continuous Delivery
  23. 23. Continuous Delivery • SVN post commit hooks • Windows batch file & scheduled task • Jenkins
  24. 24. Continuous Delivery
  25. 25. Consolidated Codebase • 10 clients • 6 applications • 12,000,000 lines of code
  26. 26. Consolidated Codebase • 30 clients • 12 applications • 72,000,000 million lines of code (estimate) • 800,000 (actual)
  27. 27. Consolidated Codebase • New framework • Moved ‘core’ applications to a new repository • ‘Core’ deployed independently • Integration issues • Shared database adds complexity (more debt)
  28. 28. Consolidated Codebase • 3 new clients • Usual copy and paste or… • ConcretePlatform v2
  29. 29. Consolidated Codebase • Merged ‘core’ back into a site • Manually diff databases • Merged everything on deployment • Still deploying 60 copies of code base • Deployments could take up to 1 hour
  30. 30. Consolidated Codebase ‘CP2’ New Clients ‘Coring’
  31. 31. Consolidated Codebase • True single codebase (so close!) • Deployed 10 times • Each instance can run any client site • Configuration driven • Designed at SOTR 2013!
  32. 32. 404 Handlers • Tidy up static assets • Enabled migration to Railo
  33. 33. 404 Handlers • <cfdirectory> • QoQ • Redirect or fail
  34. 34. Keeping dev sites up to date • Code • Database • Configuration
  35. 35. Release Process • Monthly releases • Run pre_core.sql • Run pre_client.sql for each client db • Deploy code (anything up to 30 minutes) • Run post_core.sql • Run post_client.sql for each client db
  36. 36. Database Rebuilds • Take back up from live • Copy backup to dev server • Drop the dev db • Restore the backup from live • ‘Massage’ the data • Perform the release • Every night
  37. 37. Database Rebuilds
  38. 38. Local Development • Self installed + Configured • Virtualbox Image shared • Vagrant + Puppet
  39. 39. Monitoring and Metrics • Statsd & Graphite • Make it easy • <cfset statsd.increment(“app.sessions.expired”) /> • Think about your metric structure
  40. 40. Monitoring and Metrics • Logstash and Kibana • Make it easy
  41. 41. Monitoring and Metrics file {
 type => "statsD"
 path => "/var/www/*/WEB-INF/railo/logs/statsd.log"
 } grok { 
 type => “statsD"
 match => [ "message", 
 ‘"%{DATA:level}"
 ,"%{DATA:thread}"
 ,"%{DATA:date}"
 ,"%{DATA:time}"
 ,”%{DATA:application}"
 ,"%{GREEDYDATA:app_message}"']
 }
  42. 42. Monitoring and Metrics
  43. 43. So what’s next?
  44. 44. More to do • Decouple our backend services from the databases • Decouple the front end applications from the asset storage • Consolidate our web servers • Improve application usage metrics • Unit testing - thanks Kev! • Migrate to AWS
  45. 45. Codebase VCS Deployment Monitoring Logging Crawl Individual _old FTP Nagios <cfmail Walk ‘Core’ SubVersion Batch files Pingdom + PagerDuty <cflog Run 1 Git Jenkins Cedexis + Statsd Kibana Baby steps
  46. 46. Conclusion • Technical debt isn’t bad • Debt can give you a quicker time to market • There isn’t a silver bullet • You have to fight your corner • It’s worth paying it off
  47. 47. Thanks • clarkie@concrete.cc • http://clarkie.io • @clarkieclarkie

×