Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evolution of Our Software Architecture

267 views

Published on

Evolution of Our Software Architecture

Published in: Software
  • Be the first to comment

Evolution of Our Software Architecture

  1. 1. EVOLUTION OF OUR SOFTWARE ARCHITECTURE Presented at Aarhus Clojure Meetup, June 2017 by Paul Lam (@Quantisan),VP Engineering, Motiva
  2. 2. A BRAIN FOR CAMPAIGN • Motiva AI is a messaging platform that optimizes messaging campaigns and improves targeting, conversions, and engagement - automatically
  3. 3. – https://en.wikipedia.org/wiki/Software_architecture “Software architecture refers to the high level structures of a software system, the discipline of creating such structures, and the documentation of these structures.”
  4. 4. ARCHITECTURE EVOLUTION • How can a system be adapted to support uncertain business goals? • How resilient is the system to planned and unplanned change? • How can the quality of a deficient architecture be improved? http://www.sei.cmu.edu/architecture/research/previousresearch/evolution.cfm
  5. 5. WHY NOT BUILD FROM SCRATCH EVERYTIME? Cost Percentage of Architecture Re-use But! https://martinfowler.com/bliki/SacrificialArchitecture.html
  6. 6. • LAYERED • MICROSERVICES • EVENT-DRIVEN • MICROKERNEL • SPACE-BASED
  7. 7. JUST-IN-TIME DEVELOPMENT
  8. 8. LAYERED MONOLITH
  9. 9. Database Frontend Business Logic Persistence Presentation
  10. 10. D F Bus Per Pre
  11. 11. D F Bus Per Pre TESTING
  12. 12. D F Bus Per Pre DEPLOY
  13. 13. MACHINE LEARNING? D F Bus Per Pre
  14. 14. COUNTING WITH CLOJURED F Bus Per Pre https://github.com/nathanmarz/specter
  15. 15. INTERACTIVE DEVELOPMENTD F Bus Per Pre
  16. 16. LAYERED MONOLITH • 3 months from nothing to production • Easy to test • Easy to deploy • Not one-size fit all • Easy to get complicated D F Bus Per Pre
  17. 17. MICRO-SERVICES
  18. 18. OUR PROBLEMS 1. Machine learning research to production process is disconnected and duplicated 2. Interface points with external APIs are complicated and fragile
  19. 19. MACHINE LEARNING AS AN API
  20. 20. CODE SHARING: RESEARCH AND PRODUCTION Jupyter Notebook $ pip install burt import burt b = burt.Burt(coll) b.getExperimentBatch() ... Production Server $ pip install burt import burt b = burt.Burt(coll) b.getExperimentBatch() ...
  21. 21. Database Frontend Business Logic Persistence Presentation
  22. 22. Databases Frontend Business Logic Persistence Presentation Machine Learning Persistence External API Persistence
  23. 23. MICRO-SERVICES • Using the right stack for the right task • Easy to change • Easy to develop • Design overhead • Multi-step orchestration getting tedious
  24. 24. EVENT-DRIVEN
  25. 25. NEW PROBLEM • Increasing number of business logic steps • Introduce self-recovery on flaky steps
  26. 26. Databases Frontend Business Logic Persistence Presentation Machine Learning Persistence External API Persistence
  27. 27. Frontend Event Mediator Presentation Machine Learning Persistence External API Persistence Event 2 Processor Persistence Event 3 Processor Persistence Event 1 Processor
  28. 28. EVENT MEDIATOR
  29. 29. DEAD LETTER QUEUE FOR RECOVERY Sender Worker Queue Task failed Dead letter queue Intended Receiver
  30. 30. EVENT-DRIVEN • Robust • Scalable • Easy to change • Integration tests not easy • Not easy to develop
  31. 31. SUMMARY • Evolve system with just-in-time development • Start simple and let demand drive architecture • Divide and conquer
  32. 32. CREDITS • Jonathan Irving, github.com/j0ni • Matt Hill, @areamatt • Chris Diehl, @chrisdiehl
  33. 33. CONTACT Paul Lam @Quantisan github.com/Quantisan paul@motiva.ai

×