Hacking the Monolith
A Case Study
SVT interaktiv - JFokus 2015
Gereon Kåver
@gereonk Rickard Andersson
Keep the bar low
2009
2011
2014TEAM 2015
Core
Video Core
≈120 man-year code
New
Architecture!
Culture
Autonomy
Opt-in
Free vs standardized tech choices
Custom API:s vs standardized API:s
Common operations, infrastructure, logging,…
Reusability vs versatility
Distributed vs centralized ownership
…
Free over standardized tech choices
Custom API:s over standardized API:s
Recommended over common operations, infra,
logging
Versatility over reusability
Distributed over centralized ownership
…
Microservice Architecture
1. Easier to create team
autonomy
2. Simplify scaling and
maintenance
3. Improve resilience and
monitoring
4. Make it more fun to develop
Why micro
services?
“Create greater ownership” - LinkedIn
Give the team full control, no central
rules
Create a platform
1. You should (from a newly installed computer)
be able to modify and deploy an existing
project to production in less than 6 hours.
2. It should take less than two hours to create a
new service with CI, monitoring, logging.
Automated deploy at SVT
Jenkins deploys to Heroku or internal to
“Molnet”
Automated deploy at SVT - “Molnet”
$> moln app create crash-course
$> moln app image set crash-course /hello-world:1.0
$> moln app targets set crash-course
sto.molnet.svt.se=stomoln01-agent01
drs.molnet.svt.se=drsmoln01-agent01
$> moln app deploy crash-course:v1
$> moln app undeploy crash-course:v1
Automated deploy at SVT - “Molnet”
- Docker. Packaging and running services
- Helios. Orchestration of services.
- Consul. System for configuration and service
discovery.
Automated deploy at SVT - “Molnet”
Open source: https://github.com/SVT/helios-consul
Micro services gives (new)
requirements
1.Automated deploy - Contious
Integration
2.Monitoring
3.Resilience
Resilience at
SVT
Hystrix from Netflix.
1.Isolate network interaction using Circuit-
Breakers, BulkRequests, Timeouts
2.Fallbacks and graceful degradation
3.Metrics. req/sec, failures, responsetimes
https://github.com/Netflix/Hystrix
SVT services
API-guidelines at SVT
1.We do not violate HTTP 1.1
2.We uses rest, Richardson Maturity Model
up to and including level 2
3.All API:s uses swagger for documentations.
(swagger.io)
4.All clients adds ‘user-agent’ to create
traceability
5.We uses CORS (not JSONP)
Keep the bar low
Follow us on blogg.svt.se/testbild
Gereon Kåver
gereon.kaver@svt.se
@gereonk
Rickard Andersson
rickard.andersson@svt.se

JFokus 2015 - Hacking the-monolith

  • 1.
    Hacking the Monolith ACase Study SVT interaktiv - JFokus 2015 Gereon Kåver @gereonk Rickard Andersson
  • 2.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
    Free vs standardizedtech choices Custom API:s vs standardized API:s Common operations, infrastructure, logging,… Reusability vs versatility Distributed vs centralized ownership …
  • 11.
    Free over standardizedtech choices Custom API:s over standardized API:s Recommended over common operations, infra, logging Versatility over reusability Distributed over centralized ownership …
  • 12.
  • 13.
    1. Easier tocreate team autonomy 2. Simplify scaling and maintenance 3. Improve resilience and monitoring 4. Make it more fun to develop Why micro services?
  • 14.
    “Create greater ownership”- LinkedIn Give the team full control, no central rules
  • 15.
    Create a platform 1.You should (from a newly installed computer) be able to modify and deploy an existing project to production in less than 6 hours. 2. It should take less than two hours to create a new service with CI, monitoring, logging.
  • 16.
    Automated deploy atSVT Jenkins deploys to Heroku or internal to “Molnet”
  • 17.
    Automated deploy atSVT - “Molnet” $> moln app create crash-course $> moln app image set crash-course /hello-world:1.0 $> moln app targets set crash-course sto.molnet.svt.se=stomoln01-agent01 drs.molnet.svt.se=drsmoln01-agent01 $> moln app deploy crash-course:v1 $> moln app undeploy crash-course:v1
  • 18.
    Automated deploy atSVT - “Molnet” - Docker. Packaging and running services - Helios. Orchestration of services. - Consul. System for configuration and service discovery.
  • 19.
    Automated deploy atSVT - “Molnet” Open source: https://github.com/SVT/helios-consul
  • 20.
    Micro services gives(new) requirements 1.Automated deploy - Contious Integration 2.Monitoring 3.Resilience
  • 21.
    Resilience at SVT Hystrix fromNetflix. 1.Isolate network interaction using Circuit- Breakers, BulkRequests, Timeouts 2.Fallbacks and graceful degradation 3.Metrics. req/sec, failures, responsetimes https://github.com/Netflix/Hystrix
  • 24.
  • 25.
    API-guidelines at SVT 1.Wedo not violate HTTP 1.1 2.We uses rest, Richardson Maturity Model up to and including level 2 3.All API:s uses swagger for documentations. (swagger.io) 4.All clients adds ‘user-agent’ to create traceability 5.We uses CORS (not JSONP)
  • 26.
  • 27.
    Follow us onblogg.svt.se/testbild Gereon Kåver gereon.kaver@svt.se @gereonk Rickard Andersson rickard.andersson@svt.se

Editor's Notes

  • #2 Glad to be here development manager, lead architect In the process of moving to micro architecture experiences from that yourney
  • #3 If you have to leave early, this is the key takeaway make it easy to start - to create and deploy new services
  • #5 fast growth less people use broadcast - more watch on-demand The choices we made 2010 is not valid now
  • #6 You have all been here Lot of dependencies Hard to find errors Vague ownership Heavy deploy
  • #7 [TA BORT?]
  • #9 teams have a lot of freedom choosing language and platform high mandate and responsibility Eat your own dogshit everyone should understand the objectives Design Studio
  • #10 Sam Newman compares the architect role with a city planner
  • #11 choices before moving forward
  • #12 removed versus There is no contradiction
  • #14 So why change the architecture to micro services. These are our four main reasons. Micro services are built by Single Responsibility Principle. It easy to make a team owns a micro service as a whole. All micro services should run it its own process. Therefore easier to scale and deploy. Micro services forces the developer to think about resilience and monitoring. 4. Micro services gives more opportunities to try out new frameworks and test ideas. We think a happy developer creates better code and services.
  • #15 In order to scale up and deliver working software, we needed the teams to take responsibility for their services as a whole. Know the domain, taking technical decisions, handling deploy and maintenance. As an architect, my role is to make it easy for the team to take this responsibility. What LinkedIn calls ”Create greater ownership". Lowering the bar to make the “right thing” by creating tools and good examples. We tries to get as as close as possible to the extreme "no central rules”.
  • #16 Last fall we started a project to create a platform and deploy pipeline to make it easy to create new services. These where our goals… Lowering the bar to create a new service instead of extend an existing service with functions that does not belong there. That is the way to create a badly designed monolith.
  • #17 Our solution where to use Jenkins to deploy to Heroku or to an internal environment called “Molnet” Primary we uses Heroku, but for performance och legal reasons we uses “Molnet”
  • #18 “Molnet” got a similar DSL as Heroku, all to make it easier for the developers to use them.
  • #19 For automated deploys you need 3 things: Orchestration - Deploy a service to the environment Configuration - Handle configurations through different environments. Service Discovery - Keep track of where a specific service is running (ip and port).
  • #20 The connection between helios and consul is a SVT written plugin. Open sourced at.
  • #21 These are not really new requirements, but we think they need extra attendance. 10 or a 100 services need to be build and deployed when needed. Solved by Heroku and “Molnet” You have to be able to see how the system behaves, as a whole and as individual parts. Logging, key-value pairs via stdout -> an indexing service like elastic search Metrics (cpu, threads, response times). NewRelic and Graphite. The system should be able to work even if one part of the system responds slowly or is down. No domino effects.
  • #22 Hystrix a library from Netflix We uses the following to create resilience. Databases and other services. Default values, caching or other fallback solutions.
  • #23 This produces metrics to create dashboards like these… NewRelic, Jenkins, Hystrix and collectd
  • #24 How do we keep track of all services? All projects must include a description file (svtservice.yml). We scan our repositories and creates a list of all the services we have. Includes description, product owner and responsible team.
  • #25 The yml-files also contains dependencies, which makes it possible to create graphs like these.
  • #26 And we do have some central rules. But we try to not tie up the teams. To build a micro service API at SVT, you need to stick to these rules. Part from them the team could do what ever they like.
  • #27 Make it easy to do the right things Let the team do the rest.