Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Zero to scaleable
in ten minutes
Less than obvious tools & tips for writing software that lasts
Matt Walters
@mateodelnort...
What do you mean… ‘scaleable’?
As businesses grow, their understanding of
problems they’re solving also grows
Scaleable = ...
Combining standard, available tools, and repeating
simple, predictable patterns during development will
keep your code und...
gitslave
Why have a monolith, when you can have a
metarepo?
> man gits
Gitslave Home Page: <http://gitslave.sf.net>
NAME
g...
gitslave
Why have a monolith, when you can have a
metarepo?
gitslave allows you to have all of your code in one location,
...
gitslave
Why have a monolith, when you can have a
metarepo?
> ls -la
drwxr-xr-x 18 mateodelnorte staff 612 Jul 14 00:52 .
...
gitslave
Why have a monolith, when you can have a
metarepo?
> cat .gitignore
/denormalizer/
/my-service-one/
/my-service-t...
gitslave
Why have a monolith, when you can have a
metarepo?
> cat .gitslave
"../denormalizer" “denormalizer"
“../my-servic...
gitslave
brew install gitslave
tmuxinator
Heads up. You need a good view of your
system.
tmuxinator
.yml-based config-driven tmux. that’s it.
name: my-system
root: ./
windows:
- site:
layout: tiled
panes:
- my-s...
tmuxinator
Heads up. You need a good view of your
system.
Panes per process
tmuxinator
Heads up. You need a good view of your
system.
Windows per context, business domain, etc
tmuxinator
gem install tmuxinator
Neat whats next?
We’ve talked about where you’ll house your project, and
the different sub-projects within it.
(hint: one ...
Why messaging?
Your processes need an interface
Why messaging?
Your processes need an interface
REST
Why messaging?
Your processes need an interface
REST
?What happens when a process dies?
Why messaging?
Your processes need an interface
REST
??
What happens when a process dies?
Why messaging?
Your processes need an interface
Why messaging?
Your processes need an interface
Shared DB
Why messaging?
Your processes need an interface
Shared Spaghetti
Doable, but usually unclear which process ‘owns’ database...
Why messaging?
Your processes need an interface
Why messaging?
Your processes need an interface
Queues
Why messaging?
Your processes need an interface
Queues
?
Processes go down with queues…
Why messaging?
Your processes need an interface
Queues
And come right back up with no message loss.
Why messaging?
Your system gets a safety net
Queues
?x5Added benefit:
Planned human
intervention for
errors.
rabbitmq
messaging that just works
• direct send to queue
• fanout / topic routing
• highly available
• highly performant
...
rabbitmq
messaging that just works
brew install rabbitmq
servicebus
super simple messaging in node
• direct send
• pub / sub / fanout / topic -routing
• simple to set up
• highly ...
servicebus
super simple messaging in node
npm install servicebus —save
servicebus
super simple send+listen messaging in node
// process 1
var bus = require('servicebus').bus();
bus.listen('my.e...
servicebus
super simple pub+sub messaging in node
// process 1
var bus = require('servicebus').bus();
bus.subscribe('my.ev...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘servicebus').bus();
const c...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘servicebus’).bus();
const r...
(micro)servicebus
some simple patterns get us going
// order-svc refactored bus to ./lib/bus.js
const bus = require(‘servi...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
// moved bus initi...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
// moved bus initi...
commands tell services
when an actor wants an action
client send commands to instruct a service to do work
commands are se...
events tell the world
when you’re done
services publish when done performing work
services publish messages to any service...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
// moved bus initi...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
// moved bus initi...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
// moved bus initi...
(micro)servicebus
some simple patterns get us going
// order-svc index.js
const bus = require(‘./bus’);
const log = requir...
Show the stuff!
quick show and tell
Let’s recap!
‘cause there was a lot!
1. gitslave lets you structure a project like a monolith,
but have the convenience of...
Thanks!
We’ll cover more on event-driven
architecture, microsvc, CQRS, and
more in a future talk!
Zero to scaleable
in ten minutes
Matt Walters
github & twitter: @mateodelnorte
email: mattwalters5@gmail.com
https://githu...
Upcoming SlideShare
Loading in …5
×

Zero to scaleable in ten minutes

224 views

Published on

Less than obvious tools & tips for writing software that lasts

Published in: Technology
  • Be the first to comment

Zero to scaleable in ten minutes

  1. 1. Zero to scaleable in ten minutes Less than obvious tools & tips for writing software that lasts Matt Walters @mateodelnorte mattwalters5@gmail.com
  2. 2. What do you mean… ‘scaleable’? As businesses grow, their understanding of problems they’re solving also grows Scaleable = easy to understand, maintain, and add to During that growth, software changes. Good software is easiest to change, while still getting things done.
  3. 3. Combining standard, available tools, and repeating simple, predictable patterns during development will keep your code understandable, maintainable, and easily improved. Our toolkit will be: • gitslave • tmuxinator • rabbitmq • servicebus *and some very simple patterns How do we scale an architecture?
  4. 4. gitslave Why have a monolith, when you can have a metarepo? > man gits Gitslave Home Page: <http://gitslave.sf.net> NAME gits - The git slave repository tool for multi-repository management SYNOPSIS gits [-p|--parallel COUNT] [-v|--verbose]+ [--quiet] [--help] [--version] [-n|--no-pager] [--paginate] [--eval-args] [--exclude SLAVE-REGEXP] [--keep-going] [--no-commit] [--no-hide] [--no-progress] [--no-master] [--with-ifpresent|--just-ifpresent] SUBCOMMAND [ARGS]... OVERVIEW gits is a program that assists in assembling a meta-project from a number of individual git repositories which operate (when using gits) as if they were one git repository instead of many, similar to the way that CVS works by default and svn (v1.5) can be coerced to work. Some of these individual git repositories may be part of other meta- projects as well.
  5. 5. gitslave Why have a monolith, when you can have a metarepo? gitslave allows you to have all of your code in one location, like a monolithic app, while allowing fine grained updates of each of your solution’s component projects. one ‘gits pull origin master’ gets all new changes, for all related projects / repositories, while you can still perform fine grained operations on any particular repo.
  6. 6. gitslave Why have a monolith, when you can have a metarepo? > ls -la drwxr-xr-x 18 mateodelnorte staff 612 Jul 14 00:52 . drwxr-xr-x 77 mateodelnorte staff 2618 Jul 19 23:01 .. -rw-r--r-- 1 mateodelnorte staff 411 Jul 12 22:22 .eslintrc.js drwxr-xr-x 12 mateodelnorte staff 408 Jul 20 11:34 .git -rw-r--r-- 1 mateodelnorte staff 59 Jul 14 00:25 .gitignore -rw-r--r-- 1 mateodelnorte staff 127 Jul 14 00:26 .gitslave -rw-r--r-- 1 mateodelnorte staff 2849 Jul 5 15:56 Makefile drwxr-xr-x 16 mateodelnorte staff 544 Jul 14 01:18 denormalizer drwxr-xr-x 16 mateodelnorte staff 544 Jul 10 17:51 my-service-one drwxr-xr-x 18 mateodelnorte staff 612 Jul 14 00:39 my-service-two drwxr-xr-x 18 mateodelnorte staff 612 Jul 14 00:39 my-site -rw-r--r-- 1 mateodelnorte staff 1184 Jul 19 23:32 tmuxinator.all.yml
  7. 7. gitslave Why have a monolith, when you can have a metarepo? > cat .gitignore /denormalizer/ /my-service-one/ /my-service-two/ /my-site/
  8. 8. gitslave Why have a monolith, when you can have a metarepo? > cat .gitslave "../denormalizer" “denormalizer" “../my-service-one“ “my-service-one“ "../my-service-two" “my-service-two“ “../my-site" “my-site"
  9. 9. gitslave brew install gitslave
  10. 10. tmuxinator Heads up. You need a good view of your system.
  11. 11. tmuxinator .yml-based config-driven tmux. that’s it. name: my-system root: ./ windows: - site: layout: tiled panes: - my-site: - printf ‘033]2;my-site033' - cd ./my-site - make run - services: layout: tiled panes: - my-service-one: - printf '033]2;my-service-one033' - cd ./my-service-one - make run - my-service-two: - printf '033]2;my-service-two033' - cd ./my-service-two =
  12. 12. tmuxinator Heads up. You need a good view of your system. Panes per process
  13. 13. tmuxinator Heads up. You need a good view of your system. Windows per context, business domain, etc
  14. 14. tmuxinator gem install tmuxinator
  15. 15. Neat whats next? We’ve talked about where you’ll house your project, and the different sub-projects within it. (hint: one gitslave metarepo and n project repos) Let’s talk a bit about messaging. And we’ve talked about how you’ll stare at it while it’s on your screen. (hint: tmuxinator)
  16. 16. Why messaging? Your processes need an interface
  17. 17. Why messaging? Your processes need an interface REST
  18. 18. Why messaging? Your processes need an interface REST ?What happens when a process dies?
  19. 19. Why messaging? Your processes need an interface REST ?? What happens when a process dies?
  20. 20. Why messaging? Your processes need an interface
  21. 21. Why messaging? Your processes need an interface Shared DB
  22. 22. Why messaging? Your processes need an interface Shared Spaghetti Doable, but usually unclear which process ‘owns’ database structure and data. Can lead to maintenance hell.
  23. 23. Why messaging? Your processes need an interface
  24. 24. Why messaging? Your processes need an interface Queues
  25. 25. Why messaging? Your processes need an interface Queues ? Processes go down with queues…
  26. 26. Why messaging? Your processes need an interface Queues And come right back up with no message loss.
  27. 27. Why messaging? Your system gets a safety net Queues ?x5Added benefit: Planned human intervention for errors.
  28. 28. rabbitmq messaging that just works • direct send to queue • fanout / topic routing • highly available • highly performant • used in financial exchanges, industrial applications and more • open source • free
  29. 29. rabbitmq messaging that just works brew install rabbitmq
  30. 30. servicebus super simple messaging in node • direct send • pub / sub / fanout / topic -routing • simple to set up • highly performant • used in financial exchanges, online advertising and more • open source • free • perfect for creating microservices!
  31. 31. servicebus super simple messaging in node npm install servicebus —save
  32. 32. servicebus super simple send+listen messaging in node // process 1 var bus = require('servicebus').bus(); bus.listen('my.event', function (event) { console.log(event); }); // process 2 var bus = require('servicebus').bus(); setInterval(function () { bus.send('my.event', { my: 'event' }); }, 1000);
  33. 33. servicebus super simple pub+sub messaging in node // process 1 var bus = require('servicebus').bus(); bus.subscribe('my.event', function (event) { console.log(event); }); // process 2 var bus = require('servicebus').bus(); setInterval(function () { bus.publish('my.event', { my: 'event' }); }, 1000); // process 3
  34. 34. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘servicebus').bus(); const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event); });
  35. 35. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘servicebus’).bus(); const retry = require(‘servicebus-retry’); bus.use(retry({ store: new retry.RedisStore({ host: process.env.REDIS_HOST, port: process.env.REDIS_PORT }) })); const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event, (err) => { if (err) return event.handle.reject(); event.handle.ack(); }); });
  36. 36. (micro)servicebus some simple patterns get us going // order-svc refactored bus to ./lib/bus.js const bus = require(‘servicebus’).bus(); const retry = require(‘servicebus-retry’); bus.use(retry({ store: new retry.RedisStore({ host: process.env.REDIS_HOST, port: process.env.REDIS_PORT }) })); module.exports = bus;
  37. 37. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); // moved bus initialization to own module const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event, (err) => { if (err) return event.handle.reject(); event.handle.ack(); }); });
  38. 38. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); // moved bus initialization to own module const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.created’, order, () => { event.handle.ack(); }); }); }); service publishes to the world when it’s done!
  39. 39. commands tell services when an actor wants an action client send commands to instruct a service to do work commands are sent async, fire and forget commands are sent directives: order.create web app order-svc order.create!
  40. 40. events tell the world when you’re done services publish when done performing work services publish messages to any services that wish to listen events are sent past-tense: order.created order-svc fulfillment-svc order.created! order.created!
  41. 41. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); // moved bus initialization to own module const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.created’, order, () => { event.handle.ack(); }); }); });
  42. 42. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); // moved bus initialization to own module const cancel = require(‘./lib/cancel’); const create = require(‘./lib/create’); bus.listen(‘order.create', function (event) { create(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.created’, order); bus.publish(‘order.created’, order, () => { event.handle.ack(); }); }); }); bus.listen(‘order.cancel', function (event) { cancel(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.canceled’, order); bus.publish(‘order.created’, order, () => { event.handle.ack(); }); }); code buildup increases cognitive load
  43. 43. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); // moved bus initialization to own module require(‘./lib/handlers/orderCanceled’); require(‘./lib/handlers/orderCreated’); // moved handlers to their own modules, // hiding their complexity from other code
  44. 44. (micro)servicebus some simple patterns get us going // order-svc index.js const bus = require(‘./bus’); const log = require(‘llog’); const registerHandler = require(‘servicebus-register-handlers’); registerHandlers({ bus, handleError: function handleError (msg, err) { log.error(`error handling ${msg.type}: ${err}. rejecting message` + `w/cid ${msg.cid} and correlationId ${this.correlationId}.`); log.fatal(err); msg.handle.reject(() => { bus.close(); process.exit(1); }); }, path: ‘./lib/handlers', // handlers are now one listen or subscribe per file queuePrefix: 'order-svc', });
  45. 45. Show the stuff! quick show and tell
  46. 46. Let’s recap! ‘cause there was a lot! 1. gitslave lets you structure a project like a monolith, but have the convenience of many repositories 2. tmuxinator gives you a head’s up view of your system, all running locally 3. messaging provides a durable, convenient interface between your processes 4. rabbitmq + servicebus enable easy send+listen & pub +sub messaging patterns between processes 5. processes send commands and publish events 6. simple, expected, structure in your services keeps cognitive load low, and projects easy to maintain
  47. 47. Thanks! We’ll cover more on event-driven architecture, microsvc, CQRS, and more in a future talk!
  48. 48. Zero to scaleable in ten minutes Matt Walters github & twitter: @mateodelnorte email: mattwalters5@gmail.com https://github.com/tmuxinator/tmuxinator rabbitmq.com http://gitslave.sourceforge.net/ https://www.npmjs.com/package/servicebus

×