Services Inception in Ruby, presented on Friday Sept. 30th @ RubyConf US 2011. Covering Metaprogramming and other options for building and consuming Cloud services from inside of Apps deployed to PaaS clouds. Download from GitHub @ https://github.com/mccrory/ServicesInception-RubyConf-US-2011

1. Services

2. Dave McCrory
@mccrory on Twitter
Senior Architect, Cloud Foundry, VMware

3. Wordpress?

4. Facebook apps on Heroku: 34,000 in 24 Hours
How To:
curl ‐u “$EMAIL:$PASSWORD” ‐X POST https://api.heroku.com/apps

5. PaaS Provider Your App
MySQL

6. 1
PaaS Provider Your App
MySQL

7. 1
PaaS Provider Your App
2
MySQL

8. 1
PaaS Provider Your App
3
2
MySQL

9. Setting the Stage
“The Dream State”
• This is specifically targeted at PaaS
(Platform as a Service)
• All providers have an interface (CLI
or otherwise) to control services in
their platform

10. -Cobb

11. “...Inception, now before you bother
telling me it's impossible...”
-Cobb

12. -Eames

13. “No, it's perfectly possible. It's just
bloody difficult.”
-Eames

14. Dependencies
• Most Apps Have Them...
• I’m not just talking about
• require ‘rubygems’
• Bundler
• etc...

15. I’m talking about
Service Dependencies
• “Assumes Redis”
• “Built for MongoDB”
• “Works with MySQL, other support
coming”
• “Requires/Assumes X Service”

16. YOU need to go setup
that Service...
You need
• Resources
• the bits for the service (the correct version)
• and YOU must configure it
• and in some cases you must install it

17. An Example from Today

18. https://github.com/defunkt/resque
Resque is our Redis-backed library for creating
background jobs, placing those jobs on multiple queues,
and processing them later.

19. https://github.com/defunkt/resque
Resque is our Redis-backed library for creating
background jobs, placing those jobs on multiple queues,
and processing them later.
Installing Redis
Resque requires Redis 0.900 or higher.

20. https://github.com/defunkt/resque
Resque is our Redis-backed library for creating
background jobs, placing those jobs on multiple queues,
and processing them later.
Installing Redis
Resque requires Redis 0.900 or higher.
Resque Dependencies
gem install redis redis-namespace yajl-ruby

21. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.

22. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.
Resque-mongo uses a fairly new feature of mongo, findAndModify().
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.

23. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.
Resque-mongo uses a fairly new feature of mongo, findAndModify().
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.
Also, check your mongo version: 1.3.0 or higher

24. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.
Resque-mongo uses a fairly new feature of mongo, findAndModify().
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.
Also, check your mongo version: 1.3.0 or higher
Mysql::Error: MySQL server has gone away
If your workers remain idle for too long they may lose their MySQL connection.

25. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.
Resque-mongo uses a fairly new feature of mongo, findAndModify().
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.
Also, check your mongo version: 1.3.0 or higher
Mysql::Error: MySQL server has gone away
If your workers remain idle for too long they may lose their MySQL connection.
Installing Redis
Resque requires Redis 0.900 or higher.

26. https://github.com/wistia/resque-mongo
Resque-mongo is a fork of [Resque][resque] that uses
MongoDB as a backend instead of Redis.
Resque-mongo uses a fairly new feature of mongo, findAndModify().
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.
Also, check your mongo version: 1.3.0 or higher
Mysql::Error: MySQL server has gone away
If your workers remain idle for too long they may lose their MySQL connection.
Installing Redis
Resque requires Redis 0.900 or higher.
Resque Dependencies
gem install redis redis-namespace yajl-ruby

27. -Eames

28. “If you’re going to perform inception,
you need imagination.”
-Eames

29. Imagine an application
that can create the
services that it needs and
depends on...

30. Why do you need this?
• Why does the Application have a
Dependency on a Service that you
have to create outside of your code?
• Worse, replicate someone else’s steps/
configuration to get working
properly?
• Ideally you shouldn’t even have to find
the resources they should build
themselves!

31. Not meant to directly reflect resque-mongo
A Basic App
App
$redis $mongo
connection connection
Redis MongoDB
PaaS

32. Services Inception
Services inside Services inside...
App
Depends on
$redis $mongo Redis and Mongo
connection connection
You are expected to
Redis MongoDB
set these up yourself
using a command line
PaaS tool in most cases

33. What happens if you
forget or misconfigure a
Service?

34. Services Limbo
-Cobb
“We drop into limbo.”
“Are you serious?”
-Arthur
“Limbo?”
“Unconstructed dream space.”
-Ariadne

35. Why isn’t this part of the
code itself?

36. Services Inception
• The App should ask the Platform for
the Service and get it
• If the Platform doesn’t have a service
due to complexity, the app should be
able to create/build it
• The App should have the ability to
automatically bootstrap, update, and
repair itself if need be
• Include the steps, let me choose if I
want to change them

37. We need a Services
Architect
-Ariadne
“I designed the place.”

38. We need a Services
Architect
-Ariadne
“I designed the place.”
The original Developer of the code knows what
it expects, they should leave this behind with the
code (either in it or as an artifact)

39. PaaS Provider Your App
MySQL

40. 1
PaaS Provider Your App
MySQL

41. 1
PaaS Provider Your App
2
MySQL

42. 1
PaaS Provider Your App
3
2
MySQL

43. PaaS Provider Your App
MySQL

44. 1
PaaS Provider Your App
MySQL

45. 1
PaaS Provider Your App
2
MySQL

46. 1
PaaS Provider Your App
2
3 MySQL

47. 1
PaaS Provider Your App
2 4
3 MySQL

48. PaaS Provider App
Redis MongoDB

49. PaaS Provider App
Redis MongoDB

50. PaaS Provider App
Redis MongoDB

51. PaaS Provider App
Redis MongoDB

52. One Approach

53. Planning Inception
manifest.yml
App
....
$redis $mongo
connection connection
services:
Redis MongoDB redis_test:
PaaS type: :redis
if_exists: keep
mongo_test:
type: :mongodb
if_exists: keep

54. A Second Approach

55. DevOps?
Maybe an extension or file format reuse?

56. A Third Approach

57. Why not do this in the
App
App code?
$redis $mongo
connection connection
Redis MongoDB
PaaS
DependsOn = {
'foo' => 'redis',
'bar' => 'mongo'
}

58. configure do
$client = Paasclient.new('MyPassword',nil)
resolve_deps = Dependency.new(DependsOn)
end

59. Dependency.each do |dep_name, dep_type|
case dep_type
when 'redis'
unless $client.exists(dep_name)
$client.create_redis(dep_name)
end
unless $client.bound(dep_name)
$client.bind(dep_name)
end
when 'mongo'
....

60. PaaS Provider App
Redis MongoDB

61. The Penrose Stairs
Be Careful with
Metaprogramming Services

62. Methods
• service exists / create / delete
• service bound / bind / unbind
• app exists / create / delete
• app start / stop / restart / instances
existed? history?

64. A Sample More Complex
Application
Consuming App
(Uses Infrastructure Below)
Twitter http service GridFS
$redis $mongo
connection connection
Redis MongoDB
Internet
PaaS

65. Questions?
Opinions?
Other Approaches?

66. Thank You!