Asynchronous web frameworks excel at integrating with external services and can offer higher throughput then traditional synchronous systems. However asynchronous programming is not without its challenges and porting your existing codebase to an asynchronous platform may not be a viable option.
In this presentation Nolan will outline how you can RESTfully integrate an asynchronous framework named Goliath [ https://github.com/postrank-labs/goliath] with your existing infrastructure and external services enabling you to build scalable richer interactions.
2. Nolan Evans
- Software Engineer @Square
- Enjoys coding
- Fan of silly hats
github: nolman
twitter @nolman
www.nolanevans.com
3. Outline
• What problem are we trying to solve?
• Consider other approaches
• Put together the pieces of Goliath
- EventMachine
- Fibers
• Example Goliath apps
• Integrating with your existing stack
4. User Experience Impacting
External Services
• Real time feedback required
• Dependency on external systems
• Outages should not impact other systems
9. Why not your normal Rails?
• External Services can will be slow
• Typical Rails stack is blocking/single threaded
• Passengers back up on blocking requests
• All Passengers blocked = 503 Service Unavailable
10. Why not background Workers & Polling?
• Communication overhead
• Polling overhead
• Typical workers still blocking/single threaded
• Memory footprint
11. Why Asynchronous IO works
• Continue to process incoming requests when
external services are slow
• No cross processes communication (vs workers)
• Increased throughput
• Low memory footprint
• Fault tolerance
• Better resource utilization
12. How does Asynchronous IO work?
Evented IO uses the Reactor Pattern
nginx
node.js
varnish
EventMachine
Twisted
Your Browser
13. How EventMachine Works
while (true) {
_UpdateTime();
_RunTimers();
_AddNewDescriptors();
_ModifyDescriptors();
_RunOnce();
if (bTerminateSignalReceived)
break;
}
http://github.com/eventmachine/eventmachine
14. _RunOnce();
• Find time until next timer event
• Runs select, epoll(linux), or kqueue(bsd)
select - check all file descriptors for changes
epoll - multiplexing I/O
Create via epoll_create
Register interest via epoll_ctl
Wait for input with epoll_wait get list of descriptors that changed
• Run associated callbacks
• epoll and kqueue are generally more performant
15. Downsides of Asynchronous IO
• Confusing control flow
• Error handling not obvious
• Need to spin up a reactor to run tests
• Blocking calls block the entire reactor
16. Callback Spaghetti
EventMachine.run do
http = EM::HttpRequest.new('http://www.google.com').get
begin
puts "Before"
http.callback {
raise "Stuff Happens"
}
puts "After"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
end
code at: https://gist.github.com/
17. Callback Spaghetti
EventMachine.run do
http = EM::HttpRequest.new('http://www.google.com').get
begin
puts "Before"
http.callback { Prints:
raise "Stuff Happens"
raise "Stuff Happens" Before
} After
puts "After" All Done?
rescue Exception => ex #Stuff Happens (RuntimeError)
puts "Rescued!"
end
puts "All Done?"
end
code at: https://gist.github.com/
18. Fibers
• Developer scheduled threads
• Lets you pause/resume execution of code
• Cooperative scheduling
• Only one executing at a time
• No semaphores needed
• Can make asynchronous code quack like
synchronous code
19. Fibers
fiber = Fiber.new do
puts 'inside the fiber'
Fiber.yield 1 #Yields control back out to the root fiber
puts 'after yield'
2
end
puts "before starting the fiber"
puts fiber.resume # Initially Starts the fiber
puts "we have control again"
puts fiber.resume # Returns control back to the fiber
puts "Done!"
20. Fibers
fiber = Fiber.new do
puts 'inside the fiber'
Fiber.yield 1 #Yields control back out to the root fiber
puts 'after yield'
2 Prints:
end before starting the fiber
inside the fiber
puts "before starting the fiber" 1
puts fiber.resume # Initially Starts the fiber
we have control again
puts "we have control again"
puts fiber.resume # Returns control back to the fiber after yield
puts "Done!" 2
Done!
21. Asynchronous Code with Fibers
EventMachine.run do
def syncify(url)
fiber = Fiber.current
http = EM::HttpRequest.new(url).get
http.callback { fiber.resume(http) }
Fiber.yield
end
Fiber.new {
begin
puts "Before"
http = syncify("http://www.google.com")
puts "After"
raise "Stuff Happens"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
}.resume
end
code at: https://gist.github.com/
22. Asynchronous Code with Fibers
EventMachine.run do
def syncify(url)
fiber = Fiber.current
http = EM::HttpRequest.new(url).get Prints:
http.callback { fiber.resume(http) }
Fiber.yield
Before
end After
Fiber.new { Rescued!
begin All Done?
puts "Before"
http = syncify("http://www.google.com")
puts "After"
raise "Stuff Happens"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
}.resume
end
code at: https://gist.github.com/
23. Async IO with Fibers vs Threads
Pros
• Many Ruby libraries are not Thread Safe
• Creating fibers is cheap & fast
• Getting Semaphores right isn’t easy
• Writing tests for semaphores is tough
• Cooperative scheduling is better then round robin
Cons
• Blocking the reactor will stop all other processing
24. Goliath
• An Asynchronous Web Framework
• Written by PostRank
• Rack compatible(ish)
• Powered by EventMachine
• Every request run inside a fiber
• It’s fast! (3000 requests/second)
http://postrank-labs.github.com/
25. XML as JSON proxy
• Same Origin Policy restricts XMLHttpRequest
• JSONP lets us get around same origin policy
• External Service has to implement the JSONP api
• Many sites/services do not have a JSON nevermind JSONP api
• Use Goliath to convert XML to JSON
26. XML as JSON proxy
class XmlAsJsonProxy < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Formatters::JSON # JSON output formatter
use Goliath::Rack::Render # auto-negotiate response format
def response(env)
http = EM::HttpRequest.new(params['url']).get(:redirects => 1)
converter = DocumentConverter.new(http.response, params['mapping'])
[200, {'X-Goliath' => 'Proxy', 'Content-Type' => 'application/json'},
converter.mapping_to_json.merge(:redirected_to => http.last_effective_url)]
end
end
code at: https://www.github.com/nolman/proxy_service
32. Integrating Goliath with external
services and our existing stack
• Have Goliath Proxy Requests
• Forward the response to our stack
33. Browser Goliath Web Rails
GET http://foo.com/?
url=http://github.com/users/fred
forward_to=http://foo.com/render GET http://github.com/users/fred
<xml><name>fred...
POST http://foo.com/render
body=<xml><name>fred...
<div>Welcome fred ... </div>
<div>Welcome fred ... </div>
34. Forwarding Proxy
class ForwardProxy < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Render # auto-negotiate response format
def response(env)
http = EM::HttpRequest.new(params['url']).get(:redirects => 1)
http = EM::HttpRequest.new(params['forward_to']).post(:body => {:document =>
http.response})
[http.response_header.status, http.response_header, http.response]
end
end
code at: https://www.github.com/nolman/proxy_service
35. Rack Routes
class ProxyServer < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Render # auto-negotiate response format
map "/forward" do
run ForwardProxy.new
end
map "/as_json" do
run XmlAsJsonProxy.new
end
end
code at: https://www.github.com/nolman/proxy_service
36. Fin
• Have an idea how Goliath works
- EventMachine
- Fibers
• Advantages of Asynchronous IO
• Basic Goliath App
• You should try Goliath
37. Any Questions?
github: nolman
twitter @nolman
www.nolanevans.com
Editor's Notes
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
So if we look at this code, we can see that we are requesting the document found on google.com, and lets say that there is some sort of problem parsing the dom so we raise an error stuff happens\n\n
so surprisingly here&#x2019;s what&#x2019;s printed out when this code runs, the callback is executed outside of the context of the error handler and any exceptions raised are unhandled\n
in ruby 1.9 fibers provide a way for us to untangle this callback spaghetti\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
\n
in ruby 1.9 fibers provide a way for us to untangle this callback spaghetti\n
rack middleware\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
for example facebook&#x2019;s link scraper\n
for example facebook&#x2019;s link scraper\n
for example facebook&#x2019;s link scraper\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
for example facebook&#x2019;s link scraper\n
This approach is useful for when you need to integrate external services with some part of your website/services user experience\n
This approach is useful for when you need to integrate external services with some part of your website/services user experience\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
so if we look at this code you will see we wrap the fetching of the document in a syncify function\n
Talk about me\n
right now our card auth&#x2019;s happen in our passenger&#x2019;s which are single threaded and blocking\nfibers would be a better solution to this problem and remove a point of failure from our systems\n