This document discusses using Sinatra to build a JSON query service with the following key points: - It describes building a Sinatra app frontend to query portions of JSON documents stored in a Redis backend cache. - The backend uses Redis as a fast key-value store to cache full JSON documents and return requested portions based on a JSON query syntax. - Testing the Sinatra app is discussed as well as potential issues like query syntax, caching at scale, and performance under load.

1. Sinatra, Testing in Sinatra, Redis
Vassilis Vatikiotis, Athens Ruby Meetup 7

2. Sinatra
 Web micro framework and DSL.
 > 4000 LOC, 6 Classes. Study!
 Modular & standalone style.
 Small apps & web services.
 Rack compatible.
 Extendible.
 Very popular: 182 gems in rubygems.org
 MRI & 1.9, Rubinius 1.2.3, Jruby 1.6 full support.
 MacRuby, IronRuby (reported OK).

3. Hello world
require 'sinatra'
get '/' do
'Hello world!'
end
Can't beat that!

4. Sinatra basics (1.2.6)
 Route matching: HTTP method + URL pattern.
 Strings, regexps. Conditions allowed.
get '/say/:what/to/*', :host_name => /^admin./ do
params[:what]
what = #{what}
params[:splat] # => Array
erb :index
end
 Views and Templates.
 ERB, Haml, Builder, Nokogiri, Sass, Markdown,
Coffescript and more.
 Embedded, inline, file.

5. Sinatra basics
 Filters
 Before and after filters.
 URL patterns and conditions (just like routes).
 Helpers support a host of tools:
 Sessions.
 Request “flow” control: halt, pass, trigger route.
 Set HTTP headers, status, body, mime type.
 .configure to set things to run once.
 Examine request, logging.
 Error handlers look like routes.

6. Sinatra – scope/binding
 Everything inherits from Sinatra::Base
 Request object accessible within route blocks, helper
methods, filters, views, own methods.
 Single application class for all requests; you cannot access
request or session at class level.
 Application scope accessible within:
 settings.get within request scope.
 class body & block to helper method.
 blocks/procs used as value in set
 block to Sinatra.new

7. Sinatra and Rack stack
 Sits on top of Rack.
 middleware pipelines via Sinatra::Base.use
 Modular and Classic style.
 Classic Sinatra::Application pollutes global namespace.
 Modular Sinatra::Base ­ we can build a stack e.g. more
sinatra, padrino, ramaze, rails, any rack abiding citizen
 We can dynamically create a Sinatra app...
+ (Object) Sinatra.new(base = Base, options = {}, &block)
 ...and use it somewhere in the stack!
use Sinatra { get('/') { ... } }
run My::EndPoint

8. Shamelessly ripped example
require 'sinatra/base'
class LoginScreen < Sinatra::Base
enable :sessions
set :session_secret, 'super secret'
get('/login') { haml :login }
post('/login') do
if params[:name] == 'admin' && params[:password] == 'admin'
session['user_name'] = params[:name]
else
redirect '/login'
end
end
end
class MyApp < Sinatra::Base
use LoginScreen
before do
unless session['user_name']
halt "Access denied, please <a href='/login'>login</a>."
end
end
get('/') { "Hello #{session['user_name']}." }
end

9. Extending Sinatra
 Instance context: using helpers method
module Sinatra
module HTMLEscapeHelper
def h(text)
Rack::Utils.escape_html(text)
end
end
helpers HTMLEscapeHelper
end
 Class context: using register method
module Sinatra
module LinkBlocker
def block_links_from(host)
before {
halt 403, "Go Away!" if request.referer.match(host) }
end
end
register LinkBlocker
end

10. Testing in Sinatra
 Use generic test frameworks – mix in Rack::Test::Methods.
require 'rack/test' Check source!
 Webrat / Capybara require minimal wiring.
 Use last_response object, query everything you want.
it "says hello" do
get '/hello'
response = JSON.parse(last_response.body)
response["ProxyResponse"].should == @page_id
last_response.should be_ok
end
it "says hello" do
visit '/hello'
page.should have_content('hello_world')
end

11. Sinatra & testing
 Sinatra is excellent for implementing web services.
 Lean framework.
 Views? I don't need views.
 Rack middleware in place.
 Sinatra/WS is excellent for getting comfy with testing.
 Integration testing is easy; exercise and verify!
 You are doing state verification.
 Don't get confused with behavior verification
(Rspec?)
 Want to grok testing? Code a Sinatra web service.

12. Intermission I
Bite the bullet: Test it!
 Beginnings are hard! Grok testing, figure out the api(s),
figure out which api(s), figure out when... go figure!
 Test cycle: setup, exercise, verify, teardown.
 Exercise SUT, use doubles for collaboration.
 Test Double is a generic term for a test obj.
 A stub provides a canned answer.
 A mock “is a stub” which enforces behavior checks.
 Classic TDD vs Mockist TDD vs BDD.

13. Intermission II
Stub example
Project.stub(:find).and_return( Project.new(
:name => “Greek reboot”))
stub_project = Project.find(1)
previous_count = manager.projects.count
manager.projects << stub_project
manager.projects.count.should == previous_count + 1
 SUT is the manager object.
 A stub is a collaborator used to help testing the SUT.

14. Intermission III
Mock example
post '/manager/:role' do
manager.switch_role params[:role]
end
class Manager
attr_accessor :role
def switch_role( role )
@role = (role == 'team leader' ? 'project leader' : 'team
leader')
end
end
#this is my test
it “/manager/role spews back the role” do
mock_manager = Manager.new( :role => 'team leader' )
mock_manager.should_receive(:switch_role).and_return('project
leader')
post '/manager/:role', mock_manager
last_response.body.should == 'project leader'
end

15. Steps
 Sinatra and HTTP services is ideal for getting you started
with testing.
 Write a small sinatra app, no views, think in term of HTTP
endpoints.
 REST is not required!
 Integration test it. Minimal api knowledge. Want to try
stubs? stub a :find ­ user maybe?
 Now do it backwards. Write tests first!
 I'm severely TDDing!

16. JSON Query HTTP Service
Goal
 Service provides data in json format.
 I need a portion of a map.
 I need just a portion of a big data chunk.
 I want to be able to query for my bits of data.
 Ideally I want the data service to provide me with such a
facility.
 JSON Query Service deals with the above.

17. JSON Query HTTP Service
Issues
 JSON selector as a service.
 JSONSelect, JSONPath, XPATH.
 Issue: there is no established json selector mechanism.
 Work around it or use non established methods,
 own implementation.
 Need to fetch and cache the requested source.
 Issue: cache component need to scale out and be consistent.
 Issue: potentially massive service load.

18. JSON Query HTTP Service
Frontend
 Sinatra.
 No RESTful API. Do not need one (atm).
 Request a portion of a json page.
 Response is the requested json portion, with metadata. All
wrapped in a json response.
 Works for multiple URLs.

19. JSON Query HTTP Service
Backend
 Redis. key­value data store, can be used as cache.
 Fast! compared with memcached? YMMV.
 Clustering? Not here yet!
 Used it as a concept. Selling point? Virtual Memory
 What if we run out of memory?
 suitable for large values, keys are requested URLs
 Keys in memory, values on disk.
 Idea: SSD for memory, slower disk for values.

20. How can I use it?
require 'net/http'
require 'json'
post_data = {
:apikey => "dc25ece96592813032f3605e95e8c6e9669cad46",
:guid => "123123",
:xpath => "//leg[2], //leg[1]",
:url => "http://maps.googleapis.com/maps/api/directions/json?
origin=Chicago,IL&destination=Los+Angeles,CA&waypoints=Joplin,MO|
Oklahoma+City,OK&sensor=false”
}
document =
Net::HTTP.post_form( URI.parse('http://jsonquery.heroku.com/v1/fetch_page') ,
post_data )
response = JSON.parse( document.body )
xpath_1 = response["ProxyResponse"]["url_1"]["xpath_1"][“response”]
xpath_2 = response["ProxyResponse"]["url_1"]["xpath_2"][“response”]

21. Future
 XML support.
 Redis VM support is dropped in 2.4
 Need to investigate memcache, membase, riak and so
on. Many solutions according to domain.
 Scale. How?
 Usual approach so far is to spawn multiple processes.
 Threaded vs Evented.
 Unicorn, Mongrel, Thin, Rainbows, Passenger,
Zbattery, Goliath.

22. Bibliography
1) Mocks aren't Stubs. Martin Fowler
2) Service­oriented design with Ruby and Rails. Paul Dix et
al
3) The Rspec book, David Chelimsky et al
4) Rails test prescriptions, Noel Rappin
5) Sinatra and testing from a pro: Peepcode Play­by­Play
(episode 54) with J. Barnette