Camping: Going off the
Rails with Ruby
Adventures in creative coding
for people who should know better
The weasel words
• This presentation contains code
That code is probably broken
• If that bothers you - fix it
That’s called a learning experience
Who are these lunatics?
Romek Szczesniak
romek@spikyblackcat.co.uk
He does security
Eleanor McHugh
eleanor@games-with-brains.com
She does real-time systems
Alright, but what are
they doing here?
• Ruby
Pcap & BitStruct
• WEBrick
• Camping
• but no Rails...
No Rails?
• That’s right, we don’t use Rails
But we do use Ruby
• And we do write web applications
So how is that possible?
Camping!!!
• That’s right, we use Camping
It’s by Why The Lucky Stiff
• It’s cool
• It’s really cool
• It’s so damn cool you’d have to be mad not
to use it!!!
It’s this simple!
%w[rubygems active_record markaby metaid ostruct].each {|lib| require lib}
module Camping;C=self;module Models;end;Models::Base=ActiveRecord::Base
module Helpers;def R c,*args;p=/(.+?)/;args.inject(c.urls.detect{|x|x.
scan(p).size==args.size}.dup){|str,a|str.gsub(p,(a.method(a.class.primary_key
)[]rescue a).to_s)};end;def / p;File.join(@root,p) end;end;module Controllers
module Base;include Helpers;attr_accessor :input,:cookies,:headers,:body,
:status,:root;def method_missing(m,*args,&blk);str=m==:render ? markaview(
*args,&blk):eval("markaby.#{m}(*args,&blk)");str=markaview(:layout){str
}rescue nil;r(200,str.to_s);end;def r(s,b,h={});@status=s;@headers.merge!(h)
@body=b;end;def redirect(c,*args);c=R(c,*args)if c.respond_to?:urls;r(302,'',
'Location'=>self/c);end;def service(r,e,m,a);@status,@headers,@root=200,{},e[
'SCRIPT_NAME'];@cookies=C.cookie_parse(e['HTTP_COOKIE']||e['COOKIE']);cook=
@cookies.marshal_dump.dup;if ("POST"==e['REQUEST_METHOD'])and %r|Amultipart
/form-data.*boundary="?([^";,]+)"?|n.match(e['CONTENT_TYPE']);return r(500,
"No multipart/form-data supported.")else;@input=C.qs_parse(e['REQUEST_METHOD'
]=="POST"?r.read(e['CONTENT_LENGTH'].to_i):e['QUERY_STRING']);end;@body=
method(m.downcase).call(*a);@headers["Set-Cookie"]=@cookies.marshal_dump.map{
|k,v|"#{k}=#{C.escape(v)}; path=/"if v != cook[k]}.compact;self;end;def to_s
"Status: #{@status}n#{{'Content-Type'=>'text/html'}.merge(@headers).map{|k,v|
v.to_a.map{|v2|"#{k}: #{v2}"}}.flatten.join("n")}nn#{@body}";end;def 
markaby;Class.new(Markaby::Builder){@root=@root;include Views;def tag!(*g,&b)
[:href,:action].each{|a|(g.last[a]=self./(g.last[a]))rescue 0};super end}.new(
instance_variables.map{|iv|[iv[1..-1].intern,instance_variable_get(iv)]},{})
end;def markaview(m,*args,&blk);markaby.instance_eval{Views.instance_method(m
).bind(self).call(*args, &blk);self}.to_s;end;end;class R;include Base end
class NotFound<R;def get(p);r(404,div{h1("#{C} Problem!")+h2("#{p} not found")
});end end;class ServerError<R;def get(k,m,e);r(500,markaby.div{h1 "#{C} Prob
lem!";h2 "#{k}.#{m}";h3 "#{e.class} #{e.message}:";ul{e.backtrace.each{|bt|li(
bt)}}})end end;class<<self;def R(*urls);Class.new(R){meta_def(:inherited){|c|
c.meta_def(:urls){urls}}};end;def D(path);constants.each{|c|k=const_get(c)
return k,$~[1..-1] if (k.urls rescue "/#{c.downcase}").find {|x|path=~/^#{x}
/?$/}};[NotFound,[path]];end end end;class<<self;def escape(s);s.to_s.gsub(
/([^ a-zA-Z0-9_.-]+)/n){'%'+$1.unpack('H2'*$1.size).join('%').upcase}.tr(' ',
'+') end;def unescape(s);s.tr('+', ' ').gsub(/((?:%[0-9a-fA-F]{2})+)/n){[$1.
delete('%')].pack('H*')} end;def qs_parse(qs,d='&;');OpenStruct.new((qs||'').
split(/[#{d}] */n).inject({}){|hsh,p|k,v=p.split('=',2).map{|v|unescape(v)}
hsh[k]=v unless v.empty?;hsh}) end;def cookie_parse(s);c=qs_parse(s,';,') end
def run(r=$stdin,w=$stdout);w<<begin;k,a=Controllers.D "/#{ENV['PATH_INFO']}".
gsub(%r!/+!,'/');m=ENV['REQUEST_METHOD']||"GET";k.class_eval{include C
include Controllers::Base;include Models};o=k.new;o.service(r,ENV,m,a);rescue
=>e;Controllers::ServerError.new.service(r,ENV,"GET",[k,m,e]);end;end;end
module Views; include Controllers; include Helpers end;end
Why?
• For fun
For profit
For the satisfaction of knowing exactly
how your application works
• For the look on your boss’s face when he
reads the documentation
Earlier...
• But let’s not get ahead of ourselves
First we want to take you on a journey
• A journey back in time
A journey back to...
January 3rd 2006
Location: The Secret Basement Lairtm
of
Captain IP and The DNS avengers
• Their task: to launch a new Top Level
Domain which DOESN’T RESOLVE
MACHINE ADDRESSES?!?!?!
Their resources? Hands to wave with and
hit keyboards with!
Ruby to the Rescue
• It’s easy to learn
It’s quick to code in
• It’s pleasing to the eye
It’s fun!
You keep saying that
• Yes!!!
Fun makes for better coders
• Better coders write good code
• Good code stands the test of time
• If coding isn’t fun YOU’RE USING THE
WRONG TOOLS!!!!
The console jockeys
• let’s write a menu driven calculator
output: puts(), print()
• input: gets(), termios library
• old-fashioned and unattractive
• termios is fiddly
A simple calculator
#!/usr/bin/env ruby -w
require 'termios'
$total = 0
$menu_entries = [['+', "Add"], ['-', "Subtract"], ['*', "Multiply"], ['/', "Divide"], ['c', 'Clear'], ['q',"Quit"]]
$commands = $entries.inject([]) { | commands, entry |
commands << entry[0]
}
$captions = $entries.inject([]) { | captions, entry |
captions << entry[1]
}
loop do
puts "nSimple Calculatorn"
entries.each { | entry | puts "#{entry[0]}. #{entry[1]}n" }
t = Termios.tcgetattr(STDIN)
t.lflag &= ~Termios::ICANON
Termios.tcsetattr(STDIN,0,t)
begin
action = STDIN.getc.chr
end until $commands.member?(action)
exit() if action == $commands.last
action = $commands.index(action)
puts "n#{$captions[action]}nn"
case action
when 0 : $total += gets()
when 1 : $total -= gets()
when 2 : $total *= gets()
when 3 : $total /= gets()
when 4 : $total = 0
end
puts "Total = #{$total}"
end
A Ruby packet reader
The 7 layer IP model
What the heck?
• We want to look at UDP and DNS traffic
• Our first implementation is console-based,
so hold on to your hats...
We’re exploring the UDP layer
UDP header in Ruby
require 'bit-struct'
class IP < BitStruct
unsigned :ip_v, 4, "Version"
unsigned :ip_hl, 4, "Header length"
unsigned :ip_tos, 8, "TOS"
unsigned :ip_len, 16, "Length"
unsigned :ip_id, 16, "ID"
unsigned :ip_off, 16, "Frag offset"
unsigned :ip_ttl, 8, "TTL"
unsigned :ip_p, 8, "Protocol"
unsigned :ip_sum, 16, "Checksum"
octets :ip_src, 32, "Source addr"
octets :ip_dst, 32, "Dest addr"
rest :body, "Body of message"
note "rest is application defined message body"
initial_value.ip_v = 4
initial_value.ip_hl = 5
end
class UDP < BitStruct
unsigned :udp_srcport, 16, "Source Port"
unsigned :udp_dstport, 16, "Dest Port"
unsigned :udp_len, 16, "UDP Length"
unsigned :udp_chksum, 16, "UDP Checksum"
rest :body, "Body of message"
note "rest is application defined message body"
end
class DNSQueryHeader < BitStruct
unsigned :dns_id, 16, "ID"
unsigned :dns_qr, 1, "QR"
unsigned :dns_opcode,4, "OpCode"
unsigned :dns_aa, 1, "AA"
unsigned :dns_tc, 1, "TC"
unsigned :dns_rd, 1, "RD"
unsigned :dns_ra, 1, "RA"
unsigned :dns_z, 3, "Z"
unsigned :dns_rcode, 4, "RCODE"
unsigned :dns_qdcount, 16, "QDCount"
unsigned :dns_ancount, 16, "ANCount"
unsigned :dns_arcount, 16, "ARCount"
rest :data,
"Data"
end
class Time
# tcpdump style format
def to_s
sprintf "%0.2d:%0.2d:%0.2d.%0.6d", hour, min,
sec, tv_usec
end
end
udpip.rb
Capturing UDP packets
#!/usr/local/bin/ruby
require 'pcaplet'
include Pcap
require 'udpip'
DIVIDER = "-" * 50
def print_details(section)
puts DIVIDER, section, DIVIDER
end
pcaplet = Pcaplet.new('-s 1500')
pcaplet.each_packet { |pkt|
if pkt.udp?
puts "Packet: #{pkt.time} #{pkt}"
if (pkt.sport == 53)
udp = UDP.new
udp.udp_srcport = pkt.sport
udp.udp_dstport = pkt.dport
udp.udp_len = pkt.udp_len
udp.udp_chksum = pkt.udp_sum
udp.body = pkt.udp_data
print_details udp.inspect_detailed
# look for DNS request only
dns = DNSQueryHeader.new(pkt.udp_data)
bytearray = Array.new
udp.body.each_byte { |c|
bytearray.concat(c.to_s.to_a)
print c.to_s(16), ' '
}
print_details dns.inspect_detailed
end
end
}
pcaplet.close
tcpdump.rb
A live UDP packet
A live DNS packet
Can we have that on
Windows?
• A GUI? You gotta be joking!!
Why do you think we use Macs?
• How about we just turn it into a web
application instead?
• Sure, we can do that with Ruby
• [What have we let ourselves in for...]
The NDA kicks in
• Here’s where we hit the brick wall on
what we can talk about
You might imagine a DNS-sniffing web
application, but we couldn’t possibly
comment
• So lets get down to some web app basics
And yes, we will be kicking it old-skool...
Introducing WEBrick
• WEBrick is an HTTP server library
It’s part of the Ruby 1.8 release
• It can serve static documents
• It can serve HTTPS using Ruby/OpenSSL
It can serve arbitrary code blocks
• It can serve servlets
Static content
#!/usr/local/bin/ruby
require 'webrick'
server = WEBrick::HTTPServer.new(:Port => 8080, :DocumentRoot => Dir::pwd + "/htdocs")
# mount personal directory, generating directory indexes
server.mount("/~eleanor", WEBrick::HTTPServlet::FileHandler, "/Users/eleanor/Sites", true)
# catch keyboard interrupt signal to terminate server
trap("INT"){ server.shutdown }
server.start
#!/usr/local/bin/ruby
# This requires Ruby/OpenSSL
require 'webrick'
require 'webrick/https'
certificate_name = [ ["C","UK"], ["O","games-with-brains.org"], ["CN", "WWW"] ]
server = WEBrick::HTTPServer.new( :DocumentRoot => Dir::pwd + "/htdocs", :SSLEnable => true,
:SSLVerifyClient => ::OpenSSL::SSL::VERIFY_NONE, :SSLCertName =>
certificate_name )
trap("INT"){ s.shutdown }
s.start
A standard HTTP server
An HTTPS server
Servlets
#!/usr/local/bin/ruby
require 'webrick'
server = WEBrick::GenericServer.new()
trap("INT"){ server.shutdown }
server.start{|socket| socket.puts("This is a code blockr") }
#!/usr/local/bin/ruby
require 'webrick'
server = WEBrick::HTTPServer.new()
trap("INT"){ server.shutdown }
def generate_response(response)
response.body = "<HTML>hello, world.</HTML>"
response['Content-Type'] = "text/html"
end
class HelloServlet < WEBrick::HTTPServlet::AbstractServlet
def do_GET(request, response)
generate_response(response)
end
end
server.mount_proc("/hello/simple"){ | request, response | generate_response(response) }
server.mount("/hello/advanced", HelloServlet)
server.start
A Ruby code block
A WEBrick servlet
It’s that simple?
• Yes, it’s that simple
Of course these are trivial examples...
• ...so let’s build an application server
An application server
• Still wondering when we get to the really
good stuff?
Soon, we promise
• But first to show you how NOT to do it!
Wrap the request
class RequestContext
attr_reader :request, :response, :servlets, :creation_time
def initialize(request, response)
@request, @response, = request, response
@creation_time = Time.now()
end
def page_not_found
@response.status = WEBrick::HTTPStatus::NotFound.new()
end
def response_page(page)
@response['Content-Type'] = page.content_type
@response.body = CGI::pretty(page.to_str())
end
def <<(item)
@response.body << CGI::pretty(item)
end
end
A basic request context
Serve the pages
IP_ADDRESS_PATTERN = /^d{1,3}.d{1,3}.d{1,3}.d{1,3}/
class ApplicationServer
attr_reader :web_server, :server_address, :servlets, :pages
def initialize(parameters = {})
@server_address = parameters[:my_address] or raise “Please supply a server address”
raise “Invalid IP address for server” unless IP_ADDRESS_PATTERN.match(@server_address)
@web_server = WEBrick::HTTPServer.new({:BindAddress => @server_address})
@servlets = {}
@pages = {}
end
def start
trap("INT") { @web_server.shutdown }
@web_server.start
end
def register_page(path, page)
@pages[path] = page
@web_server.mount_proc(path) { | request, response |
context = RequestContext.new(request, response)
@pages[request.path] ? context.response_page(@pages[request.path]) : context.page_not_found()
}
end
def register_method(path, handler)
@servlets[path] = self.method(handler).to_proc
@web_server.mount_proc(path) { | request, response |
context = RequestContext.new(request, response)
@servlets[request.path] ? (context << @servlets[request.path].call(context).to_str()) : context.page_not_found()
}
end
end
The application server
Write the application
#!/usr/local/bin/ruby
require 'appserver.rb'
class SimpleServer < ApplicationServer
def initialize(parameters = {})
super
register_page("/hello/simple", "<HTML>Hello, world</HTML>")
register_method("/hello/advanced", :hello_world)
end
def hello_world(context)
"<HTML>Hello, world</HTML>"
end
end
begin
SimpleServer.new({:my_address => ARGV.shift()}).start()
rescue RuntimeError => e
$stderr.puts "Usage: simpleserver host-address"
$stderr.puts "address must be provided in dotted-quad format (i.e. xxx.xxx.xxx.xxx)"
end
Revisiting “hello, world”
What have we done?!?
• On the surface this is elegant
• But underneath it sucks
• There’s no support for HTML
• Only methods can be used as servlets
• We’re tied to WEBrick - which is slow
The road to perdition
• So we added an HTML 4 library
• And a server pages container
• And ActiveRecord
• We meta’d the code to death
• But it still lacked va-va-voom...
The case for Rails
• So perhaps we should have just used Rails
in the first place
• We’d be another of those “Rails saved my
career” success stories!
• Hindsight’s always 20/20
• But we’re old-school coders and it’s far
too user friendly for our comfort
The pressure against
• Working at a very low level
• Simple code required
• Can Rails talk nicely to low-level code?
• Strong management resistance - too high
a learning curve?
So why Camping?
• Camping is beauty incarnate
• It’s less than 4K of code
• It uses Markaby and ActiveRecord
• It runs on JRuby!!!
• Oh, and it’s great fun to abuse...
Gratuitous diagram
lifted from
http://redhanded.hobix.com/bits/campingAMicroframework.html
How Why? The Lucky Stiff teaches it
Markaby
• An XHTML Domain Specific Language
• Allows you to embed XHTML code in Ruby
code without building a complex object
hierarchy
• Can be used with Rails
But that’s so simple!
require 'markaby'
page = Markaby::Builder.new
page.xhtml_strict do
head { title "Camping Presentation" }
body do
h1.page_heading "Camping: Going off the Rails with Ruby"
ul.page_index do
li.page_index { a “introduction”, :href => ‘#introduction’ }
li.page_index { a “the presentation”, :href => ‘/presentation’ }
li.page_index { a “comments”, :href => ‘#comments’ }
end
div.introduction! { “Everything will be alright!!!” }
div.comments! { “Have your say” }
end
end
puts page.to_s
Markaby embedded in Ruby
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>
<title>Camping Presentation</title>
</head>
<body>
<h1 class="page_heading">Camping: Going off the Rails with Ruby</h1>
<ul class="page_index">
<li class="page_index"><a href="#introduction">introduction</a></li>
<li class="page_index"><a href="/presentation">the presentation</a></li>
<li class="page_index"><a href="#comments">comments</a></li>
</ul>
<div id="introduction">Just breathe deeply...</div>
<div id="comments">Have your say</div>
</body>
</html>
Creates this
ActiveRecord
• An Object-Relational Mapper
• Implements the Active Record pattern
• Supports many popular databases
• A key component of Rails
ORMtastic
Using Active Record
require 'rubygems'
require_gem ‘activerecord’
ActiveRecord::Base.establish_connection(:adapter => “sqlite3”, :host => “localhost”, :database => “test.db”)
class User < ActiveRecord::Base
end
user = User.new()
user.id = “ellie”
user.name = “Eleanor McHugh”
user.password = “somerandomtext”
user.save
user = User.find(“ellie”)
user.destroy()
Totally RAD
• Camping builds small applications
• Why’s guideline? One file per application
• If that’s how you prefer it...
A simple example
Basic setup
#!/usr/bin/env ruby
$:.unshift File.dirname(__FILE__) + "/../../lib"
require 'camping'
require 'camping/session'
Camping.goes :Jotter
module Blog
include Camping::Session
end
• Load the camping libraries
• Define a namespace for the application
• Include session support (if required)
The data model
ule Jotter::Models
class Note < Base; end
class Database < V 1.0
def self.up
create_table :jotter_notes, :force => true do |t|
t.column :id, :integer, :null => false
t.column :created_at, :interger, :null => false
t.column :title, :string, :limit => 255
t.column :body, :text
end
end
def self.down
drop_table :jotter_notes
end
end
Jotter.create
Jotter::Models.create_schema
Defining the data model
• We mark our database as version 1.0
• A create method builds the database
The controllers
Adding controllers
module Jotter::Controllers
class Static < R '/static/(.+)'
MIME_TYPES = {'.css' => 'text/css', '.js' => 'text/javascript', '.jpg' => 'image/jpeg'}
PATH = __FILE__[/(.*)//, 1]
def get(path)
@headers['Content-Type'] = MIME_TYPES[path[/.w+$/, 0]] || "text/plain"
@headers['X-Sendfile'] = "#{PATH}/static/#{path}"
end
end
class Index < R '/'
def get
@notes = Note.find :all
render :index
end
end
class View < R '/view/(d+)'
def get note_id
@note = Note.find post_id
render :view
end
end
class Add < R ‘/add/’
def get
@note = Note.new
render :add
end
def post
note = Note.create :title => input.post_title, :body => input.post_body
redirect View, post
end
end
The controllers
class Edit < R '/edit/(d+)', '/edit'
def get note_id
@note = Note.find note_id
render :edit
end
def post
@note = Note.find input.note_id
@note.update_attributes :title => input.post_title, :body => input.post_body
redirect View, @note
end
end
class Delete < R '/delete/(d+)'
def get note_id
@note = Note.find note_id
@note.destroy
redirect Index
end
end
end
Adding controllers
• Respond to HTTP GET and POST requests
• Perform database operations
The views
Application views
module Jotter::Views
def layout
xhtml_strict do
head do
title 'blog'
link :rel => 'stylesheet', :type => 'text/css', :href => '/static/styles.css', :media => 'screen'
end
body do
h1.header { a 'jotter', :href => R(Index) }
div.body do
self << yield
end
end
end
end
def index
@notes.empty? (p 'No posts found.') : (ol.row! { _list_notes(@notes) })
p { a 'new note', :href => R(Add) }
end
def edit
_form(@note, :action => R(Edit))
end
def view
h1 @note.title
h2 @note.created_at
p @note.body
p do
[ a("View", :href => R(View, @note)),
a("Edit", :href => R(Edit, @note)),
a("Delete", :href => R(View, @note)) ].join " | "
end
end
The views
def _list_notes(notes)
@notes.each do | note |
li do
ul do
li { a note.title, :href => R(View, note) }
li note.created_at
li { a "Edit", :href => R(Edit, note) }
li { a "Delete", :href => R(Delete, note) }
end
end
end
end
def _form(note, opts)
form({:method => 'post'}.merge(opts)) do
label 'Title', :for => 'note_title'; br
input :name => 'note_title', :type => 'text', :value => note.title; br
label 'Body', :for => 'note_body'; br
textarea note.body, :name => 'note_body'; br
input :type => 'hidden', :name => 'note_id', :value => note.id
input :type => 'submit'
end
end
Application views
• Views incorporate Markaby for XHTML
• Have access to controller data
The post-amble
A basic CGI post-amble
if __FILE__ == $0
Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db'
Jotter::Models::Base.logger = Logger.new('camping.log')
Jotter.create if Jotter.respond_to? :create
puts Jotter.run
end
if __FILE__ == $0
Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db'
Jotter::Models::Base.logger = Logger.new('camping.log')
Jotter::Models::Base.threaded_connections = false
Jotter.create if Jotter.respond_to? :create
server = Mongrel::Camping::start(“0.0.0.0”, 3000, “/jotter”, Jotter)
puts “Jotter application running at http://localhost:3000/jotter”
server.run.join
end
A Mongrel post-amble
• Allows an application to self-execute
• Can be customised to suit your platform
The style-sheet
A simple style sheet
body {
font-family: Utopia, Georga, serif;
}
h1.header {
background-color: #fef;
margin: 0;
padding: 10px;
}
div.body {
padding: 10px;
}
#row ul {
list-style: none;
margin: 0;
padding: 0;
padding-top: 4px;
}
#row li {
display: inline;
}
#row a:link, #row a:visited {
padding: 3px 10px 2px 10px;
color: #FFFFFF;
background-color: #B51032;
text-decoration: none;
border: 1px solid #711515;
}
Larger applications
• One application per file is a nice idea
• But what about large applications?
• Each can be broken down into discrete
micro-applications
• Each micro-application has its own file
and mount points
Sharing a database
• Camping apps keep their database tables
in separate namespaces
• Larger applications will want to share
state between micro-applications
• We could do some ActiveRecord voodoo
• Or we could cheat... guess which?
Camping in the wilds
require 'rubygems'
require_gem 'camping', '>=1.4'
require 'camping/session'
module Camping
module Models
def self.schema(&block)
@@schema = block if block_given?
@@schema
end
class User < Base
validates_uniqueness_of :name, :scope => :id
validates_presence_of :password
end
end
def self.create
Camping::Models::Session.create_schema
ActiveRecord::Schema.define(&Models.schema)
end
Models.schema do
unless Models::User.table_exists?
create_table :users, :force => true do | t |
t.column :id, :integer, :null => false
t.column :created_on, :integer, :null => false
t.column :name, :string, :null => false
t.column :password, :string, :null => false
t.column :comment, :string, :null => false
end
execute "INSERT INTO users (created_on, name, password, comment) VALUES ('#{Time.now}', 'admin', 'admin', 'system administrator')"
end
end
end
Installing a database in the framework
Camping server
• The camping server ties together a series
of web applications
• A simple implementation ships with the
framework
The server rules
• Monitor a directory
• load/reload all camping apps that appear
in it or a subdirectory
• Mount apps according to the filenames
(i.e. jotter.rb mounts as /jotter)
• Run create method on app startup
• Support the X-Sendfile header
Summing up
• Web applications are useful outside the
usual web app environment
• Cross platform is easy when you only need
an XHTML browser
• These tasks need a lightweight design
• Camping is a good way to solve them
• And as you can see, Ruby rocks!!!
• http://code.whytheluckystiff.net/camping/wiki
http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html
• http://raa.ruby-lang.org/project/bit-struct/
http://raa.ruby-lang.org/project/ruby-termios/
Where to next?

Test

  • 1.
    Camping: Going offthe Rails with Ruby Adventures in creative coding for people who should know better
  • 2.
    The weasel words •This presentation contains code That code is probably broken • If that bothers you - fix it That’s called a learning experience
  • 3.
    Who are theselunatics? Romek Szczesniak romek@spikyblackcat.co.uk He does security Eleanor McHugh eleanor@games-with-brains.com She does real-time systems
  • 4.
    Alright, but whatare they doing here? • Ruby Pcap & BitStruct • WEBrick • Camping • but no Rails...
  • 5.
    No Rails? • That’sright, we don’t use Rails But we do use Ruby • And we do write web applications So how is that possible?
  • 6.
    Camping!!! • That’s right,we use Camping It’s by Why The Lucky Stiff • It’s cool • It’s really cool • It’s so damn cool you’d have to be mad not to use it!!!
  • 7.
    It’s this simple! %w[rubygemsactive_record markaby metaid ostruct].each {|lib| require lib} module Camping;C=self;module Models;end;Models::Base=ActiveRecord::Base module Helpers;def R c,*args;p=/(.+?)/;args.inject(c.urls.detect{|x|x. scan(p).size==args.size}.dup){|str,a|str.gsub(p,(a.method(a.class.primary_key )[]rescue a).to_s)};end;def / p;File.join(@root,p) end;end;module Controllers module Base;include Helpers;attr_accessor :input,:cookies,:headers,:body, :status,:root;def method_missing(m,*args,&#38;blk);str=m==:render ? markaview( *args,&#38;blk):eval("markaby.#{m}(*args,&#38;blk)");str=markaview(:layout){str }rescue nil;r(200,str.to_s);end;def r(s,b,h={});@status=s;@headers.merge!(h) @body=b;end;def redirect(c,*args);c=R(c,*args)if c.respond_to?:urls;r(302,'', 'Location'=&gt;self/c);end;def service(r,e,m,a);@status,@headers,@root=200,{},e[ 'SCRIPT_NAME'];@cookies=C.cookie_parse(e['HTTP_COOKIE']||e['COOKIE']);cook= @cookies.marshal_dump.dup;if ("POST"==e['REQUEST_METHOD'])and %r|Amultipart /form-data.*boundary="?([^";,]+)"?|n.match(e['CONTENT_TYPE']);return r(500, "No multipart/form-data supported.")else;@input=C.qs_parse(e['REQUEST_METHOD' ]=="POST"?r.read(e['CONTENT_LENGTH'].to_i):e['QUERY_STRING']);end;@body= method(m.downcase).call(*a);@headers["Set-Cookie"]=@cookies.marshal_dump.map{ |k,v|"#{k}=#{C.escape(v)}; path=/"if v != cook[k]}.compact;self;end;def to_s "Status: #{@status}n#{{'Content-Type'=&gt;'text/html'}.merge(@headers).map{|k,v| v.to_a.map{|v2|"#{k}: #{v2}"}}.flatten.join("n")}nn#{@body}";end;def markaby;Class.new(Markaby::Builder){@root=@root;include Views;def tag!(*g,&#38;b) [:href,:action].each{|a|(g.last[a]=self./(g.last[a]))rescue 0};super end}.new( instance_variables.map{|iv|[iv[1..-1].intern,instance_variable_get(iv)]},{}) end;def markaview(m,*args,&#38;blk);markaby.instance_eval{Views.instance_method(m ).bind(self).call(*args, &#38;blk);self}.to_s;end;end;class R;include Base end class NotFound&lt;R;def get(p);r(404,div{h1("#{C} Problem!")+h2("#{p} not found") });end end;class ServerError&lt;R;def get(k,m,e);r(500,markaby.div{h1 "#{C} Prob lem!";h2 "#{k}.#{m}";h3 "#{e.class} #{e.message}:";ul{e.backtrace.each{|bt|li( bt)}}})end end;class&lt;&lt;self;def R(*urls);Class.new(R){meta_def(:inherited){|c| c.meta_def(:urls){urls}}};end;def D(path);constants.each{|c|k=const_get(c) return k,$~[1..-1] if (k.urls rescue "/#{c.downcase}").find {|x|path=~/^#{x} /?$/}};[NotFound,[path]];end end end;class&lt;&lt;self;def escape(s);s.to_s.gsub( /([^ a-zA-Z0-9_.-]+)/n){'%'+$1.unpack('H2'*$1.size).join('%').upcase}.tr(' ', '+') end;def unescape(s);s.tr('+', ' ').gsub(/((?:%[0-9a-fA-F]{2})+)/n){[$1. delete('%')].pack('H*')} end;def qs_parse(qs,d='&#38;;');OpenStruct.new((qs||''). split(/[#{d}] */n).inject({}){|hsh,p|k,v=p.split('=',2).map{|v|unescape(v)} hsh[k]=v unless v.empty?;hsh}) end;def cookie_parse(s);c=qs_parse(s,';,') end def run(r=$stdin,w=$stdout);w&lt;&lt;begin;k,a=Controllers.D "/#{ENV['PATH_INFO']}". gsub(%r!/+!,'/');m=ENV['REQUEST_METHOD']||"GET";k.class_eval{include C include Controllers::Base;include Models};o=k.new;o.service(r,ENV,m,a);rescue =&gt;e;Controllers::ServerError.new.service(r,ENV,"GET",[k,m,e]);end;end;end module Views; include Controllers; include Helpers end;end
  • 8.
    Why? • For fun Forprofit For the satisfaction of knowing exactly how your application works • For the look on your boss’s face when he reads the documentation
  • 9.
    Earlier... • But let’snot get ahead of ourselves First we want to take you on a journey • A journey back in time A journey back to...
  • 10.
    January 3rd 2006 Location:The Secret Basement Lairtm of Captain IP and The DNS avengers • Their task: to launch a new Top Level Domain which DOESN’T RESOLVE MACHINE ADDRESSES?!?!?! Their resources? Hands to wave with and hit keyboards with!
  • 11.
    Ruby to theRescue • It’s easy to learn It’s quick to code in • It’s pleasing to the eye It’s fun!
  • 12.
    You keep sayingthat • Yes!!! Fun makes for better coders • Better coders write good code • Good code stands the test of time • If coding isn’t fun YOU’RE USING THE WRONG TOOLS!!!!
  • 13.
    The console jockeys •let’s write a menu driven calculator output: puts(), print() • input: gets(), termios library • old-fashioned and unattractive • termios is fiddly
  • 14.
    A simple calculator #!/usr/bin/envruby -w require 'termios' $total = 0 $menu_entries = [['+', "Add"], ['-', "Subtract"], ['*', "Multiply"], ['/', "Divide"], ['c', 'Clear'], ['q',"Quit"]] $commands = $entries.inject([]) { | commands, entry | commands << entry[0] } $captions = $entries.inject([]) { | captions, entry | captions << entry[1] } loop do puts "nSimple Calculatorn" entries.each { | entry | puts "#{entry[0]}. #{entry[1]}n" } t = Termios.tcgetattr(STDIN) t.lflag &= ~Termios::ICANON Termios.tcsetattr(STDIN,0,t) begin action = STDIN.getc.chr end until $commands.member?(action) exit() if action == $commands.last action = $commands.index(action) puts "n#{$captions[action]}nn" case action when 0 : $total += gets() when 1 : $total -= gets() when 2 : $total *= gets() when 3 : $total /= gets() when 4 : $total = 0 end puts "Total = #{$total}" end
  • 15.
    A Ruby packetreader The 7 layer IP model
  • 16.
    What the heck? •We want to look at UDP and DNS traffic • Our first implementation is console-based, so hold on to your hats... We’re exploring the UDP layer
  • 17.
    UDP header inRuby require 'bit-struct' class IP < BitStruct unsigned :ip_v, 4, "Version" unsigned :ip_hl, 4, "Header length" unsigned :ip_tos, 8, "TOS" unsigned :ip_len, 16, "Length" unsigned :ip_id, 16, "ID" unsigned :ip_off, 16, "Frag offset" unsigned :ip_ttl, 8, "TTL" unsigned :ip_p, 8, "Protocol" unsigned :ip_sum, 16, "Checksum" octets :ip_src, 32, "Source addr" octets :ip_dst, 32, "Dest addr" rest :body, "Body of message" note "rest is application defined message body" initial_value.ip_v = 4 initial_value.ip_hl = 5 end class UDP < BitStruct unsigned :udp_srcport, 16, "Source Port" unsigned :udp_dstport, 16, "Dest Port" unsigned :udp_len, 16, "UDP Length" unsigned :udp_chksum, 16, "UDP Checksum" rest :body, "Body of message" note "rest is application defined message body" end class DNSQueryHeader < BitStruct unsigned :dns_id, 16, "ID" unsigned :dns_qr, 1, "QR" unsigned :dns_opcode,4, "OpCode" unsigned :dns_aa, 1, "AA" unsigned :dns_tc, 1, "TC" unsigned :dns_rd, 1, "RD" unsigned :dns_ra, 1, "RA" unsigned :dns_z, 3, "Z" unsigned :dns_rcode, 4, "RCODE" unsigned :dns_qdcount, 16, "QDCount" unsigned :dns_ancount, 16, "ANCount" unsigned :dns_arcount, 16, "ARCount" rest :data, "Data" end class Time # tcpdump style format def to_s sprintf "%0.2d:%0.2d:%0.2d.%0.6d", hour, min, sec, tv_usec end end udpip.rb
  • 18.
    Capturing UDP packets #!/usr/local/bin/ruby require'pcaplet' include Pcap require 'udpip' DIVIDER = "-" * 50 def print_details(section) puts DIVIDER, section, DIVIDER end pcaplet = Pcaplet.new('-s 1500') pcaplet.each_packet { |pkt| if pkt.udp? puts "Packet: #{pkt.time} #{pkt}" if (pkt.sport == 53) udp = UDP.new udp.udp_srcport = pkt.sport udp.udp_dstport = pkt.dport udp.udp_len = pkt.udp_len udp.udp_chksum = pkt.udp_sum udp.body = pkt.udp_data print_details udp.inspect_detailed # look for DNS request only dns = DNSQueryHeader.new(pkt.udp_data) bytearray = Array.new udp.body.each_byte { |c| bytearray.concat(c.to_s.to_a) print c.to_s(16), ' ' } print_details dns.inspect_detailed end end } pcaplet.close tcpdump.rb
  • 19.
    A live UDPpacket
  • 20.
    A live DNSpacket
  • 21.
    Can we havethat on Windows? • A GUI? You gotta be joking!! Why do you think we use Macs? • How about we just turn it into a web application instead? • Sure, we can do that with Ruby • [What have we let ourselves in for...]
  • 22.
    The NDA kicksin • Here’s where we hit the brick wall on what we can talk about You might imagine a DNS-sniffing web application, but we couldn’t possibly comment • So lets get down to some web app basics And yes, we will be kicking it old-skool...
  • 23.
    Introducing WEBrick • WEBrickis an HTTP server library It’s part of the Ruby 1.8 release • It can serve static documents • It can serve HTTPS using Ruby/OpenSSL It can serve arbitrary code blocks • It can serve servlets
  • 24.
    Static content #!/usr/local/bin/ruby require 'webrick' server= WEBrick::HTTPServer.new(:Port => 8080, :DocumentRoot => Dir::pwd + "/htdocs") # mount personal directory, generating directory indexes server.mount("/~eleanor", WEBrick::HTTPServlet::FileHandler, "/Users/eleanor/Sites", true) # catch keyboard interrupt signal to terminate server trap("INT"){ server.shutdown } server.start #!/usr/local/bin/ruby # This requires Ruby/OpenSSL require 'webrick' require 'webrick/https' certificate_name = [ ["C","UK"], ["O","games-with-brains.org"], ["CN", "WWW"] ] server = WEBrick::HTTPServer.new( :DocumentRoot => Dir::pwd + "/htdocs", :SSLEnable => true, :SSLVerifyClient => ::OpenSSL::SSL::VERIFY_NONE, :SSLCertName => certificate_name ) trap("INT"){ s.shutdown } s.start A standard HTTP server An HTTPS server
  • 25.
    Servlets #!/usr/local/bin/ruby require 'webrick' server =WEBrick::GenericServer.new() trap("INT"){ server.shutdown } server.start{|socket| socket.puts("This is a code blockr") } #!/usr/local/bin/ruby require 'webrick' server = WEBrick::HTTPServer.new() trap("INT"){ server.shutdown } def generate_response(response) response.body = "<HTML>hello, world.</HTML>" response['Content-Type'] = "text/html" end class HelloServlet < WEBrick::HTTPServlet::AbstractServlet def do_GET(request, response) generate_response(response) end end server.mount_proc("/hello/simple"){ | request, response | generate_response(response) } server.mount("/hello/advanced", HelloServlet) server.start A Ruby code block A WEBrick servlet
  • 26.
    It’s that simple? •Yes, it’s that simple Of course these are trivial examples... • ...so let’s build an application server
  • 27.
    An application server •Still wondering when we get to the really good stuff? Soon, we promise • But first to show you how NOT to do it!
  • 28.
    Wrap the request classRequestContext attr_reader :request, :response, :servlets, :creation_time def initialize(request, response) @request, @response, = request, response @creation_time = Time.now() end def page_not_found @response.status = WEBrick::HTTPStatus::NotFound.new() end def response_page(page) @response['Content-Type'] = page.content_type @response.body = CGI::pretty(page.to_str()) end def <<(item) @response.body << CGI::pretty(item) end end A basic request context
  • 29.
    Serve the pages IP_ADDRESS_PATTERN= /^d{1,3}.d{1,3}.d{1,3}.d{1,3}/ class ApplicationServer attr_reader :web_server, :server_address, :servlets, :pages def initialize(parameters = {}) @server_address = parameters[:my_address] or raise “Please supply a server address” raise “Invalid IP address for server” unless IP_ADDRESS_PATTERN.match(@server_address) @web_server = WEBrick::HTTPServer.new({:BindAddress => @server_address}) @servlets = {} @pages = {} end def start trap("INT") { @web_server.shutdown } @web_server.start end def register_page(path, page) @pages[path] = page @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @pages[request.path] ? context.response_page(@pages[request.path]) : context.page_not_found() } end def register_method(path, handler) @servlets[path] = self.method(handler).to_proc @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @servlets[request.path] ? (context << @servlets[request.path].call(context).to_str()) : context.page_not_found() } end end The application server
  • 30.
    Write the application #!/usr/local/bin/ruby require'appserver.rb' class SimpleServer < ApplicationServer def initialize(parameters = {}) super register_page("/hello/simple", "<HTML>Hello, world</HTML>") register_method("/hello/advanced", :hello_world) end def hello_world(context) "<HTML>Hello, world</HTML>" end end begin SimpleServer.new({:my_address => ARGV.shift()}).start() rescue RuntimeError => e $stderr.puts "Usage: simpleserver host-address" $stderr.puts "address must be provided in dotted-quad format (i.e. xxx.xxx.xxx.xxx)" end Revisiting “hello, world”
  • 31.
    What have wedone?!? • On the surface this is elegant • But underneath it sucks • There’s no support for HTML • Only methods can be used as servlets • We’re tied to WEBrick - which is slow
  • 32.
    The road toperdition • So we added an HTML 4 library • And a server pages container • And ActiveRecord • We meta’d the code to death • But it still lacked va-va-voom...
  • 33.
    The case forRails • So perhaps we should have just used Rails in the first place • We’d be another of those “Rails saved my career” success stories! • Hindsight’s always 20/20 • But we’re old-school coders and it’s far too user friendly for our comfort
  • 34.
    The pressure against •Working at a very low level • Simple code required • Can Rails talk nicely to low-level code? • Strong management resistance - too high a learning curve?
  • 35.
    So why Camping? •Camping is beauty incarnate • It’s less than 4K of code • It uses Markaby and ActiveRecord • It runs on JRuby!!! • Oh, and it’s great fun to abuse...
  • 36.
  • 37.
    Markaby • An XHTMLDomain Specific Language • Allows you to embed XHTML code in Ruby code without building a complex object hierarchy • Can be used with Rails
  • 38.
    But that’s sosimple! require 'markaby' page = Markaby::Builder.new page.xhtml_strict do head { title "Camping Presentation" } body do h1.page_heading "Camping: Going off the Rails with Ruby" ul.page_index do li.page_index { a “introduction”, :href => ‘#introduction’ } li.page_index { a “the presentation”, :href => ‘/presentation’ } li.page_index { a “comments”, :href => ‘#comments’ } end div.introduction! { “Everything will be alright!!!” } div.comments! { “Have your say” } end end puts page.to_s Markaby embedded in Ruby <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd"> <html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/> <title>Camping Presentation</title> </head> <body> <h1 class="page_heading">Camping: Going off the Rails with Ruby</h1> <ul class="page_index"> <li class="page_index"><a href="#introduction">introduction</a></li> <li class="page_index"><a href="/presentation">the presentation</a></li> <li class="page_index"><a href="#comments">comments</a></li> </ul> <div id="introduction">Just breathe deeply...</div> <div id="comments">Have your say</div> </body> </html> Creates this
  • 39.
    ActiveRecord • An Object-RelationalMapper • Implements the Active Record pattern • Supports many popular databases • A key component of Rails
  • 40.
    ORMtastic Using Active Record require'rubygems' require_gem ‘activerecord’ ActiveRecord::Base.establish_connection(:adapter => “sqlite3”, :host => “localhost”, :database => “test.db”) class User < ActiveRecord::Base end user = User.new() user.id = “ellie” user.name = “Eleanor McHugh” user.password = “somerandomtext” user.save user = User.find(“ellie”) user.destroy()
  • 41.
    Totally RAD • Campingbuilds small applications • Why’s guideline? One file per application • If that’s how you prefer it...
  • 42.
    A simple example Basicsetup #!/usr/bin/env ruby $:.unshift File.dirname(__FILE__) + "/../../lib" require 'camping' require 'camping/session' Camping.goes :Jotter module Blog include Camping::Session end • Load the camping libraries • Define a namespace for the application • Include session support (if required)
  • 43.
    The data model uleJotter::Models class Note < Base; end class Database < V 1.0 def self.up create_table :jotter_notes, :force => true do |t| t.column :id, :integer, :null => false t.column :created_at, :interger, :null => false t.column :title, :string, :limit => 255 t.column :body, :text end end def self.down drop_table :jotter_notes end end Jotter.create Jotter::Models.create_schema Defining the data model • We mark our database as version 1.0 • A create method builds the database
  • 44.
    The controllers Adding controllers moduleJotter::Controllers class Static < R '/static/(.+)' MIME_TYPES = {'.css' => 'text/css', '.js' => 'text/javascript', '.jpg' => 'image/jpeg'} PATH = __FILE__[/(.*)//, 1] def get(path) @headers['Content-Type'] = MIME_TYPES[path[/.w+$/, 0]] || "text/plain" @headers['X-Sendfile'] = "#{PATH}/static/#{path}" end end class Index < R '/' def get @notes = Note.find :all render :index end end class View < R '/view/(d+)' def get note_id @note = Note.find post_id render :view end end class Add < R ‘/add/’ def get @note = Note.new render :add end def post note = Note.create :title => input.post_title, :body => input.post_body redirect View, post end end
  • 45.
    The controllers class Edit< R '/edit/(d+)', '/edit' def get note_id @note = Note.find note_id render :edit end def post @note = Note.find input.note_id @note.update_attributes :title => input.post_title, :body => input.post_body redirect View, @note end end class Delete < R '/delete/(d+)' def get note_id @note = Note.find note_id @note.destroy redirect Index end end end Adding controllers • Respond to HTTP GET and POST requests • Perform database operations
  • 46.
    The views Application views moduleJotter::Views def layout xhtml_strict do head do title 'blog' link :rel => 'stylesheet', :type => 'text/css', :href => '/static/styles.css', :media => 'screen' end body do h1.header { a 'jotter', :href => R(Index) } div.body do self << yield end end end end def index @notes.empty? (p 'No posts found.') : (ol.row! { _list_notes(@notes) }) p { a 'new note', :href => R(Add) } end def edit _form(@note, :action => R(Edit)) end def view h1 @note.title h2 @note.created_at p @note.body p do [ a("View", :href => R(View, @note)), a("Edit", :href => R(Edit, @note)), a("Delete", :href => R(View, @note)) ].join " | " end end
  • 47.
    The views def _list_notes(notes) @notes.eachdo | note | li do ul do li { a note.title, :href => R(View, note) } li note.created_at li { a "Edit", :href => R(Edit, note) } li { a "Delete", :href => R(Delete, note) } end end end end def _form(note, opts) form({:method => 'post'}.merge(opts)) do label 'Title', :for => 'note_title'; br input :name => 'note_title', :type => 'text', :value => note.title; br label 'Body', :for => 'note_body'; br textarea note.body, :name => 'note_body'; br input :type => 'hidden', :name => 'note_id', :value => note.id input :type => 'submit' end end Application views • Views incorporate Markaby for XHTML • Have access to controller data
  • 48.
    The post-amble A basicCGI post-amble if __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter.create if Jotter.respond_to? :create puts Jotter.run end if __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter::Models::Base.threaded_connections = false Jotter.create if Jotter.respond_to? :create server = Mongrel::Camping::start(“0.0.0.0”, 3000, “/jotter”, Jotter) puts “Jotter application running at http://localhost:3000/jotter” server.run.join end A Mongrel post-amble • Allows an application to self-execute • Can be customised to suit your platform
  • 49.
    The style-sheet A simplestyle sheet body { font-family: Utopia, Georga, serif; } h1.header { background-color: #fef; margin: 0; padding: 10px; } div.body { padding: 10px; } #row ul { list-style: none; margin: 0; padding: 0; padding-top: 4px; } #row li { display: inline; } #row a:link, #row a:visited { padding: 3px 10px 2px 10px; color: #FFFFFF; background-color: #B51032; text-decoration: none; border: 1px solid #711515; }
  • 50.
    Larger applications • Oneapplication per file is a nice idea • But what about large applications? • Each can be broken down into discrete micro-applications • Each micro-application has its own file and mount points
  • 51.
    Sharing a database •Camping apps keep their database tables in separate namespaces • Larger applications will want to share state between micro-applications • We could do some ActiveRecord voodoo • Or we could cheat... guess which?
  • 52.
    Camping in thewilds require 'rubygems' require_gem 'camping', '>=1.4' require 'camping/session' module Camping module Models def self.schema(&block) @@schema = block if block_given? @@schema end class User < Base validates_uniqueness_of :name, :scope => :id validates_presence_of :password end end def self.create Camping::Models::Session.create_schema ActiveRecord::Schema.define(&Models.schema) end Models.schema do unless Models::User.table_exists? create_table :users, :force => true do | t | t.column :id, :integer, :null => false t.column :created_on, :integer, :null => false t.column :name, :string, :null => false t.column :password, :string, :null => false t.column :comment, :string, :null => false end execute "INSERT INTO users (created_on, name, password, comment) VALUES ('#{Time.now}', 'admin', 'admin', 'system administrator')" end end end Installing a database in the framework
  • 53.
    Camping server • Thecamping server ties together a series of web applications • A simple implementation ships with the framework
  • 54.
    The server rules •Monitor a directory • load/reload all camping apps that appear in it or a subdirectory • Mount apps according to the filenames (i.e. jotter.rb mounts as /jotter) • Run create method on app startup • Support the X-Sendfile header
  • 55.
    Summing up • Webapplications are useful outside the usual web app environment • Cross platform is easy when you only need an XHTML browser • These tasks need a lightweight design • Camping is a good way to solve them • And as you can see, Ruby rocks!!!
  • 56.