Published on


Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Camping: Going off the Rails with Ruby Adventures in creative coding for people who should know better
  2. 2. The weasel words • This presentation contains code That code is probably broken • If that bothers you - fix it That’s called a learning experience
  3. 3. Who are these lunatics? Romek Szczesniak romek@spikyblackcat.co.uk He does security Eleanor McHugh eleanor@games-with-brains.com She does real-time systems
  4. 4. Alright, but what are they doing here? • Ruby Pcap & BitStruct • WEBrick • Camping • but no Rails...
  5. 5. No Rails? • That’s right, we don’t use Rails But we do use Ruby • And we do write web applications So how is that possible?
  6. 6. Camping!!! • That’s right, we use Camping It’s by Why The Lucky Stiff • It’s cool • It’s really cool • It’s so damn cool you’d have to be mad not to use it!!!
  7. 7. It’s this simple! %w[rubygems active_record markaby metaid ostruct].each {|lib| require lib} module Camping;C=self;module Models;end;Models::Base=ActiveRecord::Base module Helpers;def R c,*args;p=/(.+?)/;args.inject(c.urls.detect{|x|x. scan(p).size==args.size}.dup){|str,a|str.gsub(p,(a.method(a.class.primary_key )[]rescue a).to_s)};end;def / p;File.join(@root,p) end;end;module Controllers module Base;include Helpers;attr_accessor :input,:cookies,:headers,:body, :status,:root;def method_missing(m,*args,&blk);str=m==:render ? markaview( *args,&blk):eval("markaby.#{m}(*args,&blk)");str=markaview(:layout){str }rescue nil;r(200,str.to_s);end;def r(s,b,h={});@status=s;@headers.merge!(h) @body=b;end;def redirect(c,*args);c=R(c,*args)if c.respond_to?:urls;r(302,'', 'Location'=>self/c);end;def service(r,e,m,a);@status,@headers,@root=200,{},e[ 'SCRIPT_NAME'];@cookies=C.cookie_parse(e['HTTP_COOKIE']||e['COOKIE']);cook= @cookies.marshal_dump.dup;if ("POST"==e['REQUEST_METHOD'])and %r|Amultipart /form-data.*boundary="?([^";,]+)"?|n.match(e['CONTENT_TYPE']);return r(500, "No multipart/form-data supported.")else;@input=C.qs_parse(e['REQUEST_METHOD' ]=="POST"?r.read(e['CONTENT_LENGTH'].to_i):e['QUERY_STRING']);end;@body= method(m.downcase).call(*a);@headers["Set-Cookie"]=@cookies.marshal_dump.map{ |k,v|"#{k}=#{C.escape(v)}; path=/"if v != cook[k]}.compact;self;end;def to_s "Status: #{@status}n#{{'Content-Type'=>'text/html'}.merge(@headers).map{|k,v| v.to_a.map{|v2|"#{k}: #{v2}"}}.flatten.join("n")}nn#{@body}";end;def markaby;Class.new(Markaby::Builder){@root=@root;include Views;def tag!(*g,&b) [:href,:action].each{|a|(g.last[a]=self./(g.last[a]))rescue 0};super end}.new( instance_variables.map{|iv|[iv[1..-1].intern,instance_variable_get(iv)]},{}) end;def markaview(m,*args,&blk);markaby.instance_eval{Views.instance_method(m ).bind(self).call(*args, &blk);self}.to_s;end;end;class R;include Base end class NotFound<R;def get(p);r(404,div{h1("#{C} Problem!")+h2("#{p} not found") });end end;class ServerError<R;def get(k,m,e);r(500,markaby.div{h1 "#{C} Prob lem!";h2 "#{k}.#{m}";h3 "#{e.class} #{e.message}:";ul{e.backtrace.each{|bt|li( bt)}}})end end;class<<self;def R(*urls);Class.new(R){meta_def(:inherited){|c| c.meta_def(:urls){urls}}};end;def D(path);constants.each{|c|k=const_get(c) return k,$~[1..-1] if (k.urls rescue "/#{c.downcase}").find {|x|path=~/^#{x} /?$/}};[NotFound,[path]];end end end;class<<self;def escape(s);s.to_s.gsub( /([^ a-zA-Z0-9_.-]+)/n){'%'+$1.unpack('H2'*$1.size).join('%').upcase}.tr(' ', '+') end;def unescape(s);s.tr('+', ' ').gsub(/((?:%[0-9a-fA-F]{2})+)/n){[$1. delete('%')].pack('H*')} end;def qs_parse(qs,d='&;');OpenStruct.new((qs||''). split(/[#{d}] */n).inject({}){|hsh,p|k,v=p.split('=',2).map{|v|unescape(v)} hsh[k]=v unless v.empty?;hsh}) end;def cookie_parse(s);c=qs_parse(s,';,') end def run(r=$stdin,w=$stdout);w<<begin;k,a=Controllers.D "/#{ENV['PATH_INFO']}". gsub(%r!/+!,'/');m=ENV['REQUEST_METHOD']||"GET";k.class_eval{include C include Controllers::Base;include Models};o=k.new;o.service(r,ENV,m,a);rescue =>e;Controllers::ServerError.new.service(r,ENV,"GET",[k,m,e]);end;end;end module Views; include Controllers; include Helpers end;end
  8. 8. Why? • For fun For profit For the satisfaction of knowing exactly how your application works • For the look on your boss’s face when he reads the documentation
  9. 9. Earlier... • But let’s not get ahead of ourselves First we want to take you on a journey • A journey back in time A journey back to...
  10. 10. January 3rd 2006 Location: The Secret Basement Lairtm of Captain IP and The DNS avengers • Their task: to launch a new Top Level Domain which DOESN’T RESOLVE MACHINE ADDRESSES?!?!?! Their resources? Hands to wave with and hit keyboards with!
  11. 11. Ruby to the Rescue • It’s easy to learn It’s quick to code in • It’s pleasing to the eye It’s fun!
  12. 12. You keep saying that • Yes!!! Fun makes for better coders • Better coders write good code • Good code stands the test of time • If coding isn’t fun YOU’RE USING THE WRONG TOOLS!!!!
  13. 13. The console jockeys • let’s write a menu driven calculator output: puts(), print() • input: gets(), termios library • old-fashioned and unattractive • termios is fiddly
  14. 14. A simple calculator #!/usr/bin/env ruby -w require 'termios' $total = 0 $menu_entries = [['+', "Add"], ['-', "Subtract"], ['*', "Multiply"], ['/', "Divide"], ['c', 'Clear'], ['q',"Quit"]] $commands = $entries.inject([]) { | commands, entry | commands << entry[0] } $captions = $entries.inject([]) { | captions, entry | captions << entry[1] } loop do puts "nSimple Calculatorn" entries.each { | entry | puts "#{entry[0]}. #{entry[1]}n" } t = Termios.tcgetattr(STDIN) t.lflag &= ~Termios::ICANON Termios.tcsetattr(STDIN,0,t) begin action = STDIN.getc.chr end until $commands.member?(action) exit() if action == $commands.last action = $commands.index(action) puts "n#{$captions[action]}nn" case action when 0 : $total += gets() when 1 : $total -= gets() when 2 : $total *= gets() when 3 : $total /= gets() when 4 : $total = 0 end puts "Total = #{$total}" end
  15. 15. A Ruby packet reader The 7 layer IP model
  16. 16. What the heck? • We want to look at UDP and DNS traffic • Our first implementation is console-based, so hold on to your hats... We’re exploring the UDP layer
  17. 17. UDP header in Ruby require 'bit-struct' class IP < BitStruct unsigned :ip_v, 4, "Version" unsigned :ip_hl, 4, "Header length" unsigned :ip_tos, 8, "TOS" unsigned :ip_len, 16, "Length" unsigned :ip_id, 16, "ID" unsigned :ip_off, 16, "Frag offset" unsigned :ip_ttl, 8, "TTL" unsigned :ip_p, 8, "Protocol" unsigned :ip_sum, 16, "Checksum" octets :ip_src, 32, "Source addr" octets :ip_dst, 32, "Dest addr" rest :body, "Body of message" note "rest is application defined message body" initial_value.ip_v = 4 initial_value.ip_hl = 5 end class UDP < BitStruct unsigned :udp_srcport, 16, "Source Port" unsigned :udp_dstport, 16, "Dest Port" unsigned :udp_len, 16, "UDP Length" unsigned :udp_chksum, 16, "UDP Checksum" rest :body, "Body of message" note "rest is application defined message body" end class DNSQueryHeader < BitStruct unsigned :dns_id, 16, "ID" unsigned :dns_qr, 1, "QR" unsigned :dns_opcode,4, "OpCode" unsigned :dns_aa, 1, "AA" unsigned :dns_tc, 1, "TC" unsigned :dns_rd, 1, "RD" unsigned :dns_ra, 1, "RA" unsigned :dns_z, 3, "Z" unsigned :dns_rcode, 4, "RCODE" unsigned :dns_qdcount, 16, "QDCount" unsigned :dns_ancount, 16, "ANCount" unsigned :dns_arcount, 16, "ARCount" rest :data, "Data" end class Time # tcpdump style format def to_s sprintf "%0.2d:%0.2d:%0.2d.%0.6d", hour, min, sec, tv_usec end end udpip.rb
  18. 18. Capturing UDP packets #!/usr/local/bin/ruby require 'pcaplet' include Pcap require 'udpip' DIVIDER = "-" * 50 def print_details(section) puts DIVIDER, section, DIVIDER end pcaplet = Pcaplet.new('-s 1500') pcaplet.each_packet { |pkt| if pkt.udp? puts "Packet: #{pkt.time} #{pkt}" if (pkt.sport == 53) udp = UDP.new udp.udp_srcport = pkt.sport udp.udp_dstport = pkt.dport udp.udp_len = pkt.udp_len udp.udp_chksum = pkt.udp_sum udp.body = pkt.udp_data print_details udp.inspect_detailed # look for DNS request only dns = DNSQueryHeader.new(pkt.udp_data) bytearray = Array.new udp.body.each_byte { |c| bytearray.concat(c.to_s.to_a) print c.to_s(16), ' ' } print_details dns.inspect_detailed end end } pcaplet.close tcpdump.rb
  19. 19. A live UDP packet
  20. 20. A live DNS packet
  21. 21. Can we have that on Windows? • A GUI? You gotta be joking!! Why do you think we use Macs? • How about we just turn it into a web application instead? • Sure, we can do that with Ruby • [What have we let ourselves in for...]
  22. 22. The NDA kicks in • Here’s where we hit the brick wall on what we can talk about You might imagine a DNS-sniffing web application, but we couldn’t possibly comment • So lets get down to some web app basics And yes, we will be kicking it old-skool...
  23. 23. Introducing WEBrick • WEBrick is an HTTP server library It’s part of the Ruby 1.8 release • It can serve static documents • It can serve HTTPS using Ruby/OpenSSL It can serve arbitrary code blocks • It can serve servlets
  24. 24. Static content #!/usr/local/bin/ruby require 'webrick' server = WEBrick::HTTPServer.new(:Port => 8080, :DocumentRoot => Dir::pwd + "/htdocs") # mount personal directory, generating directory indexes server.mount("/~eleanor", WEBrick::HTTPServlet::FileHandler, "/Users/eleanor/Sites", true) # catch keyboard interrupt signal to terminate server trap("INT"){ server.shutdown } server.start #!/usr/local/bin/ruby # This requires Ruby/OpenSSL require 'webrick' require 'webrick/https' certificate_name = [ ["C","UK"], ["O","games-with-brains.org"], ["CN", "WWW"] ] server = WEBrick::HTTPServer.new( :DocumentRoot => Dir::pwd + "/htdocs", :SSLEnable => true, :SSLVerifyClient => ::OpenSSL::SSL::VERIFY_NONE, :SSLCertName => certificate_name ) trap("INT"){ s.shutdown } s.start A standard HTTP server An HTTPS server
  25. 25. Servlets #!/usr/local/bin/ruby require 'webrick' server = WEBrick::GenericServer.new() trap("INT"){ server.shutdown } server.start{|socket| socket.puts("This is a code blockr") } #!/usr/local/bin/ruby require 'webrick' server = WEBrick::HTTPServer.new() trap("INT"){ server.shutdown } def generate_response(response) response.body = "<HTML>hello, world.</HTML>" response['Content-Type'] = "text/html" end class HelloServlet < WEBrick::HTTPServlet::AbstractServlet def do_GET(request, response) generate_response(response) end end server.mount_proc("/hello/simple"){ | request, response | generate_response(response) } server.mount("/hello/advanced", HelloServlet) server.start A Ruby code block A WEBrick servlet
  26. 26. It’s that simple? • Yes, it’s that simple Of course these are trivial examples... • ...so let’s build an application server
  27. 27. An application server • Still wondering when we get to the really good stuff? Soon, we promise • But first to show you how NOT to do it!
  28. 28. Wrap the request class RequestContext attr_reader :request, :response, :servlets, :creation_time def initialize(request, response) @request, @response, = request, response @creation_time = Time.now() end def page_not_found @response.status = WEBrick::HTTPStatus::NotFound.new() end def response_page(page) @response['Content-Type'] = page.content_type @response.body = CGI::pretty(page.to_str()) end def <<(item) @response.body << CGI::pretty(item) end end A basic request context
  29. 29. Serve the pages IP_ADDRESS_PATTERN = /^d{1,3}.d{1,3}.d{1,3}.d{1,3}/ class ApplicationServer attr_reader :web_server, :server_address, :servlets, :pages def initialize(parameters = {}) @server_address = parameters[:my_address] or raise “Please supply a server address” raise “Invalid IP address for server” unless IP_ADDRESS_PATTERN.match(@server_address) @web_server = WEBrick::HTTPServer.new({:BindAddress => @server_address}) @servlets = {} @pages = {} end def start trap("INT") { @web_server.shutdown } @web_server.start end def register_page(path, page) @pages[path] = page @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @pages[request.path] ? context.response_page(@pages[request.path]) : context.page_not_found() } end def register_method(path, handler) @servlets[path] = self.method(handler).to_proc @web_server.mount_proc(path) { | request, response | context = RequestContext.new(request, response) @servlets[request.path] ? (context << @servlets[request.path].call(context).to_str()) : context.page_not_found() } end end The application server
  30. 30. Write the application #!/usr/local/bin/ruby require 'appserver.rb' class SimpleServer < ApplicationServer def initialize(parameters = {}) super register_page("/hello/simple", "<HTML>Hello, world</HTML>") register_method("/hello/advanced", :hello_world) end def hello_world(context) "<HTML>Hello, world</HTML>" end end begin SimpleServer.new({:my_address => ARGV.shift()}).start() rescue RuntimeError => e $stderr.puts "Usage: simpleserver host-address" $stderr.puts "address must be provided in dotted-quad format (i.e. xxx.xxx.xxx.xxx)" end Revisiting “hello, world”
  31. 31. What have we done?!? • On the surface this is elegant • But underneath it sucks • There’s no support for HTML • Only methods can be used as servlets • We’re tied to WEBrick - which is slow
  32. 32. The road to perdition • So we added an HTML 4 library • And a server pages container • And ActiveRecord • We meta’d the code to death • But it still lacked va-va-voom...
  33. 33. The case for Rails • So perhaps we should have just used Rails in the first place • We’d be another of those “Rails saved my career” success stories! • Hindsight’s always 20/20 • But we’re old-school coders and it’s far too user friendly for our comfort
  34. 34. The pressure against • Working at a very low level • Simple code required • Can Rails talk nicely to low-level code? • Strong management resistance - too high a learning curve?
  35. 35. So why Camping? • Camping is beauty incarnate • It’s less than 4K of code • It uses Markaby and ActiveRecord • It runs on JRuby!!! • Oh, and it’s great fun to abuse...
  36. 36. Gratuitous diagram lifted from http://redhanded.hobix.com/bits/campingAMicroframework.html How Why? The Lucky Stiff teaches it
  37. 37. Markaby • An XHTML Domain Specific Language • Allows you to embed XHTML code in Ruby code without building a complex object hierarchy • Can be used with Rails
  38. 38. But that’s so simple! require 'markaby' page = Markaby::Builder.new page.xhtml_strict do head { title "Camping Presentation" } body do h1.page_heading "Camping: Going off the Rails with Ruby" ul.page_index do li.page_index { a “introduction”, :href => ‘#introduction’ } li.page_index { a “the presentation”, :href => ‘/presentation’ } li.page_index { a “comments”, :href => ‘#comments’ } end div.introduction! { “Everything will be alright!!!” } div.comments! { “Have your say” } end end puts page.to_s Markaby embedded in Ruby <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd"> <html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"> <head> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/> <title>Camping Presentation</title> </head> <body> <h1 class="page_heading">Camping: Going off the Rails with Ruby</h1> <ul class="page_index"> <li class="page_index"><a href="#introduction">introduction</a></li> <li class="page_index"><a href="/presentation">the presentation</a></li> <li class="page_index"><a href="#comments">comments</a></li> </ul> <div id="introduction">Just breathe deeply...</div> <div id="comments">Have your say</div> </body> </html> Creates this
  39. 39. ActiveRecord • An Object-Relational Mapper • Implements the Active Record pattern • Supports many popular databases • A key component of Rails
  40. 40. ORMtastic Using Active Record require 'rubygems' require_gem ‘activerecord’ ActiveRecord::Base.establish_connection(:adapter => “sqlite3”, :host => “localhost”, :database => “test.db”) class User < ActiveRecord::Base end user = User.new() user.id = “ellie” user.name = “Eleanor McHugh” user.password = “somerandomtext” user.save user = User.find(“ellie”) user.destroy()
  41. 41. Totally RAD • Camping builds small applications • Why’s guideline? One file per application • If that’s how you prefer it...
  42. 42. A simple example Basic setup #!/usr/bin/env ruby $:.unshift File.dirname(__FILE__) + "/../../lib" require 'camping' require 'camping/session' Camping.goes :Jotter module Blog include Camping::Session end • Load the camping libraries • Define a namespace for the application • Include session support (if required)
  43. 43. The data model ule Jotter::Models class Note < Base; end class Database < V 1.0 def self.up create_table :jotter_notes, :force => true do |t| t.column :id, :integer, :null => false t.column :created_at, :interger, :null => false t.column :title, :string, :limit => 255 t.column :body, :text end end def self.down drop_table :jotter_notes end end Jotter.create Jotter::Models.create_schema Defining the data model • We mark our database as version 1.0 • A create method builds the database
  44. 44. The controllers Adding controllers module Jotter::Controllers class Static < R '/static/(.+)' MIME_TYPES = {'.css' => 'text/css', '.js' => 'text/javascript', '.jpg' => 'image/jpeg'} PATH = __FILE__[/(.*)//, 1] def get(path) @headers['Content-Type'] = MIME_TYPES[path[/.w+$/, 0]] || "text/plain" @headers['X-Sendfile'] = "#{PATH}/static/#{path}" end end class Index < R '/' def get @notes = Note.find :all render :index end end class View < R '/view/(d+)' def get note_id @note = Note.find post_id render :view end end class Add < R ‘/add/’ def get @note = Note.new render :add end def post note = Note.create :title => input.post_title, :body => input.post_body redirect View, post end end
  45. 45. The controllers class Edit < R '/edit/(d+)', '/edit' def get note_id @note = Note.find note_id render :edit end def post @note = Note.find input.note_id @note.update_attributes :title => input.post_title, :body => input.post_body redirect View, @note end end class Delete < R '/delete/(d+)' def get note_id @note = Note.find note_id @note.destroy redirect Index end end end Adding controllers • Respond to HTTP GET and POST requests • Perform database operations
  46. 46. The views Application views module Jotter::Views def layout xhtml_strict do head do title 'blog' link :rel => 'stylesheet', :type => 'text/css', :href => '/static/styles.css', :media => 'screen' end body do h1.header { a 'jotter', :href => R(Index) } div.body do self << yield end end end end def index @notes.empty? (p 'No posts found.') : (ol.row! { _list_notes(@notes) }) p { a 'new note', :href => R(Add) } end def edit _form(@note, :action => R(Edit)) end def view h1 @note.title h2 @note.created_at p @note.body p do [ a("View", :href => R(View, @note)), a("Edit", :href => R(Edit, @note)), a("Delete", :href => R(View, @note)) ].join " | " end end
  47. 47. The views def _list_notes(notes) @notes.each do | note | li do ul do li { a note.title, :href => R(View, note) } li note.created_at li { a "Edit", :href => R(Edit, note) } li { a "Delete", :href => R(Delete, note) } end end end end def _form(note, opts) form({:method => 'post'}.merge(opts)) do label 'Title', :for => 'note_title'; br input :name => 'note_title', :type => 'text', :value => note.title; br label 'Body', :for => 'note_body'; br textarea note.body, :name => 'note_body'; br input :type => 'hidden', :name => 'note_id', :value => note.id input :type => 'submit' end end Application views • Views incorporate Markaby for XHTML • Have access to controller data
  48. 48. The post-amble A basic CGI post-amble if __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter.create if Jotter.respond_to? :create puts Jotter.run end if __FILE__ == $0 Jotter::Models::Base.establish_connection :adapter => 'sqlite3', :database => 'notes.db' Jotter::Models::Base.logger = Logger.new('camping.log') Jotter::Models::Base.threaded_connections = false Jotter.create if Jotter.respond_to? :create server = Mongrel::Camping::start(“”, 3000, “/jotter”, Jotter) puts “Jotter application running at http://localhost:3000/jotter” server.run.join end A Mongrel post-amble • Allows an application to self-execute • Can be customised to suit your platform
  49. 49. The style-sheet A simple style sheet body { font-family: Utopia, Georga, serif; } h1.header { background-color: #fef; margin: 0; padding: 10px; } div.body { padding: 10px; } #row ul { list-style: none; margin: 0; padding: 0; padding-top: 4px; } #row li { display: inline; } #row a:link, #row a:visited { padding: 3px 10px 2px 10px; color: #FFFFFF; background-color: #B51032; text-decoration: none; border: 1px solid #711515; }
  50. 50. Larger applications • One application per file is a nice idea • But what about large applications? • Each can be broken down into discrete micro-applications • Each micro-application has its own file and mount points
  51. 51. Sharing a database • Camping apps keep their database tables in separate namespaces • Larger applications will want to share state between micro-applications • We could do some ActiveRecord voodoo • Or we could cheat... guess which?
  52. 52. Camping in the wilds require 'rubygems' require_gem 'camping', '>=1.4' require 'camping/session' module Camping module Models def self.schema(&block) @@schema = block if block_given? @@schema end class User < Base validates_uniqueness_of :name, :scope => :id validates_presence_of :password end end def self.create Camping::Models::Session.create_schema ActiveRecord::Schema.define(&Models.schema) end Models.schema do unless Models::User.table_exists? create_table :users, :force => true do | t | t.column :id, :integer, :null => false t.column :created_on, :integer, :null => false t.column :name, :string, :null => false t.column :password, :string, :null => false t.column :comment, :string, :null => false end execute "INSERT INTO users (created_on, name, password, comment) VALUES ('#{Time.now}', 'admin', 'admin', 'system administrator')" end end end Installing a database in the framework
  53. 53. Camping server • The camping server ties together a series of web applications • A simple implementation ships with the framework
  54. 54. The server rules • Monitor a directory • load/reload all camping apps that appear in it or a subdirectory • Mount apps according to the filenames (i.e. jotter.rb mounts as /jotter) • Run create method on app startup • Support the X-Sendfile header
  55. 55. Summing up • Web applications are useful outside the usual web app environment • Cross platform is easy when you only need an XHTML browser • These tasks need a lightweight design • Camping is a good way to solve them • And as you can see, Ruby rocks!!!
  56. 56. • http://code.whytheluckystiff.net/camping/wiki http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html • http://raa.ruby-lang.org/project/bit-struct/ http://raa.ruby-lang.org/project/ruby-termios/ Where to next?