• Save
CloudTunnel Atlanta Ruby Users Group October 2012
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

CloudTunnel Atlanta Ruby Users Group October 2012

on

  • 286 views

 

Statistics

Views

Total Views
286
Views on SlideShare
264
Embed Views
22

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 22

http://blog.manuzak.com 22

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • - Let’s get the hard questions out of the way first.\n- For those of you who don’t know me, my name is Jonathan Manuzak\n- I’m the lead developer at a startup called CodeGuard, where we do website backup and monitoring\n\n
  • \n
  • Like most of you, I do most of my development on a laptop\nOk, maybe it looks more like this...\n
  • What’s more interesting is where I could be working from. I might be at the office, my house, a coffee shop, an airport or who knows where\n\n
  • - The one thing all of these situations have in common is that I’m behind a firewall or NAT device\n\nLet’s take a second to talk about these firewalls or nat devices\n
  • Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
  • Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
  • Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
  • Firewalls are like diodes, they really prefer for things to go in one direction. In the case of firewalls, it’s out. You can connect to websites, email, maybe FTP.\n\n- Even if you could make these changes they can be a security risk\n- It’s really never accessible to the rest of the world\n
  • - But, the rest of the world has lots of cool stuff and people!\n\n\n
  • - You may want to share an app you’re working on with a client instead of sending screenshots\n\n
  • - Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nThis stuff can be painful to develop if you have to keep deploying, testing, deploying, testing\n\n\n\n\n
  • - Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nThis stuff can be painful to develop if you have to keep deploying, testing, deploying, testing\n\n\n\n\n
  • - Then there are services, full of webhooks and callbacks\n- OAuth Authentication providers\n- Payments\n- Communications\n\nWith all of these big companies -- facebook, twitter, google...\n\n
  • - With all of those companies on the previous page who claim to support developers, this should be a non-issue, right?\n
  • Sure, there are a few ways around it\n
  • Sure, there are a few ways around it\n
  • - Just fire up a ssh tunnel\n- Has anyone here used SSH tunnels before?\nDid you know that SSH was released in 1995 and by the end of that year it had over 20,000 users. Not bad growth for one year, right?\n
  • Assume you have a local webserver running on port 8080\nIt’s sunday afternoon, you’re at home, and you want to show someone this cool app you’re working on\nMaybe it’s your Grandma\n
  • ... and she’s at her house in a different state\nLets also assume that you have a linode or slicehost...\n
  • ...some kind server floating around...\n
  • ... out there on the internet\nNow, how do we get grandma to see your local webserver?\nUsing some variation of that ugly command I showed earlier, we can create a tunnel from your server to your laptop \n\n
  • The first part of the tunnel is an SSH connection that’s made from your laptop out, through your firewall/router to the remote server\nThen the tunnel is created which essentially binds two ports on either side of the ssh connection\n\n
  • Thereby sending all traffic from port 80 on the server through the tunnel to port 8080 on your local machine\n\n
  • Then all grandma has to do is connect to the remote server in her browser and she’ll see what you see\n\n
  • SSH tunnels are easy if you have all of the right ingredients\n\n- Outside of your firewall\n- For ssh\n- ...because you’re probably going to need at root or sudo access to do this\n\nSo if you have all of that, it’s just a simple one-line command\n
  • I’m lazy, so the appeal of running my own server and having to run this command every time I want to use an ssh tunnel is not appealing.\n\n
  • \n
  • Localhost maintains the public server for you \nIt also provides a nice client to hide the ugly tunnel creation stuff\n\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
  • Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
  • Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
  • Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
  • Localtunnel is awesome, but it has a few issues that make it a pain to use more than occasionally\n\n- First, the hostname is random. It changes every time you connect.\n- Idle timeout\n\n- If you change network connections, restart your server, put your laptop to sleep or do anything else to interrupt that socket connection you’re going to have to restart localtunnel.\n
  • \n
  • - There are some alternatives to localtunnel\n- Both showoff io and pagekite allow custom cnames, but as far as I can tell still suffer from the other issues.\n- So, just like many engineers, rather than spending the princely sum of $4 or $5 per month, I decided to solve the solution myself using localtunnel.\n
  • Quick side note. This is what you see if you search for ‘cloud tunnel’ \n\nIt’s a type of arcus cloud, called a roll cloud. They’re relatively rare, and appear to be when an incoming cold sea breeze meets the warmer opposing landmass.\n
  • The service maintains a record of your localtunnel address and forwards users from a hostname you specify to your current localtunnel\n- Runs on Heroku\n
  • The service maintains a record of your localtunnel address and forwards users from a hostname you specify to your current localtunnel\n- Runs on Heroku\n
  • Lets take a closer look at what was going on during the demo\nIf we look at the processlist, we’d notice 3 cloud_tunnel processes\nThe monitor is created by the deamon to watch over, start, and stop the primary cloud_tunnel process\n- If you noticed during the first demo, the localtunnel client blocked until it was killed\n
  • Lets take a closer look at what was going on during the demo\nIf we look at the processlist, we’d notice 3 cloud_tunnel processes\nThe monitor is created by the deamon to watch over, start, and stop the primary cloud_tunnel process...\n\n
  • This so called primary tunnel process handles the creation and maintenance of the localtunnel client connection\n- If you noticed during the first demo, the localtunnel client blocked until it was killed\n- For compatibility, I wanted to re-use the localtunnel client instead of reimplementing it myself\nThis adds a bit of complexity, but with a new version of localtunnel on the horizon it will make forward compatibility easier\n\n
  • Finally, this is the localtunnel client which was forked from the primary cloud_tunnel process\n\n
  • This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • After creating a new instance of the LocalTunnel client, the tunnel is opened (which blocks), so we detach the process\n
  • Process allows better management\nBut we’re kind of using threads. Thread waits on \n
  • This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
  • This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
  • This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
  • This is the wrapper class that handles the LocalTunnel client. The meat of it is here...\n
  • Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
  • Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
  • Now that we detached that process, we have to keep an eye on it.\nThis is a script that starts and maintains that tunnel which and is invoked from the daemon\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • You can find everything I talked about here today on Github. I’d encourage you to give it a try and let me know what you think. Pull requests are welcome.\nQuestions?\n
  • \n

CloudTunnel Atlanta Ruby Users Group October 2012 Presentation Transcript

  • 1. TUNNELING FOR FUN Jonathan Manuzak / @jonmanuzak #ATLRUG / Oct 10, 2012
  • 2. WHO ARE YOU?
  • 3. WHAT’S YOUR PROBLEM?
  • 4. MOST OF MY DEVELOPMENT HAPPENS HERE Photo: http://www.flickr.com/photos/blueace/797823265
  • 5. MOST OF MY DEVELOPMENT HAPPENS HERE
  • 6. FIREWALL / NAT
  • 7. FIREWALL / NAT
  • 8. FIREWALL / NAT •Permissive with outgoing requests •Restrictive with incoming requests •Not accessible if behind NAT
  • 9. OUTSIDE THE FIREWALL BUT THE RESTOF THE WORLD IS OVER HERE
  • 10. BUT, SURELY THIS IS A SOLVED PROBLEM. RIGHT?
  • 11. BUT, SURELY THIS IS A SOLVED PROBLEM. RIGHT? DEPENDS ON YOUR DEFINITION OF SOLVED
  • 12. BUT, SURELY THIS IS A SOLVED PROBLEM. RIGHT? DEPENDS ON YOUR DEFINITION OF SOLVED CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY,TEST, FAIL, CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY, TEST, FAIL, CODE, DEPLOY, TEST, FAIL, RINSE, REPEAT
  • 13. JUST USE A SSH TUNNEL!
  • 14. ANATOMY OF A SSH TUNNELhttp://localhost:8080
  • 15. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 16. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 S e r v e r Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 17. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 I n t e r n e t S e r v e r Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 18. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 I n Ports t 605 e 8080 r n e t Ports S e 80 r 22 v e r Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 19. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 I n Ports t 605 e 8080 r n e t Ports S e 80 r 22 v e r Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 20. ANATOMY OF A SSH TUNNEL U s e rhttp://localhost:8080 I n Ports t 605 e 8080 r n e t Ports S e 80 r 22 v e r Photos: http://eandt.theiet.org/magazine/2012/07/interview-andrew-blum.cfm, http://www.quickmeme.com/meme/35583x/
  • 21. SSH TUNNELS HAVE BAGGAGE YOU STILL NEED...•A server that’s somewhere on the internet• Running some kind of *nix• That you control
  • 22. I’M LAZY ssh -tR {remote_port}:127.0.0.1:{local_port} {user}@{remote_host} “sudo ssh -Nl $USER -L {host_alias}: {local_port}:127.0.0.1:{remote_port} {remote_host}” Source: https://gist.github.com/945690
  • 23. THERE MUST BE AN EASIER WAY
  • 24. ENTER LOCALTUNNEL
  • 25. NEAT! HOW DOES IT WORK? CLIENT SERVICE
  • 26. NEAT! HOW DOES IT WORK? CLIENT SERVICE
  • 27. NEAT! HOW DOES IT WORK? CLIENT SERVICE1. Makes HTTPPOST to localtunnel
  • 28. NEAT! HOW DOES IT WORK? CLIENT SERVICE1. Makes HTTPPOST to localtunnel 2. Responds with an open hostname and port
  • 29. NEAT! HOW DOES IT WORK? CLIENT SERVICE1. Makes HTTPPOST to localtunnel 2. Responds with an open hostname and port3. Opens a tunnelusing specified port
  • 30. NEAT! HOW DOES IT WORK? CLIENT SERVICE1. Makes HTTPPOST to localtunnel 2. Responds with an open hostname and port3. Opens a tunnelusing specified port 4. Proxies traffic from your assigned hostname to your tunnel
  • 31. TROUBLE IN PARADISE
  • 32. TROUBLE IN PARADISE• Random hostname, changes every time.
  • 33. TROUBLE IN PARADISE• Random hostname, changes every time.• Idle timeout
  • 34. TROUBLE IN PARADISE• Random hostname, changes every time.• Idle timeout• Change network connections?
  • 35. TROUBLE IN PARADISE• Random hostname, changes every time.• Idle timeout• Change network connections?• Put your laptop to sleep?
  • 36. TROUBLE IN PARADISE• Random hostname, changes every time.• Idle timeout• Change network connections?• Put your laptop to sleep?• Stop and start your server?
  • 37. WHAT I REALLY WANT IS... MACBOOK.MANUZAK.COM ALL. THE. TIME.
  • 38. ALTERNATIVESshowoff.io• Allows custom CNAMEs!• $5/mopagekite.net• Allows custom CNAMEs!• ~$4/mo
  • 39. CLOUDTUNNEL TO THE RESCUE! Photo: http://commons.wikimedia.org/wiki/File:Roll-cloud.JPG
  • 40. WHAT IS CLOUDTUNNEL?CLIENT SERVICE
  • 41. WHAT IS CLOUDTUNNEL? CLIENT SERVICE•Keeps the connectionalive•Restarts the connectionif needed•Runs localtunnel as adaemon
  • 42. WHAT IS CLOUDTUNNEL? CLIENT SERVICE•Keeps the connection •Allows you to specify youralive hostname•Restarts the connection •Supports CNAMES!if needed •Redirects users to your•Runs localtunnel as a current tunnel addressdaemon
  • 43. CLIENT➜ ps aux | grep cloud_tunnel19036 1.2 0.1 2509496 12164 7:22AM 0:03.37 cloud_tunnel19024 0.0 0.1 2490680 14504 7:22AM 0:00.29 cloud_tunnel_monitor19023 0.0 0.1 2501304 18332 7:22AM 0:00.42 cloud_tunnel
  • 44. CLIENT➜ ps aux | grep cloud_tunnel19036 1.2 0.1 2509496 12164 7:22AM 0:03.37 cloud_tunnel19024 0.0 0.1 2490680 14504 7:22AM 0:00.29 cloud_tunnel_monitor19023 0.0 0.1 2501304 18332 7:22AM 0:00.42 cloud_tunnel
  • 45. CLIENT➜ ps aux | grep cloud_tunnel19036 1.2 0.1 2509496 12164 7:22AM 0:03.37 cloud_tunnel19024 0.0 0.1 2490680 14504 7:22AM 0:00.29 cloud_tunnel_monitor19023 0.0 0.1 2501304 18332 7:22AM 0:00.42 cloud_tunnel
  • 46. CLIENT➜ ps aux | grep cloud_tunnel19036 1.2 0.1 2509496 12164 7:22AM 0:03.37 cloud_tunnel19024 0.0 0.1 2490680 14504 7:22AM 0:00.29 cloud_tunnel_monitor19023 0.0 0.1 2501304 18332 7:22AM 0:00.42 cloud_tunnel
  • 47. CLIENT - TUNNELmodule CloudTunnel  class Tunnel    TUNNEL_SERVICE_HOST_DOMAIN = localtunnel.com    attr_reader :host, :pid    def open_tunnel     start_tunnel_in_background    end    def connected?      return false unless @pid      ping_tunnel && process_alive?(@pid)    end    def keep_alive      ping_tunnel    end
  • 48. CLIENT - TUNNEL
  • 49. CLIENT - TUNNEL    def start_tunnel_in_background      key = File.open(KEY_FILE).read      t = LocalTunnel::Tunnel.new(PORT, key)      t.register_tunnel      pid = fork { t.start_tunnel }      @host = t.host      @pid = pid      Process.detach(@pid)    end
  • 50. CLIENT - TUNNEL    def start_tunnel_in_background      key = File.open(KEY_FILE).read      t = LocalTunnel::Tunnel.new(PORT, key)      t.register_tunnel      pid = fork { t.start_tunnel }      @host = t.host      @pid = pid      Process.detach(@pid)    end
  • 51. FORKING AND PIDS• PID = Process ID• Unique ID for each running process• Allows direct access for management• Why Process (fork)?• Process.detach == Thread.new { Process.wait(pid) }
  • 52. CLIENT - TUNNEL    def process_alive?(pid)      begin        Process.getpgid(pid)        true      rescue Errno::ESRCH        false      end    end    def ping_tunnel      url = "http://#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"      uri = URI(url)      response = Net::HTTP.get_response(uri)      case response      when Net::HTTPSuccess        true      else        false      end    end  endend
  • 53. CLIENT - TUNNEL    def process_alive?(pid)      begin        Process.getpgid(pid)        true      rescue Errno::ESRCH        false      end    end    def ping_tunnel      url = "http://#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"      uri = URI(url)      response = Net::HTTP.get_response(uri)      case response      when Net::HTTPSuccess        true      else        false      end    end  endend
  • 54. CLIENT - TUNNEL    def process_alive?(pid)      begin        Process.getpgid(pid)        true      rescue Errno::ESRCH        false      end    end    def ping_tunnel      url = "http://#{@host}.#{TUNNEL_SERVICE_HOST_DOMAIN}"      uri = URI(url)      response = Net::HTTP.get_response(uri)      case response      when Net::HTTPSuccess        true      else        false      end    end  endend
  • 55. CLIENT - DAEMONrequire cloud_tunnelSLEEP_INTERVAL = 180loop do  @tunnel ||=nil  start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?  sleep(SLEEP_INTERVAL)enddef start_tunnel_and_register_pid  # Clean up if process was running previously  kill_background_process(@pid) unless @pid.nil?  # Open the tunnel  @tunnel = CloudTunnel::start  @pid = @tunnel.pid  # Ensure the background processes get killed on exit  at_exit do    kill_background_process(@pid)  endend
  • 56. CLIENT - DAEMONrequire cloud_tunnelSLEEP_INTERVAL = 180loop do  @tunnel ||=nil  start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?  sleep(SLEEP_INTERVAL)enddef start_tunnel_and_register_pid  # Clean up if process was running previously  kill_background_process(@pid) unless @pid.nil?  # Open the tunnel  @tunnel = CloudTunnel::start  @pid = @tunnel.pid  # Ensure the background processes get killed on exit  at_exit do    kill_background_process(@pid)  endend
  • 57. CLIENT - DAEMONrequire cloud_tunnelSLEEP_INTERVAL = 180loop do  @tunnel ||=nil  start_tunnel_and_register_pid unless @tunnel && @tunnel.connected?  sleep(SLEEP_INTERVAL)enddef start_tunnel_and_register_pid  # Clean up if process was running previously  kill_background_process(@pid) unless @pid.nil?  # Open the tunnel  @tunnel = CloudTunnel::start  @pid = @tunnel.pid  # Ensure the background processes get killed on exit  at_exit do    kill_background_process(@pid)  endend
  • 58. CLIENT - REGISTRATIONmodule CloudTunnel  class RedirectClient    def create_or_update(source, destination)      uri = URI(http://ctr.herokuapp.com/route)      req = Net::HTTP::Post.new(uri.path)      req.set_form_data(:source => source, :destination => destination)      res = Net::HTTP.start(uri.hostname, uri.port) do |http|          http.request(req)      end      case res      when Net::HTTPSuccess        true      else        raise Could not update redirect server      end    end  endend
  • 59. SERVICE•Sinatra app•Running on Heroku•Dead simple API get / post /route delete /route
  • 60. SERVICE - ROUTErequire sinatrarequire dm-core’class Route  include DataMapper::Resource  property :id, Serial  property :source, String, :unique => true  property :destination, String  property :port, Integerend
  • 61. SERVICE - REDIRECTget / do  route = Route.last(:source => parse_hostname(request.host))  if route    redirect_path = "#{route.destination}" + (route.port.nil? ? : ":#{route.port}")    redirect redirect_path, 302  else    halt 404, Not found.  endend
  • 62. SERVICE - REDIRECTget / do  route = Route.last(:source => parse_hostname(request.host))  if route    redirect_path = "#{route.destination}" + (route.port.nil? ? : ":#{route.port}")    redirect redirect_path, 302  else    halt 404, Not found.  endend
  • 63. SERVICE - REDIRECTget / do  route = Route.last(:source => parse_hostname(request.host))  if route    redirect_path = "#{route.destination}" + (route.port.nil? ? : ":#{route.port}")    redirect redirect_path, 302  else    halt 404, Not found.  endend
  • 64. ¿QUESTIONS? https://github.com/jmanuzakJonathan Manuzak / @jonmanuzak #ATLRUG / Oct 10, 2012
  • 65. LINKS• https://github.com/jmanuzak/cloud_tunnel_redirect• https://github.com/jmanuzak/cloud_tunnel• https://github.com/progrium/localtunnel• http://progrium.com/localtunnel/• https://showoff.io/• https://pagekite.net/• http://daemons.rubyforge.org/