• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Nanite (And An Introduction To Cloud Computing)
 

Nanite (And An Introduction To Cloud Computing)

on

  • 5,318 views

 

Statistics

Views

Total Views
5,318
Views on SlideShare
5,299
Embed Views
19

Actions

Likes
10
Downloads
141
Comments
0

2 Embeds 19

http://www.slideshare.net 15
http://links.jetztgrad.net 4

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • - Well welcome everybody to the first talk in a while for NWRUG! We’ve got more planned and I hope people will come along for the social meetups when there are no talks. <br /> - We have sponsorship this month from EngineYard, free pizzas and drinks, T-shirts. Full-disclosure, I work for engineyard. <br /> - No food or drinks in the auditorium, please leave the building in the same state you found it. <br />
  • - Talking about Nanite today, and cloud computing, it’s related (why) <br /> - If this looks familiar it’s because I had the same problems George Palmer had when doing his talk at RubyManor <br /> - transition <br />
  • - But that’s OK, Vertebra is getting there and Nanite is cool <br />
  • - So, Why not Vertebra? <br /> - wasn’t working in time <br /> - however, seen on EY nodes, actively developed <br /> - Might do a talk on it at some point <br /> - unless someone else gets there first (hint) <br /> - On to the first part of the talk, cloud-computing <br />
  • - so what is it? <br /> - Ask for volunteer descriptions of cloud-computing. <br />
  • - I came up with this description myself <br /> - Seems to be no solid definition <br /> - Some use the term to refer to the internet itself, but that already has a name <br /> - Could even apply to the use of APIs <br />
  • - smooth transition <br />
  • - old-school provisioning, had to provision for maximum resource requirement (users in the case of web-apps) <br /> - during slack periods unused capacity wastes money, idle servers <br /> - cloud-computing for me means only dipping into my beer-fund when necessary <br />
  • - OK, this should be fairly obvious by now <br /> - go through points <br /> -- transition <br /> - At EY we see both expected (time-of-day) and unexpected bumps <br />
  • - Best example I have seen <br /> - mongrel queue hit ~1000 <br /> - Fixed by adding 2xCPUs per VM and more memory. Client added some cacheing. <br />
  • - One more just for fun (can’t resist pretty graphs). <br /> - Can’t remember the exact circumstances, but it’s pretty :) <br />
  • - So we have an idea what it is, and why you would do it, but how? <br /> - Doing it right, there’s no hard-and-fast point at which you are ‘cloud-computing’ <br /> - At what point does computing in the cloud stop being cloud-computing when provisioning takes too long? <br /> - Could be dependent on the situation, ‘in time’ <br /> - Discovery. You don’t have to do this, you can configure everything yourself. Slows you down. <br /> - Talk about discovery. <br />
  • - Just for fun, what’s the earliest use of cloud-computing anyone can think of? <br /> - wait for answers then transition <br /> - Good discovery, easy provisioning (thanks Microsoft!) <br />
  • - conclusion, iterate over points <br /> <br />
  • - conclusion, iterate over points <br /> <br />
  • - conclusion, iterate over points <br /> <br />
  • - conclusion, iterate over points <br /> <br />
  • - conclusion, iterate over points <br /> <br />
  • - conclusion, iterate over points <br /> <br />
  • - So what does this have to do with Nanite? <br /> - Background processing is getting more important, web-apps are doing more, more processing power needed <br /> - Website users get bored easily, they go somewhere else. <br /> - Image transformations, Amazon S3 uploads, Email sending etc. all takes time. <br /> - We move the heavy lifting out of the front-end process, freeing it up. <br /> - Want to remain scalable <br />
  • - So, on to Nanite! <br /> - project started by Ezra, Open source, on github, under active development <br />
  • - So, on to Nanite! <br /> - project started by Ezra, Open source, on github, under active development <br />
  • So what is it? <br /> - As described in the README <br /> - A bit wordy, but a pretty good description <br /> It really is: <br /> - Nanite is a background processing system, like Dj, Bj or BackgroundRb but on crack (explain) <br /> - Allows you to shunt off work synchronously or asynchronously to back-end runners <br /> - Written in Ruby, basically an abstraction of AMQP <br />
  • - Scaleable, multiple rabbitMQ back-ends can be run <br /> - Jobs can be put on the queue for running now or later <br /> - Discovery. Agents advertise mappers, mappers can get a list of available agents and actions <br /> - load-based dispatch, discuss <br />
  • - Though any AMQP compliant queue should do <br />
  • - RabbitMQ is the queue, it talks AMQP <br /> - An Agent is a single process, it advertises multiple actors <br /> - mapper is just a fancy name for the client that sends work and maybe received results <br /> - Agents report there status by default every 15 seconds, mappers track the state of all nanites removing those that have not checked in within a timeout. <br /> - Mappers send data to agent chosen using a fitness function. Default is least loaded (using uptime), but others are available as i’ll show you. <br />
  • - Or at least we can get a lot of power by using an abstraction <br /> - so we use nanite <br />
  • - Or at least we can get a lot of power by using an abstraction <br /> - so we use nanite <br />
  • - Really easy on debian <br /> - just follow the instructions on the nanite github page <br />
  • - Sets up the queues and users in rabbitMQ <br /> - Only needs to be done once. <br /> - To process any jobs you then need an agent (transition) <br />
  • - We will run nanite-actor in this directory <br /> discuss <br />
  • - Simple, just stick in the actors directory <br /> - by default all methods exposed, can explicitly expose with ‘expose’ method <br />
  • - The agent + actors do the work <br />
  • - From the command line <br /> - I am starting these using a bask wrapper from monit, works well. <br />
  • - Mappers are the clients for agents/actors <br /> - Just code <br />
  • - this is where the mappers fit into the architecture <br />
  • - Let’s attempt to summon the god of FAIL by doing a live-action demo <br />
  • <br />
  • I am going to put up a blog post probably tomorrow with all the relevant links available. <br />
  • <br />

Nanite (And An Introduction To Cloud Computing) Nanite (And An Introduction To Cloud Computing) Presentation Transcript

  • NWRUG February 2009 Sponsored by Engine Yard Friday, February 20, 2009
  • Vertebra Nanite (and an introduction to cloud computing) Will Jessop Friday, February 20, 2009
  • Friday, February 20, 2009
  • Vertebra? • Wasn’t working in time to write this… • …seen in use on Engine Yard nodes • Actively developed, improving Friday, February 20, 2009
  • (an introduction to cloud computing) What is it? (Discuss) Friday, February 20, 2009
  • My definition Friday, February 20, 2009
  • My definition Cloud computing means taking the resources you need for a computing task at the time you need it from a pool of available processing capacity. Friday, February 20, 2009
  • What does that really mean? • Provision the minimum amount of processing capacity you need • Add more processing capacity as needed Friday, February 20, 2009
  • Friday, February 20, 2009
  • Why compute in the cloud? • Servers cost money to run, even when idle • Demand for capacity can peak Friday, February 20, 2009
  • Why compute in the cloud? • Servers cost money to run, even when idle • Demand for capacity can peak Oh yes, it can: Friday, February 20, 2009
  • Uh-oh: Friday, February 20, 2009
  • Yikes! Friday, February 20, 2009
  • Doing it right • Monitoring • Provisioning • Discovery Friday, February 20, 2009
  • Doing it right • Monitoring • Provisioning • Discovery Friday, February 20, 2009
  • Friday, February 20, 2009
  • Botnets of infected windows machines running eggdrop (or similar), an IRC server and a 1337 h4x0r Friday, February 20, 2009
  • (an introduction to cloud computing) Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand • Only pay for what you use Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning • Discovery Friday, February 20, 2009
  • (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning • Discovery • Questions? Friday, February 20, 2009
  • The Future is in the Background • Web serving processes need a fast turnaround, users won’t wait • Don’t want to tie-up processes with slow stuff Friday, February 20, 2009
  • Friday, February 20, 2009
  • Nanite Friday, February 20, 2009
  • Nanite Ezra Zygmuntowicz Friday, February 20, 2009
  • Nanite is: “Nanite is a new way of thinking about building cloud ready web applications. Having a scalable message queueing back-end with all the discovery and dynamic load based dispatch that Nanite has is a very scalable way to construct web application back-ends.” Friday, February 20, 2009
  • The interesting bits • Scaleable back-end • Message queueing • Discovery • Load based dispatch (by default) Friday, February 20, 2009
  • Technical stuff • RabbitMQ + AMQP • Ruby • Can send data JSON, Marshalled or YAML Friday, February 20, 2009
  • Architecture Friday, February 20, 2009
  • AMQP We could talk AMQP directly but… Friday, February 20, 2009
  • AMQP Friday, February 20, 2009
  • Installation • RabbitMQ • Erlang • RabbitMQ • Nanite • Ruby (Duh) • AMQP & ezmobius-nanite Friday, February 20, 2009
  • Setup • git clone git://github.com/ezmobius/ nanite.git • cd nanite • sudo ./examples/rabbitconf.rb Friday, February 20, 2009
  • Agent structure Friday, February 20, 2009
  • Actors simple.rb Friday, February 20, 2009
  • Agent + Actors Friday, February 20, 2009
  • Starting up an agent $ cd /path/to/agent_directory $ /var/lib/gems/1.8/bin/nanite-agent -t fred -f json -ldebugquot; Friday, February 20, 2009
  • Mappers Friday, February 20, 2009
  • Mappers Friday, February 20, 2009
  • Let’s try it out Friday, February 20, 2009
  • FIN Friday, February 20, 2009
  • Resources • http://willj.net/… Friday, February 20, 2009
  • Next Month • Social meetup, want to talk? • Thursday16th April: Ashley Moran - From Specification to Success, a talk on BDD Friday, February 20, 2009