Search

Nanite (And An Introduction To Cloud Computing)

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    - Well welcome everybody to the first talk in a while for NWRUG! We’ve got more planned and I hope people will come along for the social meetups when there are no talks.
    - We have sponsorship this month from EngineYard, free pizzas and drinks, T-shirts. Full-disclosure, I work for engineyard.
    - No food or drinks in the auditorium, please leave the building in the same state you found it.

    - Talking about Nanite today, and cloud computing, it’s related (why)
    - If this looks familiar it’s because I had the same problems George Palmer had when doing his talk at RubyManor
    - transition

    - But that’s OK, Vertebra is getting there and Nanite is cool

    - So, Why not Vertebra?
    - wasn’t working in time
    - however, seen on EY nodes, actively developed
    - Might do a talk on it at some point
    - unless someone else gets there first (hint)
    - On to the first part of the talk, cloud-computing

    - so what is it?
    - Ask for volunteer descriptions of cloud-computing.

    - I came up with this description myself
    - Seems to be no solid definition
    - Some use the term to refer to the internet itself, but that already has a name
    - Could even apply to the use of APIs

    - smooth transition

    - old-school provisioning, had to provision for maximum resource requirement (users in the case of web-apps)
    - during slack periods unused capacity wastes money, idle servers
    - cloud-computing for me means only dipping into my beer-fund when necessary

    - OK, this should be fairly obvious by now
    - go through points
    -- transition
    - At EY we see both expected (time-of-day) and unexpected bumps

    - Best example I have seen
    - mongrel queue hit ~1000
    - Fixed by adding 2xCPUs per VM and more memory. Client added some cacheing.

    - One more just for fun (can’t resist pretty graphs).
    - Can’t remember the exact circumstances, but it’s pretty :)

    - So we have an idea what it is, and why you would do it, but how?
    - Doing it right, there’s no hard-and-fast point at which you are ‘cloud-computing’
    - At what point does computing in the cloud stop being cloud-computing when provisioning takes too long?
    - Could be dependent on the situation, ‘in time’
    - Discovery. You don’t have to do this, you can configure everything yourself. Slows you down.
    - Talk about discovery.

    - Just for fun, what’s the earliest use of cloud-computing anyone can think of?
    - wait for answers then transition
    - Good discovery, easy provisioning (thanks Microsoft!)

    - conclusion, iterate over points

    - conclusion, iterate over points

    - conclusion, iterate over points

    - conclusion, iterate over points

    - conclusion, iterate over points

    - conclusion, iterate over points

    - So what does this have to do with Nanite?
    - Background processing is getting more important, web-apps are doing more, more processing power needed
    - Website users get bored easily, they go somewhere else.
    - Image transformations, Amazon S3 uploads, Email sending etc. all takes time.
    - We move the heavy lifting out of the front-end process, freeing it up.
    - Want to remain scalable

    - So, on to Nanite!
    - project started by Ezra, Open source, on github, under active development

    - So, on to Nanite!
    - project started by Ezra, Open source, on github, under active development

    So what is it?
    - As described in the README
    - A bit wordy, but a pretty good description
    It really is:
    - Nanite is a background processing system, like Dj, Bj or BackgroundRb but on crack (explain)
    - Allows you to shunt off work synchronously or asynchronously to back-end runners
    - Written in Ruby, basically an abstraction of AMQP

    - Scaleable, multiple rabbitMQ back-ends can be run
    - Jobs can be put on the queue for running now or later
    - Discovery. Agents advertise mappers, mappers can get a list of available agents and actions
    - load-based dispatch, discuss

    - Though any AMQP compliant queue should do

    - RabbitMQ is the queue, it talks AMQP
    - An Agent is a single process, it advertises multiple actors
    - mapper is just a fancy name for the client that sends work and maybe received results
    - Agents report there status by default every 15 seconds, mappers track the state of all nanites removing those that have not checked in within a timeout.
    - Mappers send data to agent chosen using a fitness function. Default is least loaded (using uptime), but others are available as i’ll show you.

    - Or at least we can get a lot of power by using an abstraction
    - so we use nanite

    - Or at least we can get a lot of power by using an abstraction
    - so we use nanite

    - Really easy on debian
    - just follow the instructions on the nanite github page

    - Sets up the queues and users in rabbitMQ
    - Only needs to be done once.
    - To process any jobs you then need an agent (transition)

    - We will run nanite-actor in this directory
    discuss

    - Simple, just stick in the actors directory
    - by default all methods exposed, can explicitly expose with ‘expose’ method

    - The agent + actors do the work

    - From the command line
    - I am starting these using a bask wrapper from monit, works well.

    - Mappers are the clients for agents/actors
    - Just code

    - this is where the mappers fit into the architecture

    - Let’s attempt to summon the god of FAIL by doing a live-action demo


    I am going to put up a blog post probably tomorrow with all the relevant links available.


    8 Favorites

    Nanite (And An Introduction To Cloud Computing) - Presentation Transcript

    1. NWRUG February 2009 Sponsored by Engine Yard Friday, February 20, 2009
    2. Vertebra Nanite (and an introduction to cloud computing) Will Jessop Friday, February 20, 2009
    3. Friday, February 20, 2009
    4. Vertebra? • Wasn’t working in time to write this… • …seen in use on Engine Yard nodes • Actively developed, improving Friday, February 20, 2009
    5. (an introduction to cloud computing) What is it? (Discuss) Friday, February 20, 2009
    6. My definition Friday, February 20, 2009
    7. My definition Cloud computing means taking the resources you need for a computing task at the time you need it from a pool of available processing capacity. Friday, February 20, 2009
    8. What does that really mean? • Provision the minimum amount of processing capacity you need • Add more processing capacity as needed Friday, February 20, 2009
    9. Friday, February 20, 2009
    10. Why compute in the cloud? • Servers cost money to run, even when idle • Demand for capacity can peak Friday, February 20, 2009
    11. Why compute in the cloud? • Servers cost money to run, even when idle • Demand for capacity can peak Oh yes, it can: Friday, February 20, 2009
    12. Uh-oh: Friday, February 20, 2009
    13. Yikes! Friday, February 20, 2009
    14. Doing it right • Monitoring • Provisioning • Discovery Friday, February 20, 2009
    15. Doing it right • Monitoring • Provisioning • Discovery Friday, February 20, 2009
    16. Friday, February 20, 2009
    17. Botnets of infected windows machines running eggdrop (or similar), an IRC server and a 1337 h4x0r Friday, February 20, 2009
    18. (an introduction to cloud computing) Friday, February 20, 2009
    19. (an introduction to cloud computing) • Computing on demand Friday, February 20, 2009
    20. (an introduction to cloud computing) • Computing on demand • Only pay for what you use Friday, February 20, 2009
    21. (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring Friday, February 20, 2009
    22. (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning Friday, February 20, 2009
    23. (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning • Discovery Friday, February 20, 2009
    24. (an introduction to cloud computing) • Computing on demand • Only pay for what you use • Monitoring • Provisioning • Discovery • Questions? Friday, February 20, 2009
    25. The Future is in the Background • Web serving processes need a fast turnaround, users won’t wait • Don’t want to tie-up processes with slow stuff Friday, February 20, 2009
    26. Friday, February 20, 2009
    27. Nanite Friday, February 20, 2009
    28. Nanite Ezra Zygmuntowicz Friday, February 20, 2009
    29. Nanite is: “Nanite is a new way of thinking about building cloud ready web applications. Having a scalable message queueing back-end with all the discovery and dynamic load based dispatch that Nanite has is a very scalable way to construct web application back-ends.” Friday, February 20, 2009
    30. The interesting bits • Scaleable back-end • Message queueing • Discovery • Load based dispatch (by default) Friday, February 20, 2009
    31. Technical stuff • RabbitMQ + AMQP • Ruby • Can send data JSON, Marshalled or YAML Friday, February 20, 2009
    32. Architecture Friday, February 20, 2009
    33. AMQP We could talk AMQP directly but… Friday, February 20, 2009
    34. AMQP Friday, February 20, 2009
    35. Installation • RabbitMQ • Erlang • RabbitMQ • Nanite • Ruby (Duh) • AMQP & ezmobius-nanite Friday, February 20, 2009
    36. Setup • git clone git://github.com/ezmobius/ nanite.git • cd nanite • sudo ./examples/rabbitconf.rb Friday, February 20, 2009
    37. Agent structure Friday, February 20, 2009
    38. Actors simple.rb Friday, February 20, 2009
    39. Agent + Actors Friday, February 20, 2009
    40. Starting up an agent $ cd /path/to/agent_directory $ /var/lib/gems/1.8/bin/nanite-agent -t fred -f json -ldebug\" Friday, February 20, 2009
    41. Mappers Friday, February 20, 2009
    42. Mappers Friday, February 20, 2009
    43. Let’s try it out Friday, February 20, 2009
    44. FIN Friday, February 20, 2009
    45. Resources • http://willj.net/… Friday, February 20, 2009
    46. Next Month • Social meetup, want to talk? • Thursday16th April: Ashley Moran - From Specification to Success, a talk on BDD Friday, February 20, 2009
    SlideShare Zeitgeist 2009

    + will_jwill_j Nominate

    custom

    1871 views, 8 favs, 0 embeds more stats

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1871
      • 1871 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 8
    • Downloads 58
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    -->
    Search
    RSS Feed
    What's new?

    © 2009 SlideShare Inc. All Rights Reserved