Other distributed systems


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This allows multiple machines named “CS” in the world For unfortunate reasons, the most significant label is to the right, instead of the left.
  • Pitfalls of this design: -- requires literally billions or more queries to root servers in a day – far too much stress -- requires every individual computer to make lots of requests to many different machines
  • Alright, maybe Senator Stevens isn’t entirely correct, but he’s right about one thing: the internet is composed of some sort of infrastructure.
  • This is a composited satellite image of the earth at night. This is meant to demonstrate that there are a lot of spaces which lack infrastructure even for electricity. A real map of internet infrastructure would probably show even greater infrastructural disparity between the wealthier developed nations and the developing nations.
  • Cheap: Fiber costs nearly $50k per mile in rural areas, Cisco routers with 80 ports are around $7k, desktop computers are $300-$400, and the cost of powering this equipment reliably is tremendously expensive in a developing nation
  • Founded by Nicholas Negroponte while he was working at MediaLab. Took the project outside of the lab to form a new non-profit NGO called OLPC. Backed by many industry leaders, including: Quanta (manufacturer), eBay, AMD, Google, Red Hat. Mission: provide the laptop at a low cost (in terms of both hardware and operation) to facilitate its purchase by governments of developing nations; provide rich, open-source software tools to allow teachers and children to create, develop and discover knowledge; provide high-bandwidth connectivity to enable the development of knowledge communities. OLPC: If you give each child access simultaneous access to the Internet (or even a local copy of Wikipedia stored at some village server) you can provide them with more knowledge than the biggest library in the world, and at a lower cost. Improving the educational experience in drastic ways to create long-term effects towards providing fair, equitable, and economically and socially viable societies.
  • Of course, all of this is good for us (and the rest of the developed nations). Simple math can tell you that an expansion of the market (potentially 5 fold!) means more commerce here too!
  • Certainly, children are the future of any nation. If we can elevate the level of education, we can create more opportunities for them to compete and grab a share of global markets, and standard of living will rise. India and China are already bringing themselves out of poverty, largely because of the commerce they have with developed nations.
  • Other distributed systems

    1. 1. Lecture 5 - Other Distributed Systems CSE 490h – Introduction to Distributed Computing, Spring 2007 Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License.
    2. 2. Outline <ul><li>DNS </li></ul><ul><li>BOINC </li></ul><ul><li>PlanetLab </li></ul><ul><li>OLPC & Ad-hoc Mesh Networks </li></ul><ul><li>Lecture content wrap-up </li></ul>
    3. 3. DNS: The Distributed System in the Distributed System
    4. 4. Domain Name System <ul><li>Mnemonic identifiers work considerably better for humans than IP addresses </li></ul><ul><li>“ www.google.com? Surely you mean!” </li></ul><ul><li>Who maintains the mappings from name  IP? </li></ul>
    5. 5. A Manageable Problem © 2006 Computer History Museum. All rights reserved. www.computerhistory.org
    6. 6. In the beginning… <ul><li>Every machine had a file named hosts.txt </li></ul><ul><li>Each line contained a name/IP mapping </li></ul><ul><li>New hosts files were updated and distributed via email </li></ul><ul><li>… This clearly wasn’t going to scale </li></ul>
    7. 7. DNS Implementations <ul><li>Modern DNS system first proposed in 1983 </li></ul><ul><li>First implementation in 1984 (Paul Mockapetris) </li></ul><ul><li>BIND (Berkeley Internet Name Domain) written by four Berkeley students in 1985. </li></ul><ul><li>Many other implementations today </li></ul>
    8. 8. Hierarchical Naming <ul><li>DNS names are arranged in a hierarchy: </li></ul><ul><li>www.cs.washington.edu </li></ul><ul><li>Entries are either subdomains or hostnames </li></ul><ul><li>subdomains contain more subdomains, or hosts (up to 127 levels deep!) </li></ul><ul><li>Hosts have individual IP addresses </li></ul>
    9. 9. Mechanics: Theory <ul><li>DNS Recurser (client) parses address from right to left </li></ul><ul><li>Asks root server (with known, static IP address) for name of first subdomain DNS server </li></ul><ul><li>Contacts successive DNS servers until it finds the host </li></ul>
    10. 10. Mechanics: In Practice <ul><li>ISPs provide a DNS recurser for clients </li></ul><ul><li>DNS recursers cache lookups for period of time after a request </li></ul><ul><li>Greatly speeds up retrieval of entries and reduces system load </li></ul>
    11. 11. BOINC
    12. 12. What is BOINC? <ul><li>“ Berkeley Open Infrastructure for Network Computing” </li></ul><ul><li>Platform for Internet-wide distributed applications </li></ul><ul><li>Volunteer computing infrastructure </li></ul><ul><ul><li>Relies on many far-flung users volunteering spare CPU power </li></ul></ul>
    13. 13. Some Facts <ul><li>1,000,000+ active nodes </li></ul><ul><li>521 TFLOPS of computing power </li></ul><ul><li>20 active projects (SETI@Home, Folding@Home, Malaria Control…) and several more in development </li></ul><ul><li>(Current as of March 2007) </li></ul>
    14. 14. Comparison to MapReduce <ul><li>Both are frameworks on which “useful” systems can be built </li></ul><ul><li>Does not prescribe particular programming style </li></ul><ul><li>Much more heterogeneous architecture </li></ul><ul><li>Does not have a formal aggregation step </li></ul><ul><li>Designed for much longer-running systems (months/years vs. minutes/hours) </li></ul>
    15. 15. Architecture <ul><li>Central server runs LAMP architecture for web + database </li></ul><ul><li>End-users run client application with modules for actual computation </li></ul><ul><li>BitTorrent used to distribute data elements efficiently </li></ul>
    16. 16. System Features <ul><li>Homogenous redundancy </li></ul><ul><li>Work unit “trickling” </li></ul><ul><li>Locality scheduling </li></ul><ul><li>Distribution based on host parameters </li></ul>
    17. 17. Client software <ul><li>Available as regular application, background “service”, or screensaver </li></ul><ul><li>Can be administered locally or LAN-administered via RPC </li></ul><ul><li>Can be configured to use only “low priority” cycles </li></ul>
    18. 18. Client/Task Interaction <ul><li>Client software runs on variety of operating systems, each with different IPC </li></ul><ul><li>Uses shared memory message passing to transmit information from “manager” to actual tasks and vice versa </li></ul>
    19. 19. Why Participate? <ul><li>Sense of accomplishment, community involvement, or scientific duty </li></ul><ul><li>Stress testing machines/networks </li></ul><ul><li>Potential for fame (if your computer “finds” an alien planet, you can name it!) </li></ul><ul><li>“ Bragging rights” for computing more units </li></ul><ul><ul><li>“ BOINC Credits” </li></ul></ul>
    20. 20. Credit & Cobblestones <ul><li>Work done is rewarded with “cobblestones” </li></ul><ul><li>100 cobblestones = 1 day of CPU time for a computer with performance equaling 1,000 double-precision floating-point MIPS (Whetstone) & 1,000 integer VAX MIPS (Dhrystone) </li></ul><ul><li>Computers are benchmarked by the BOINC system and receive credit appropriate to their machine </li></ul>
    21. 21. Anti-Cheating Measures <ul><li>Work units are computed redundantly by several different machines, and results are compared by the central server for consistency </li></ul><ul><li>Credit is awarded after the internal server validates the returned work units </li></ul><ul><li>Work units must be returned before a deadline </li></ul>
    22. 22. Conclusions <ul><li>Versatile infrastructure </li></ul><ul><ul><li>SETI tasks take a few hours </li></ul></ul><ul><ul><li>Climate simulation tasks take months </li></ul></ul><ul><ul><li>Network monitoring tasks are not CPU-bound at all! </li></ul></ul><ul><li>Scales extremely well to internet-wide applications </li></ul><ul><li>Provides another flexible middleware layer to base distributed applications on </li></ul><ul><li>Volunteer computing comes with add’l considerations (rewards, cheating) </li></ul>
    23. 23. PlanetLab
    24. 24. What if you wanted to: <ul><li>Test a new version of Bittorrent that might generate GB’s and GB’s of data? </li></ul><ul><li>Design a new distributed hashtable algorithm for thousands of nodes? </li></ul><ul><li>Create a gigantic caching structure that mirrored web pages in several sites across the USA? </li></ul>
    25. 25. Problem Similarities <ul><li>Each of these problems requires: </li></ul><ul><ul><li>Hundreds or thousands of servers </li></ul></ul><ul><ul><li>Geographic distribution </li></ul></ul><ul><ul><li>An isolated network for testing and controlled experiments </li></ul></ul><ul><li>Developing one-off systems to support these would be </li></ul><ul><ul><li>Costly </li></ul></ul><ul><ul><li>Redundant </li></ul></ul>
    26. 26. PlanetLab <ul><li>A multi-university effort to build a network for large-scale simulation, testing, and research </li></ul><ul><li>“ Simulate the Internet” </li></ul>
    27. 27. Usage Stats <ul><li>Servers: 722+ </li></ul><ul><li>Slices: 600+ </li></ul><ul><li>Users: 2500+ </li></ul><ul><li>Bytes-per-day: 3 - 4 TB </li></ul><ul><li>IP-flows-per-day: 190M </li></ul><ul><li>Unique IP-addrs-per-day: 1M </li></ul>As of Fall, 2006
    28. 28. Project Goals <ul><li>Supports short- and long-term research goals </li></ul><ul><li>System put up “as fast as possible” – PlanetLab design evolves over time to meet changing needs </li></ul><ul><ul><li>PlanetLab is a process , not a result </li></ul></ul>
    29. 29. Simultaneous Research <ul><li>Projects must be isolated from one another </li></ul><ul><li>Code from several researchers: </li></ul><ul><ul><li>Untrustworthy? Possibly buggy? </li></ul></ul><ul><ul><li>Intellectual property issues? </li></ul></ul><ul><li>Time-sensitive experiments must not interfere with one another </li></ul><ul><li>Must provide realistic workload simulations </li></ul>
    30. 30. Architecture <ul><li>Built on Linux, ssh, other standard tools </li></ul><ul><ul><li>Provides “normal” environment for application development </li></ul></ul><ul><li>Hosted at multiple universities w/ separate admins </li></ul><ul><ul><li>Requires trust relationships with respect to previous goals </li></ul></ul>
    31. 31. Architecture (cont.) <ul><li>Network is divided into “slices” – server pools created out of virtual machines </li></ul><ul><li>Trusted intermediary “PLC” system grants access to network resources </li></ul><ul><ul><li>Allows universities to specify who can use slices at each site </li></ul></ul><ul><ul><li>Distributed trust relationships </li></ul></ul><ul><ul><li>Central system control  Federated control </li></ul></ul>
    32. 32. Resource allocation <ul><li>PLC authenticates users and understands relationships between principals; issues tickets </li></ul><ul><li>SHARP system at site validates ticket + returns lease </li></ul>
    33. 33. User Verification <ul><li>Public-key cryptography used to sign modules entered into PlanetLab </li></ul><ul><ul><li>X.509 + SSL keys are used by PLC + slices to verify user authenticity </li></ul></ul><ul><ul><li>Keys distributed “out of band” ahead of time </li></ul></ul>
    34. 34. Final Thoughts <ul><li>Large system with complex relationships </li></ul><ul><li>Currently upgrading to version 4.0 </li></ul><ul><li>New systems (GENI) are being proposed </li></ul><ul><li>Still provides lots of resources to researchers </li></ul><ul><ul><li>CoralCache, several other projects run on PlanetLab </li></ul></ul>
    35. 35. OLPC “ They want to deliver vast amounts of information over the Internet. And again, the Internet is not something you just dump something on. It's not a big truck. It's a series of tubes .”
    36. 36. The Internet is a series of tubes <ul><li>The internet is composed of a lot of infrastructure: </li></ul><ul><ul><li>Clients and servers </li></ul></ul><ul><ul><li>Routers and switches </li></ul></ul><ul><ul><li>Fiber optic trunk lines, telephone lines, tubes and trucks </li></ul></ul><ul><li>And if we map the density of this infrastructure… </li></ul>
    37. 37. … it probably looks something like this Photo: cmu.edu
    38. 38. How do we distribute knowledge when there are no tubes? <ul><li>What if we wanted to share a book? </li></ul><ul><ul><li>Pass it along, door-to-door. </li></ul></ul><ul><li>What if we wanted to share 10,000 books? </li></ul><ul><ul><li>Build community library. </li></ul></ul><ul><li>How about 10 million books? Or 300 copies of one book? </li></ul><ul><ul><li>A very large library? </li></ul></ul>
    39. 39. Solutions <ul><li>We need to build infrastructure to make large-scale distribution easy (i.e., computers and networking equipment) </li></ul><ul><li>We need to be cheap </li></ul><ul><ul><li>Most of those dark spots don’t have much money </li></ul></ul><ul><li>We need reliability where reliable power is costly </li></ul><ul><ul><li>Again, did you notice that there weren’t so many lights? It’s because there’s no electricity! </li></ul></ul>
    40. 40. The traditional solution: a shared computer with Internet <ul><li>India </li></ul><ul><ul><li>75% of people in rural villages </li></ul></ul><ul><ul><li>90% of phones in urban areas </li></ul></ul><ul><li>Many villagers share a single phone, usually located in the town post office </li></ul><ul><li>Likewise, villages typically share a few computers, located at the school (or somewhere with reliable power) </li></ul><ul><li>What’s the downside to this model? </li></ul><ul><ul><li>It might provide shared access to a lot of information, but it doesn’t solve the “300 copies of a book” case </li></ul></ul>
    41. 41. The distributed solution: the XO <ul><li>AKA: Children’s Machine, OLPC, $100 laptop </li></ul><ul><li>A cheap (~$150) laptop designed for children in developing countries </li></ul><ul><li>OLPC = One Laptop Per Child. </li></ul>Photo: laptop.org
    42. 42. XO design <ul><li>Low power consumption </li></ul><ul><ul><li>No moving parts (flash memory, passive cooling) </li></ul></ul><ul><ul><li>Dual-mode display </li></ul></ul><ul><ul><ul><li>In color, the XO consumes 2-3 watts </li></ul></ul></ul><ul><ul><ul><li>In high-contrast monochrome, less than 1 watt </li></ul></ul></ul><ul><ul><li>Can be human powered by a foot-pedal </li></ul></ul><ul><li>Rugged, child-friendly design </li></ul><ul><li>Low material costs </li></ul><ul><li>Open-source software </li></ul>
    43. 43. XO networking <ul><li>The XO utilizes far-reaching, low-power wireless networking to create ad-hoc mesh networks </li></ul><ul><ul><li>If any single XO is connected to the Internet, other nearby computers can share the connection in a peer-to-peer scheme </li></ul></ul><ul><li>Networks can theoretically sprawl as far as ten miles, even connecting nearby villages </li></ul>
    44. 44. XO storage and sharing <ul><li>XO relies on network for content and collaboration </li></ul><ul><ul><li>Content is stored on a central servers </li></ul></ul><ul><ul><ul><li>Textbooks </li></ul></ul></ul><ul><ul><ul><li>Cached websites (Wikipedia) </li></ul></ul></ul><ul><ul><ul><li>User content </li></ul></ul></ul><ul><ul><li>Software makes it easy to see other users on the network and share content </li></ul></ul>
    45. 45. XO distribution <ul><li>XO must be purchased in orders of 1 million units by governments in developing nations (economies of scale help to lower costs) </li></ul><ul><li>Governments are responsible for distribution of laptops </li></ul><ul><li>Laptops are only for children, designed solely as a tool for learning </li></ul>
    46. 46. XO downfalls <ul><li>Distribution downfalls </li></ul><ul><ul><li>What about children in developed nations? </li></ul></ul><ul><ul><ul><li>Sell to developed markets at a higher price to subsidize costs for developing nations. </li></ul></ul></ul><ul><ul><li>Can governments effectively distribute? What about black markets? </li></ul></ul><ul><ul><ul><li>OLPC could perhaps partner with local schools and other NGOs to aid in distribution, training and maintenance </li></ul></ul></ul><ul><li>Too expensive? </li></ul><ul><ul><li>Some nations can only afford as much $20 per child per year. How can we cater to them? </li></ul></ul>
    47. 47. What can the XO achieve? <ul><li>Today, only 16 percent of the world’s population is estimated to have access to the Internet </li></ul><ul><li>Develop new markets </li></ul><ul><ul><li>Microcredit </li></ul></ul><ul><ul><ul><li>Make small loans to the impoverished without requiring collateral </li></ul></ul></ul><ul><ul><ul><li>Muhammad Yunus and the Grameen Bank won the 2006 Nobel Peace Prize for their work here </li></ul></ul></ul><ul><ul><li>The power of the village economy </li></ul></ul><ul><ul><ul><li>As millions of users come online in developing nations, there will be many new opportunities for commerce. </li></ul></ul></ul><ul><ul><ul><li>Helps those in developing nations to advance their economies and develop stronger economic models </li></ul></ul></ul>
    48. 48. Why give the XO to children? <ul><li>UN Millennium Development Goal #2: “achieve universal primary education” </li></ul><ul><li>Empower children to think and compete in a global space </li></ul><ul><ul><li>Children are a nations greatest resource </li></ul></ul><ul><ul><li>Backed by a bolstered economy, they will grow to solve other issues (infrastructure, poverty, famine) </li></ul></ul>
    49. 49. The Course Again (in 5 minutes) <ul><li>So what did we see in this class? </li></ul><ul><ul><li>Moore’s law is starting to fail </li></ul></ul><ul><ul><li>More computing power means more machines </li></ul></ul><ul><ul><li>This means breaking problems into sub problems </li></ul></ul><ul><ul><ul><li>Sub-problems cannot interfere with or depend on one another </li></ul></ul></ul><ul><ul><ul><li>Have to “play nice” with shared memory </li></ul></ul></ul>
    50. 50. MapReduce <ul><li>MapReduce is one paradigm for breaking problems up </li></ul><ul><ul><li>Makes the “playing nice” easy by enforcing a decoupled programming model </li></ul></ul><ul><ul><li>Handles lots of the behind-the-scenes work </li></ul></ul>
    51. 51. Distributed Systems & Networks <ul><li>The network is a fundamental part of a distributed system </li></ul><ul><ul><li>Have to plan for bandwidth, latency, etc </li></ul></ul><ul><li>We’d like to think of the network as an abstraction </li></ul><ul><ul><li>Sockets = pipes </li></ul></ul><ul><ul><li>RPC looks like a normal procedure call, handles tricky stuff under the hood </li></ul></ul><ul><li>Still have to plan for failures of all kinds </li></ul>
    52. 52. Distributed Filesystems <ul><li>The network allows us to make data available across many machines </li></ul><ul><ul><li>Network file systems can hook into existing infrastructure </li></ul></ul><ul><ul><li>Specialized file systems (like GFS) can offer better performance with loss of generality </li></ul></ul><ul><li>Raises issues of concurrency, process isolation, and how to combat stale data </li></ul>
    53. 53. And finally… <ul><li>There are lots of distributed systems out there </li></ul><ul><li>MapReduce, BOINC, MPI, several other architectures, styles, problems to solve </li></ul>