BY AARON WEISS     COMPUTING IN      THE CLOUDS         Powerful services and applications are being       integrated and ...
To some, the cloud looks like Web-      tion, of course. Desktop (and laptop)             based applications, a revival of...
small number of high-powered                 ter at a derelict factory site on theand very expensive servers, why         ...
by Google and the many other enter-        into sub-tasks. Those sub-tasks can     prises inspired by their model, each   ...
work for developers to rapidly             would submit jobs to these number-employ distributed computing in a          cr...
viduals or businesses. Although host-       machine in Amazon’s cloud. Using              ing providers can go some way   ...
Amazon then charges customers            these kinds of massive, sandboxedbased on their cumulative “instance         data...
Taken further, Web-based applica-     ruptive force and a whole new way.              tions like Google Docs threaten core...
choose efficient, uncomplicated access         Software as a service in the cloudmachines. This argument, and its         ...
Upcoming SlideShare
Loading in …5

Computing in the clouds weiss


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Computing in the clouds weiss

  1. 1. BY AARON WEISS COMPUTING IN THE CLOUDS Powerful services and applications are being integrated and packaged on the Web in what the industry now calls “cloud computing” Quick, go outside and look up at the clouds in the sky. What shapes do you see? A ball of cotton? A bunny rabbit? A massively parallel distributed data center? If the latter sounds spot on, you might already be computing in the clouds. At least, that’s the latest catchphrase buzzing around the industry. “Cloud computing,” as it’s being called by everyone from IBM to Google to Amazon to Microsoft, is supposedly the next big thing. But like the clouds themselves, “cloud computing” can take on different shapes depending on the viewer, and often seems a little fuzzy at the edges. Illustration by James O’Brien16 COMPUTING IN THE CLOUDS DECEMBER 2007
  3. 3. To some, the cloud looks like Web- tion, of course. Desktop (and laptop) based applications, a revival of the computing power has been on an thin-client. To others, the cloud looks accelerated tear for 30 years. But the like utility computing, a grid that networked era, and the data centers charges metered rates for processing that power it, are starting to make time. Then again, the cloud could be the IT industry—and its investors— distributed or parallel computing, take a fresh look at Watson’s per- designed to scale complex processes spective on centralized computing, if for improved efficiency. Maybe every- not his specific enumeration. Both Google and IBM have a vested interest in encouraging cloud computing: THEY NEED PEOPLE TO HIRE. one is right. There are many shapes But data centers aren’t new, either. in the clouds. In the dot-com boom of the mid- ’90s, many a startup invested venture Cloud Shapes: The Data Center capital into traditional enterprise It is not news that today’s major solutions like Sun SPARC servers. Is Internet companies have built massive this the cloud? data centers to power their online Yes and no. Google is often credit- businesses. Decades ago, computing ed with innovating search on the power was concentrated in main- Web and, more recently, advertising. frames tucked away behind the scenes But to many, Google’s architecture because there was no alternative— behind the scenes has spawned just only a hulking room-sized box that as significant a revolution. could contain any significant amount The data centers of the early dot- of computational power. The idea com era were, in some respects, that this power could be distributed direct descendents of Watson’s main- rather than centralized seemed like frames. Physically smaller, perhaps, such folly that in 1943, IBM but Sun servers and their ilk contin- Chairman Thomas Watson said ued to represent an exclusive kind of famously (or infamously) that “I computing—concentrated power think there’s a world market for designed, and priced, for exclusive maybe five computers.” customers—namely, enterprise. The era of the personal computer But Google turned the data that has flourished since the 1970s center model on its head. Rather directly contradicted Watson’s predic- than power a network with a18 COMPUTING IN THE CLOUDS DECEMBER 2007
  4. 4. small number of high-powered ter at a derelict factory site on theand very expensive servers, why banks of the Columbia River in thenot deploy cheap, commodity Pacific Northwest, located both nearhardware in large numbers? cheap hydroelectric energy and a Today, Google runs an estimated major trans-Pacific Internet node.half-million servers clustered into a Likewise, Microsoft, IBM, and othersdozen or so physical locations. By are following suit, scouting sites both increating a network that is spread thin the Pacific Northwest and Canada whereand wide rather than narrow and hydroelectric power is cheaper (and green-deep, Google created a new kind of er) than the coal-derived power usedconcentrated power—derived more throughout much of the U.S. The eyes offrom scale of the whole than any one investors are also turning toward China,constituent part. This, some say, where new power plants are being rapidlydescribes the cloud. built (and, some are quick to caution, As computational and networking without the “costly” burdens of environ-architecture, the cloud is very robust. mental controls).Sometimes described as “self-heal- Bing,” a thin, wide network can recov- esides cheaper power,er gracefully from the most common data centers are makingailments, such as connection and heavy use of virtualiza-hardware failures, because there are tion to squeeze the mostso many more drones available to out of the watts they’retake on the work. consuming. With major But a cloud can consume a lot of vendors like VMWarepower to run. Aside from the power and Citrix, which recently acquiredneeded to drive thousands, or hun- the Xen virtualization platform,dreds of thousands, of processors increasingly targeting the data center,and peripherals—hard drives, cooling virtualization allows a single serverfans—all these whirring machines to run multiple operating instancesgenerate lots of heat. It is estimated simultaneously. By sandboxing eachthat 50 percent of energy costs in OS inside artificial boundaries, notrunning a large data center are only can each instance run indepen-derived from cooling needs alone. dently of the others, but CPU idle Worse still for the cloud, the world time is immersed in a global energy Just what distinguishes a “cloud”crunch, as both demand and specula- from “a bunch of machines” can be ation has driven up pricing for most little fuzzy. But the next evolutionconventional energies toward record that may illuminate the fog is the so-levels. called “data center OS”—or, in the Reducing the operating costs of a spirit of the buzz, the CloudOS. InGoogle-inspired data center cloud the fall of this year, VMWare andinvolves both physical and virtual Cisco announced a joint venture tosolutions. Physically, data centers are develop such a “fabric” (to use theirlike plants arcing toward the sun- word in spite of the mixedlight, migrating toward cheap energy. metaphor).Google is building a major data cen- In a data center like that employed COMPUTING IN THE CLOUDS DECEMBER 2007 19
  5. 5. by Google and the many other enter- into sub-tasks. Those sub-tasks can prises inspired by their model, each be broken into even smaller tasks, server is fundamentally an indepen- and so on, until you’re nearly down dent machine running its own copy to “bare metal” as they say and deal- of an operating system. For servers to ing with disk and memory access. work together on common tasks I requires an abstraction layer of soft- deally, if tasks are broken into ware—often, highly specialized, cus- their smallest constituent jobs, tom software—that intelligently and each job could be complet- divvies up jobs. But a more efficient ed simultaneously using avail- and resource-friendly solution would able processing resources be a single operating system, which somewhere in the cloud, you intrinsically utilizes the resources of could achieve an optimally effi- many machines. cient architecture: the most optimistic Essentially, an operating system is definition of distributed computing. designed to manage resources—hard In reality, some jobs are dependent drive space, memory, and so on. A on the results of other jobs. true data center, or cloud, OS will Furthermore, designing algorithms to treat every processing unit available most effectively divide jobs and dis- as just another resource, relying on tribute them throughout the cloud in networking channels to replicate the real time is complex, to understate kinds of intra-server channels that the case. now coordinate events within a single But distributed computing, like the physical machine. Under the com- data center itself, is not inherently mand of a single “omniscient” oper- new to the era of the cloud. If we ating system, the cloud becomes a consider the entire Internet a cloud, more cohesive entity. one need only look at popular pro- jects like SETI@home and Cloud Shapes: Distributed Computing Folding@home to see public exam- Whether “the cloud” represents a ples of distributed computing at data center at a single physical loca- work. In these projects, individuals tion or dozens, hundreds, or thou- run software on their PC which con- sands of data centers spread around nects them to a server that divides the world, its speed and efficiency is large jobs among small clients to limited by how intelligently it dele- crunch numbers toward a particular gates responsibility. goal—in these examples, searching Completing any general computing for alien life among radio waves and task—say, retrieving the results of a computing protein folding simula- search with relevant contextual tions to aid medical research. You advertisements—requires a long could even consider botnets—soft- series of smaller jobs to be complet- ware that maliciously infects unwit- ed. Database queries, parsing of ting client machines to send out results, construction of result sets, parcels of spam—a form of nefarious and formatting of result pages, to distributed computing. name the most common. Even these The open source project Hadoop tasks can be further broken down provides a general-purpose frame-20 COMPUTING IN THE CLOUDS DECEMBER 2007
  6. 6. work for developers to rapidly would submit jobs to these number-employ distributed computing in a crunching powerhouses (in theirwide variety of projects. Actively day), and be billed for the cost ofunder development within the aus- cycles used. Processing time waspices of the Apache Foundation, delivered like electricity—you paidHadoop liberates software developers for what you used.from creating specialized custom Today, most medium to large-sizedsoftware for taking advantage of dis- organizations invest in their owntributed computing in a cloud. data centers and use them at will.Capable of distributing petabytes of Although processing cycles are notdata-processing across thousands of metered like a utility, there are signif-computing nodes, the Hadoop plat- icant costs to building a data cen-form is itself comprised of several ter—real estate, hardware, power,WHY NOT JUST MOVE ALL PROCESSINGPOWER TO THE CLOUD, and walk around with anultralight imput device with a screen?technologies to ensure efficiency and cooling, and ongoing integrity as data swirls around What’s worse for the balance sheetthe cloud. Some of these sub- is that organizations need to plan forprojects, like MapReduce which worst-case scenarios. In the case of adivides jobs into component tasks, data center, not only can this includeand HDFS, the Hadoop Distributed the costs of backup and redundancy,Filesystem, can be employed individ- but in overpowered servers capableually or together by developers in of handling loads which can peaktheir applications. high but occur infrequently. Bus- inesses may find themselves using 99Cloud Shapes: A Utility Grid percent of their computing capacityIt seems like every shape one sees in only 10 percent of the time, leavingthe cloud is inspired by a computing expensive equipment often idle andmodel from the past. Back in the angering the accountants.days of mainframes and fancy super- In the Web space there is a thriv-computers housed at research univer- ing market of hosting providers whosities, valuable processing time was invest in their own data centers andessentially for sale. Researchers sell usage to customers, be they indi- COMPUTING IN THE CLOUDS DECEMBER 2007 21
  7. 7. viduals or businesses. Although host- machine in Amazon’s cloud. Using ing providers can go some way virtualization, Amazon presents the toward minimizing underutilization machine image to the end user as if it of in-house servers by attaching sur- were a dedicated server, with the charges to peak usage, their capacity same degree of access an administra- to stretch can be limited. Providers tor would have to their own server. typically assign clients to a whole or Customers can choose configura- partial physical server, and do not tion templates for their machine truly replicate the kinds of cloud fea- instance—for example, a server with tures, like distributed computing, 1.7GB of memory, one processing Software, in a cloud, becomes a service. To a company like Microsoft, whose substantial fortunes are built on software as a purchasable, local application, THE THREAT IS NOT FALLING ON DEAF EARS. that a dedicated data center can offer. core, and 160GB of storage space. Meanwhile, massive companies Additional configurations support like Amazon, Google, and IBM have more memory, more cores, and more invested in, innovated, and become storage. But the real magic in EC2 is expert at housing their own large- that customers can create and destroy scale data centers. Invoking the idea machine instances at will. As a result, of the cloud, all three also sense software can scale itself to exactly opportunity: Why not scale up their the amount of computing power it data centers—grow the cloud—and requires. create business models to support Consider a Web-based application third-party use? running in Amazon’s cloud. Suppose In fact, Internet retail giant there is a sudden surge in visitors, Amazon is the first out of the gate to perhaps thanks to media coverage. commercialize their cloud. After a Today, many Web applications fail period of limited access testing, under the load of big traffic spikes. Amazon opened their Elastic But in the cloud, assuming that the Compute Cloud, or EC2, to fee- Web application has been designed based public use in October 2007. intelligently, additional machine To use Amazon’s EC2, customers instances can be launched on first create a virtual image of their demand. The application dynamically, complete software environment using and gracefully, scales up. When traffic provided tools. The image is then slows down, the app can scale down, used to create an “instance” of a terminating the extra instances.22 COMPUTING IN THE CLOUDS DECEMBER 2007
  8. 8. Amazon then charges customers these kinds of massive, sandboxedbased on their cumulative “instance data centers dedicated for studenthours,” a way of metering usage, use, and many lack instructors withcurrently set at $0.10 per instance leading-edge experience.hour for the most basic instance type. To some industry analysts, com-To provide consistent and predictable modified cloud computing likeperformance, machine instance types Amazon’s EC2 will change the faceare rated by what Amazon calls of enterprise computing. It may pave“compute units,” which deliver a the way to where businesses nospecific, known amount of perfor- longer invest anything into data cen-mance, regardless of the underlying ters of their own. Out goes the hard-hardware Amazon is using inside ware, out goes the power bill, andtheir cloud. A second charge applies out goes all the processing powerto data moved in and out of that rarely gets used. Instead, theyAmazon’s network, ranging from buy cloud computing time from com-$0.10 to $0.18 per gigabyte. mercial providers, who can specialize In contrast to Amazon’s commer- in building massive data centers atcial effort, Google and IBM recently sites selected to minimize operatingannounced a partnership to apply a costs.similar utility cloud model to com-puter science education. The two Cloud Shapes: Software as a Servicecompanies have put together a dedi- As radical a shift as cloud computingcated data center mixing both com- may represent for the enterprise—modity and enterprise hardware. who needs servers?—some think itRunning open source software will radically change personal com-including Linux, Xen virtualization, puting even more. Who needs com-and Apache Hadoop, the puters at all? At least, the kind thatGoogle/IBM cloud presents a distrib- eat up battery life and contain churn-uted computing canvas for educa- ing hard drives. Why not just movetional access. Both Google and IBM all processing power to the cloud,have a vested interest in encouraging and walk around with an ultralightcloud computing: They need people input device with a screen?to hire. Some say it’s already begun. Burgeoning Web applications haveA considerable concern been the rage for several years. among major enterprise Fueled by technological evolutions is that today’s computer like AJAX, which allows browser- science students lack based code to behave more like a access to the distributed local application, and people’s desire computing environ- for mobility and data ubiquity, we ments that make up the increasingly use the Web for applica-cloud. Companies like Google will tion functionality. E-mail was bothneed new hires experienced in writing the first “killer app” for the Internet,code designed to run in a cloud of and later, on the Web. For manyperhaps thousands of processors. But today, Web-based e-mail is the onlyeducational institutions don’t have kind they use. COMPUTING IN THE CLOUDS DECEMBER 2007 23
  9. 9. Taken further, Web-based applica- ruptive force and a whole new way. tions like Google Docs threaten core Carr predicts that Google and Apple productivity applications on the will partner to push the cloud com- desktop. While the features of puting/software-as-a-service model Google Docs are a slim shadow of a even further. He foresees a lightweight major, and majorly profitable, appli- mobile device crafted by Apple that cation like Microsoft Word, it pro- will tap into Google’s cloud, bringing vides a taste of a cloud-based future. together the two masters of the front Your data is reachable anywhere, and end and the back end. the only software you need to access In the near-term, a ubiquitous it is a browser. Which, in turn, means cloud faces obstacles. Critics argue THE CLOUD DEMANDS A HIGH DEGREE OF TRUST. Significant amounts of data which were previously stored only in individual offices and homes would now reside in data centers controlled by third-parties. that your data and your applications that visions like Carr’s are simply are available in one form another revivals of failed thin-client dreams from your desktop PC, your laptop, of the past. Thin clients like those and your handheld. touted by Oracle-founder Larry Software, in the cloud, becomes a Ellison in the ’90s have not managed service. To a company like Microsoft, to become cost-effective. With prices whose substantial fortunes are built plummeting and performance sky- on software as a purchasable, local rocketing for full-featured desktop application, the threat is not falling and laptop computers, it has been on deaf ears. Windows Live, difficult to produce a relatively pow- Microsoft’s “cloud,” may be today’s erless thin client at a low enough cost take on Windows 1.0—a look into to attract buyers. the future. Advocates for thin client cloud Even Adobe has begun to launch computing argue that full-fledged stripped-down versions of its compu- machines, however powerful, are also tationally-heavy marquee titles as a hassle with negative productivity. Web-based services, including They have many parts that can fail Photoshop and the video editing suite and their software needs constant Premiere. care and feeding in a world riddled Pundits and futurists like Nicholas with software updates, viruses, and Carr love this stuff because the cloud spyware. Centralizing processing represents a paradigm shift and a dis- power in the cloud liberates users to24 COMPUTING IN THE CLOUDS DECEMBER 2007
  10. 10. choose efficient, uncomplicated access Software as a service in the cloudmachines. This argument, and its can revive fears of vendor lock-in, apotential economic feasibility, gains significant consideration in the main-weight in today’s environment, where frame era. Supposing a cloud opera-laptop sales are outpacing the desk- tor and a thin-client vendor partnertop, and even more mobile devices together, it is possible that each halfare becoming commonplace. will require the other. Services from the cloud may be inaccessible toF or cloud computing to those without an access device from a move front and center, the single brand. Some fear that the networks that tie every- cloud could encourage the growth of thing together need to be walled-gardens, a potential step back extremely robust. After compared to the relatively open all, the cloud represents a Internet of today. significant inversion frompersonal computing in the 1970s and Partly Cloudy?1980s, when machines stood alone There are those who scoff thatand derived all of their utility from “cloud computing” is just the latesttheir own capabilities. Under the branding of some old, familiar com-cloud, client machines become nearly puting models—which is partly true.useless on their own. Are our net- It’s a buzzword almost designed to beworks ready to handle the load? vague, but cloud computing is more Many would say no, especially than just a lot of fog.those living in the large U.S. market, The cloud concept draws on manywhere broadband quality and pene- existing technologies and architec-tration fares poorly relative to many tures. Centralizing computing powersmaller industrialized nations. In con- is not new and, if anything, is atrast to local computing power, which return to the computer’s roots. Nor iscontinues to boom year after year, utility computing new, or distributedbroadband advances here have been computing, or even software as aslow and may create at least a short- service.term bottleneck holding back a major But the cloud is new in that it inte-shift toward mainstream cloud com- grates all of the above computingputing. models. This integration requires Network bandwidth aside, the computing’s center of power to shiftcloud raises concerns among privacy from the processing unit to the net- PERMISSION TO MAKE DIGITAL OR HARDadvocates. Most significantly, the work. Inside the cloud, processors COPIES OF ALL OR PART OF THIS WORK FOR PERSONAL OR CLASSROOM USE IS GRANT-cloud demands a high degree of trust. become commodities. It is the net- ED WITHOUT FEE PROVIDED THAT COPIESSignificant amounts of data which work that holds the cloud together, ARE NOT MADE OR DISTRIBUTED FOR PROFITwere previously stored only in indi- ~ and connects the clouds to each other OR COMMERCIAL ADVANTAGE AND THAT COPIES BEAR THIS NOTICE AND THE FULLvidual offices and homes would now and the sky to the ground. CITATION ON THE FIRST PAGE. TO COPYreside in data centers controlled by OTHERWISE, TO REPUBLISH, TO POST ONthird-parties. Do we have adequate SERVERS OR TO REDISTRIBUTE TO LISTS, Aaron Weiss is a technology writer and REQUIRES PRIOR SPECIFIC PERMISSIONprivacy laws? How should encryption Web developer shivering in upstate New York, as AND/OR A a role? well as human proprieter of © ACM 1091-556/07/01200 $5.00 COMPUTING IN THE CLOUDS DECEMBER 2007 25