Cloud Libraries


Published on

impact of cloud computing concepts on information access services and library organizations

Published in: Technology, News & Politics
1 Comment
  • Thanks! That was a great look into an area I've been researching for a presentation I'm giving next weekend. I'm doing a much simplified look at cloud work.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cloud Libraries

  1. 1. Cloud libraries April 2008
  2. 2. <ul><li>cloud computing concepts </li></ul>
  3. 3. <ul><li>get fuzzier over time ... </li></ul><ul><li>many different </li></ul><ul><li>broadly similar </li></ul><ul><li>definitions abroad </li></ul>
  4. 4. <ul><li>generally means: </li></ul><ul><li>widely distributed, </li></ul><ul><li>network based - </li></ul><ul><li>- storage, </li></ul><ul><li>- computation, </li></ul><ul><li>- SaaS models </li></ul>
  5. 5. <ul><li>single datacenter or -hosted apps > </li></ul><ul><li>moving to network based solutions </li></ul>
  6. 6. <ul><li>usually provided by </li></ul><ul><li>external parties on </li></ul><ul><li>a quanta basis, e.g. </li></ul><ul><li>compute cycle or byte traffic </li></ul>
  7. 7. <ul><li>a concept of </li></ul><ul><li>“ virtualization” </li></ul><ul><li>of infrastructure </li></ul><ul><li>or web service </li></ul><ul><li>or application </li></ul>
  8. 8. <ul><li>Poster Child: </li></ul><ul><li>Amazon’s services -- </li></ul><ul><li>S3 (simple storage system) and </li></ul><ul><li>EC2 (elastic compute cloud) </li></ul>
  9. 9. <ul><li>NY Times </li></ul><ul><li>converted its </li></ul><ul><li>TIFF backlist to PDF </li></ul><ul><li>with Amazon services (EC2 / S3) </li></ul><ul><li>and bill was ~ $500 </li></ul>
  10. 10. <ul><li>“ I then began some rough calculations and determined that if I used only four machines, it could take some time to generate all 11 million article PDFs. But thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3.” - Derek Gottfrid, NYT </li></ul>
  11. 11. <ul><li>services are offered </li></ul><ul><li>or increasingly </li></ul><ul><li>defined by </li></ul><ul><li>Google, </li></ul><ul><li>Amazon, </li></ul><ul><li>Microsoft (maybe), </li></ul><ul><li>IBM and others. </li></ul>
  12. 12. <ul><li>Google’s </li></ul><ul><li>MapReduce </li></ul><ul><li>break up a problem </li></ul><ul><li>allocate to many machines </li></ul><ul><li>reassemble for use </li></ul>
  13. 13. <ul><li>[1, 2, 3].map{|i| i * 10} </li></ul><ul><li>map{} </li></ul><ul><li>returns new array </li></ul><ul><li>[10, 20, 30] </li></ul>
  14. 14. <ul><li>map{} could be </li></ul><ul><li>{image processing} </li></ul><ul><li>{text search} </li></ul><ul><li>{metadata extraction} </li></ul>
  15. 15. <ul><li>reduce{} transforms </li></ul><ul><li>output of map{} </li></ul>
  16. 16. <ul><li>It doesn’t matter </li></ul><ul><li>how many parameters in list </li></ul><ul><li>every item handled independently </li></ul><ul><li>from every other item </li></ul>
  17. 17. <ul><li>means computation </li></ul><ul><li>over each item in list </li></ul><ul><li>can be distributed </li></ul><ul><li>to many computers </li></ul><ul><li>eg one VM per item </li></ul>
  18. 18. <ul><li>reduce{} combines results </li></ul>
  19. 19. <ul><li>“ ... a new super-group of tech companies [will] dominate the cloud computing market, each of them managing what amounts to a giant centralised computer made up of a number of big datacentres.” </li></ul><ul><li>- Steve Ballmer, MSFT </li></ul>
  20. 20. <ul><li>“ NSF Partners With Google and IBM to Enhance Academic Research Opportunities” </li></ul><ul><li>(Press Release 08-031) </li></ul><ul><li>even very high end universities </li></ul><ul><li>lag behind corporate computing </li></ul>
  21. 21. <ul><li>“ Fundamental changes in computer architecture and increases in network capacity are encouraging software developers to take new approaches to computer-science problem solving.” </li></ul>
  22. 22. <ul><li>“ In order to bridge the gap between industry and academia, it is imperative that academic researchers are exposed to the emerging computing paradigm behind the growth of &quot;Internet-scale&quot; applications.” </li></ul>
  23. 23. <ul><li>IBM is deeply committed </li></ul>
  24. 25. IBM Kittyhawk <ul><li> </li></ul><ul><li>comm/ </li></ul><ul><li>research_projects.nsf/ </li></ul><ul><li>pages/ </li></ul><ul><li>kittyhawk.index.html </li></ul>
  25. 26. IBM Kittyhawk <ul><li>&quot;We postulate that efficient, balanced machines with high-performance internal networks such as Blue Gene are not only significantly better choices for web-scale companies but can form the building blocks of one global-scale shared computer. Such a computer would be capable of hosting not only individual web-scale workloads but the entire Internet. ...” </li></ul>
  26. 27. <ul><li>in other words: </li></ul><ul><li>not your father’s </li></ul><ul><li>datacenter </li></ul>
  27. 28. “Hiding in Plain Sight, Google Seeks More Power” Photo Credit: Melanie Conner for The New York Times
  28. 29. <ul><li>cloud computing </li></ul><ul><li>has already </li></ul><ul><li>revolutionized </li></ul><ul><li>many content based </li></ul><ul><li>information services </li></ul>
  29. 30. <ul><li>Google Books </li></ul><ul><li>virtualizes </li></ul><ul><li>libraries </li></ul>
  30. 31. <ul><li>Flickr </li></ul><ul><li>virtualizes </li></ul><ul><li>image collections </li></ul>
  31. 32. <ul><li>iTunes </li></ul><ul><li>virtualizes </li></ul><ul><li>audio recordings </li></ul>
  32. 33. <ul><li>YouTube </li></ul><ul><li>virtualizes </li></ul><ul><li>video </li></ul>
  33. 34. <ul><li>Is there space for HE </li></ul><ul><li>to contribute to </li></ul><ul><li>virtualization layers ? </li></ul>
  34. 35. <ul><li>Most new initiatives </li></ul><ul><li>do not directly involve </li></ul><ul><li>research libraries as </li></ul><ul><li>central constituents </li></ul>
  35. 36. <ul><li>OpenLibrary </li></ul>
  36. 37. <ul><li>Zotero and Internet Archive </li></ul>
  37. 38. <ul><li>LibraryThing </li></ul>
  38. 39. <ul><li>If (ARTStor) did (Flickr) ... </li></ul>
  39. 40. <ul><li>libraries provide fairly </li></ul><ul><li>lightweight applications </li></ul><ul><li>or “soft infrastructure” </li></ul>
  40. 41. <ul><li>e.g. </li></ul><ul><li>faculty repository systems </li></ul><ul><li>online publishing platforms </li></ul><ul><li>boutique special collections </li></ul>
  41. 42. <ul><li>not </li></ul><ul><li>fundamental </li></ul><ul><li>infrastructure </li></ul><ul><li>provision </li></ul>
  42. 43. <ul><li>power is in aggregation </li></ul>
  43. 44. <ul><li>but there are </li></ul><ul><li>problems with </li></ul><ul><li>library collaboration </li></ul><ul><li>at scale </li></ul>
  44. 45. <ul><li>(it doesn’t happen) </li></ul>
  45. 46. <ul><li>similar to classic </li></ul><ul><li>political science issues </li></ul><ul><li>on sovereignty </li></ul>
  46. 47. <ul><li>EU one of the most successful </li></ul><ul><li>trans-national political efforts </li></ul>
  47. 48. <ul><li>Europe </li></ul><ul><li>tightly integrated </li></ul><ul><li>due to </li></ul><ul><li>- physical and </li></ul><ul><li>- historical and </li></ul><ul><li>- economic issues </li></ul>
  48. 49. <ul><li>consider </li></ul><ul><li>the difficulties </li></ul><ul><li>establishing </li></ul><ul><li>League of Nations </li></ul>
  49. 51. <ul><li>and post WW-II, </li></ul><ul><li>United Nations </li></ul>
  50. 53. <ul><li>United Nations </li></ul><ul><li>(as it exists today) </li></ul><ul><li>not (at all) what its </li></ul><ul><li>planners wished </li></ul>
  51. 54. <ul><li>Tremendous conflict </li></ul><ul><li>arose in San Francisco </li></ul><ul><li>between Russia (Molotov) </li></ul><ul><li>other Allied war parties </li></ul>
  52. 55. <ul><li>Even in the face </li></ul><ul><li>of the possibility </li></ul><ul><li>of world-ending </li></ul><ul><li>NUCLEAR WAR </li></ul>
  53. 56. <ul><li>Libraries have little beyond </li></ul><ul><li>established workflow </li></ul><ul><li>interoperation </li></ul><ul><li>like ILL </li></ul>
  54. 57. <ul><li>a few dominance hierarchies </li></ul><ul><li>like the CIC </li></ul><ul><li>or OhioLink </li></ul><ul><li>or the CDL </li></ul><ul><li>which do well in core operations </li></ul>
  55. 58. OCLC <ul><li>I see primarily as a vendor </li></ul><ul><li>in important but established </li></ul><ul><li>bibliographic service space </li></ul>
  56. 59. <ul><li>Difficult to establish </li></ul><ul><li>- much less maintain - </li></ul><ul><li>a  of large libraries </li></ul><ul><li>with serious long term </li></ul><ul><li>resource commitments </li></ul>
  57. 60. <ul><li>maybe the </li></ul><ul><li>best model is </li></ul><ul><li>support of </li></ul><ul><li>the equivalent of </li></ul><ul><li>something like </li></ul><ul><li>the IAEA </li></ul>
  58. 61. <ul><li>contribute </li></ul><ul><li>- dollars </li></ul><ul><li>- staff </li></ul><ul><li>- expertise </li></ul><ul><li>- technology </li></ul>
  59. 62. <ul><li>working </li></ul><ul><li>with others </li></ul><ul><li>but relying on </li></ul><ul><li>an independent </li></ul><ul><li>organizational </li></ul><ul><li>framework </li></ul><ul><li>for action </li></ul>
  60. 63. <ul><li>a possible role </li></ul><ul><li>for the DLF </li></ul><ul><li>“ marriage broker” </li></ul>
  61. 64. <ul><li>make introductions </li></ul><ul><li>between nascent efforts </li></ul><ul><li>and potential partners </li></ul>
  62. 65. <ul><li>like libraries </li></ul><ul><li>and museums </li></ul><ul><li>and archives </li></ul><ul><li>and tech firms </li></ul><ul><li>and societies </li></ul>
  63. 66. <ul><li>figure out domain- or discipline- </li></ul><ul><li>based early-stage faculty teams </li></ul><ul><li>seeking to develop new services </li></ul>
  64. 67. <ul><li>aid and abet -- </li></ul><ul><li>encourage faculty and </li></ul><ul><li>professional societies </li></ul><ul><li>to assume leadership </li></ul>
  65. 68. <ul><li>current thrusts </li></ul><ul><li>for the DLF ... </li></ul>
  66. 69. <ul><li>mobile apps </li></ul><ul><li>virtual worlds </li></ul><ul><li>moving images </li></ul><ul><li>science CI </li></ul>
  67. 70. <ul><li>focus today on mobile : </li></ul><ul><li>fundamentally reliant on </li></ul><ul><li>net- scale applications </li></ul>
  68. 71. mantra <ul><li>connect </li></ul><ul><li>communicate </li></ul><ul><li>collaborate </li></ul><ul><li>(CCC) </li></ul>
  69. 72. mobile apps <ul><li>iPhone </li></ul><ul><li>Android </li></ul><ul><li>wholly game changing </li></ul><ul><li>development platforms </li></ul>
  70. 73. <ul><li>coming : </li></ul><ul><li>social collaborative </li></ul><ul><li>learning environments </li></ul>
  71. 74. <ul><li>wild assed designs </li></ul><ul><li>portable </li></ul><ul><li>networked </li></ul><ul><li>computing </li></ul><ul><li>interfaces </li></ul>
  72. 75. petiteinvention
  73. 76. areas to watch Rudy de Waele - March 19, 2008 dot.Open
  74. 77. <ul><li>sensoring </li></ul><ul><li>biometrics </li></ul><ul><li>transactions </li></ul><ul><li>lifestreaming </li></ul><ul><li>recommendation </li></ul><ul><li>image recognition </li></ul><ul><li>augmented reality </li></ul><ul><li>mobile connected games </li></ul><ul><li>location based social media </li></ul><ul><li>retail proximity media consumption </li></ul><ul><li>Rudy de Waele - March 19, 2008 dot.Open </li></ul>
  75. 78. <ul><li>in  ... </li></ul>where ? libraries in the future
  76. 79. <ul><li>university libraries will not band </li></ul><ul><li>together in virtual organizations </li></ul><ul><li>to establish </li></ul><ul><li>bigger, better libraries </li></ul><ul><li>offering cloud based </li></ul><ul><li>information services </li></ul>
  77. 80. <ul><li>probably this </li></ul><ul><li>is against their </li></ul><ul><li>institutional nature </li></ul>
  78. 81. <ul><li>a significant difficulty in </li></ul><ul><li>moving into wholly different </li></ul><ul><li>domain areas (like mobile) </li></ul>
  79. 82. <ul><li>IN NEW ecological niches </li></ul><ul><ul><li>zero platform expertise </li></ul></ul><ul><ul><li>no collaboration : industry </li></ul></ul><ul><ul><li>a paradigm shift in IT </li></ul></ul>
  80. 83. <ul><li> libraries  </li></ul><ul><li>MUST </li></ul><ul><li>STOP defining </li></ul><ul><li>solutions </li></ul><ul><li>(in terms of) </li></ul><ul><li> libraries  </li></ul>
  81. 84. <ul><li>libraries must </li></ul><ul><li>learn how to : </li></ul><ul><li>partner || </li></ul><ul><li>collaborate || </li></ul><ul><li>contribute || </li></ul><ul><li>coordinate </li></ul>
  82. 85. <ul><li>via organizational frameworks </li></ul><ul><li>- external to us </li></ul><ul><li>- friendly to us </li></ul><ul><li>- supported by us </li></ul><ul><li>- sponsored by us </li></ul><ul><li>(but not us. </li></ul>
  83. 86. <ul><li>that’s a different </li></ul><ul><li>kind of library </li></ul><ul><li>more diverse </li></ul><ul><li>more opportunistic </li></ul><ul><li>more open </li></ul>
  84. 87. <ul><li>it’s a smaller library </li></ul><ul><li>it’s a more nimble library </li></ul><ul><li>it’s a library of pieces </li></ul><ul><li>broken up from something </li></ul><ul><li>unwieldy lethargic stifling </li></ul>
  85. 88. <ul><li>( It should be </li></ul><ul><li>a </li></ul><ul><li>more fun </li></ul><ul><li>LIBRARY ) </li></ul>
  86. 89. thanks peter at naypinya {twitter, twine, skype}