DLF Fall Forum 2012, Tales from the Cloud

3,246 views

Published on

Title: Tales from the Cloud: Experiences and Challenges

Event: DLF Fall Forum 2012, Nov. 4-6, Denver Colorado

Desription: This panel presentation offered first-hand institutional experiences of cloud adoption when developing and working with cloud-based solutions for the purpose of digital object preservation.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,246
On SlideShare
0
From Embeds
0
Number of Embeds
88
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

DLF Fall Forum 2012, Tales from the Cloud

  1. 1. Our Bits Sit on AWSTexas Digital Library and Amazon Ryan SteansSenior Program Coordinator, Texas Digital Library info@tdl.org
  2. 2. Agenda• Who we are• Fall 2010 – where we were• Why we looked at Amazon• Spring 2011• Today and Looking Forward TDL DLF Presentation
  3. 3. What is Texas Digital Library? • 14 Member Consortium of Libraries at  academic/ research institutions across  Texas • TDL provides centralized software support  and development • Cost share • Provide scholarly communication  options to schools all at the same level TDL DLF Presentation
  4. 4. Texas Digital Library Services • DSpace internet repositories (DuraSpace) • Vireo ETD Management System (TDL) • Open Journal System (PKP) • Open Conference System (PKP) • WordPress for research group websites and  blogs • MediaWiki for research group data sharing and  management • Training, consultation, conference, community TDL DLF Presentation
  5. 5. TDL Activities – in the works • Digital Preservation Network • Fedora/ Hydra • Data management planning • Shared research repositories/ communities • Vireo users community (thesis and dissertation  community) TDL DLF Presentation
  6. 6. Fall 2010 • TDL located on UT Austin Campus • TDL hardware at UT Data Center • Data Center move scheduled for December  2010 for all of UT campus • TDL hardware was very close to expiration  of service contracts, etc… • TDL began looking at AWS for a backup  option during the move TDL DLF Presentation
  7. 7. Texas Digital Library Inventory - Austin Make Model QuantityUsage Database / Application Server / DNS / LDAP / Handle /Servers Sun SunFire 490 2 Shib Dspace / WWW / Blogs / Wikis / Faculty Pages / Journals Sun Sunfire T5220 2 / Mail / Services / Support / SVN Sun Sunfire 245 2 Dspace Labs / Development Sun Sunfire T5120 1 Preservation (iRods/SAM-QFS) Old content management servers (positioned to help w/ Sun Sunfire X4150 2 web applications )NetworkCisco Cataylst 3750 2 Network connectivity Citrix Netscaler 2 Load Balancers BrocadeSilkworm 300E 2 SAN Connectivity AvocentCycladed ACS 2 Serial Console Connectivity Dspace Assets / Web Application Data / HomeStorage NetApp Filer 3020 2 Directories / Backup Preservation (iRods / SAM-QFS) / Virtual Machines / Sun StorageTek 614 1 Databases Sun SL500 1 Preservation (iRods / SAM-QFS) Texas Digital Library Inventory - College StationServers Sun T 5220 2 Dspace Sun T 5120 1 PreservationStorage NetApp Filer 3020 1 Dspace Assets / Backup TDL DLF Presentation
  8. 8. 2010 ‐ Our existing hardware • Very locked down with space • Had to over‐buy in order to guess need • Limited staff was worrying about physical  hardware • All eggs in one basket • Tied to university security protocols that were  difficult with multiple members • IT infrastructure had led to bad user experience – shared hardware, machines, etc… TDL DLF Presentation
  9. 9. Looking at Amazon Web Services • ELASTIC • Could increase size of a machine • Could increase number of machines • Removed guess work and bad estimates  from the equation • Elastic cost • Service Contracts were highly reasonable • Road tested.  AWS launched in 2006. TDL DLF Presentation
  10. 10. Spring 2011 • We moved to AWS for all services  TDL DLF Presentation
  11. 11. Why? • Allowed us to separate service instances,  bring each up on a VM • Created far greater user customization/  access/ control • Rapid deployment of services from master  image • Accounting TDL DLF Presentation
  12. 12. How? • Set up a duplicate system in AWS • Used load balancer as the fulcrum • Caused a minimum of downtime TDL DLF Presentation
  13. 13. Today • Running 100 separate AWS instances as of  11/19/2012 • Deploying services easily • Literally nothing is on UT’s hardware nor at  any TDL member school • Depend on standard security and AWS • Cost TDL DLF Presentation
  14. 14. TDL DLF Presentation
  15. 15. It’s actually working just fine • A bit of “beg forgiveness instead of asking  permission” • Costs are down • Members don’t notice • Our staff is handling it well (2 longtime  university IT people as sys admins) TDL DLF Presentation
  16. 16. An overly simple idea of service  provisioning TDL DLF Presentation
  17. 17. But… • Network cost could become burdensome • Cost savings per unit no longer as obvious • UT and our own team more comfortable  with VMs and could move to purchased  hardware • AWS billing not easy to manage month‐to‐ month TDL DLF Presentation
  18. 18. And… • As we look at the Digital Preservation  Network  ‐ we will rebuild on campus • Security concerns • Goal of group is to do this in academia • It may also affect how we think of  implementing Fedora/ Hydra TDL DLF Presentation
  19. 19. Week of October 22nd… • TDL learned of outage at 1:15 PM Central • Services were visible, but not accessible • Contacted members • Watched logs, news reports, etc… • 7:30 PM we were back up and running  with no data loss • Received 7 communications through  helpdesk TDL DLF Presentation
  20. 20. It’s bad, but… • AWS was in communication with users • We feel confident in them, and they got it  back up • Checking our systems was fairly easy • Took some pressure off our staff • Outages are not limited to AWS TDL DLF Presentation
  21. 21. Looking forward • We will always rethink our strategy on  AWS • AWS is providing us with a sales person  and a consultant before end of year • Cost has to remain reasonable for  uploading, downloading • Will be moving preservation efforts to  campus TDL DLF Presentation
  22. 22. Ryan Steans, Program Coordinator Kristi Park, Program Coordinator info@tdl.org
  23. 23. Tales from the Cloud:Experiences and Challenges Andrew Woods http://duracloud.org 04 Nov 2012
  24. 24. Why might I even beinterested in “the cloud” ? 2
  25. 25. What are some risks ? 3
  26. 26. How is it different from local infrastructure ? 4
  27. 27. What about this “DuraCloud” ? 5

×