Successfully reported this slideshow.

OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services

1,704 views

Published on

The Dispatch Printing Company is a leading regional media company in the USA, anchored by its flagship newspaper The Columbus Dispatch. Its Dispatch Broadcast Group owns and operates two TV stations, the WBNS radio station, the Ohio News Network radio service, and a 24-hour cable news channel.

This session is a case study in migrating OpenCms sites, generating millions of daily page views, from a traditional data center to the Amazon Web Services platform. Through this migration there were many lessons learned about how to successfully use Amazon's cloud service offerings to improve OpenCms scalability and lower total costs to the business. An overview of select Amazon services and how they have been leveraged in a production OpenCms environment will be presented.

We will talk about possible uses for a variety of Amazon services including:

EC2 - Implementation strategy for running OpenCms on Amazon's Elastic Compute Cloud virtual hardware
CloudWatch - Provide detailed visibility into the health of an OpenCms environment
Simple Storage System - Work with OpenCms's export functionality to push exported files directly to Amazon's web accessible storage space
CloudFront - Leverage the power of a content delivery network for your OpenCms environment
We will discuss the effort prior to launch to convince the business that Amazon would be reliable, allow for a disaster recovery plan, be secure, and save the business money. We will provide tips on how we setup our infrastructure to alleviate the various concerns the business had.

The first service leveraged was Amazon CloudWatch. This service can provide a detailed look at the health of the entire OpenCms infrastructure with little to no custom development effort. This includes the ability to quickly create alerts and notifications for when anything goes wrong in your environment.

We also decided to leverage Amazon Relational Data Services. We will present the trade-offs in the decision to use a managed data layer and how we justified taking the managed database approach.

Finally, we will briefly cover the other Amazon services that have been used as a part of our OpenCms deployment including ElastiCache, CloudFront, Simple Queue Service, Simple Email Service, SimpleDB, and Amazon S3.

Published in: Technology
  • Be the first to comment

OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services

  1. 1. OpenCms:Amazon WebServices (AWS)
  2. 2. The PresentersDavid Landreman Web Services IT Manager Email: dlandreman@dispatch.com Twitter: @GraphIt2000 LinkedIn: www.linkedin.com/in/davidlandremanAndrew Roth Senior Internet Development Engineer Email: aroth@dispatch.com Twitter: @guitarplayinpro LinkedIn: www.linkedin.com/in/rothandrew
  3. 3.  Newspapers, weekly periodicals, TV stations, and radio stations in Columbus, Ohio, USA 23 websites hosted on Amazon in OpenCMS 8  http://www.dispatch.com  http://www.10tv.com Millions of pageviews daily 5 Years of content  700,000 HTML files  1.5 Million Resources (images, video metadata, etc.) NginX Web Tier (4 Servers) Tomcat 7 App Tier (4 Servers)
  4. 4. OpenCms Customizations Over 40 custom modules developed Advanced RSS Feeds Video Support XML Content Editor Changes Content Relationship Engine (CRE) Custom User Administration Tools Solr Search Integration Workplace UI Enhancements  Advanced Properties Dialog  Shortcuts Bar
  5. 5. Custom RSS Feed Module More than content management for websites  Amazon Kindle  Native iPhone / Android Apps  Facebook Pages Support for value lookup in nested resources  E.g. Story Resource includes VFS path to Author and the RSS should include author’s twitter handle Filtering by content type Easily Extendable  Media RSS for Video feeds  GeoRSS
  6. 6. Video Support Video management completely inside OpenCms for content editors Admin tools  Uploading new videos  Viewing transcode status Integration with RSS module for MRSS feeds Automatic video thumbnail generation  Thumbnail added to VFS Only video metadata is stored in OpenCms  Video files stored out to a CDN  Over 1 Terrabyte video content
  7. 7. Content Relationship Engine (CRE) New user interface to support a large number of categories  700+ categories for advertising purposes Tagging support List functionality to search based on tags and categories  Developed prior to OpenCms 8
  8. 8. Solr Integration All OpenCms servers do not need to maintain independent lucene query Added features for public facing search  Faceting  Stopwords  Synonyms
  9. 9. Demo
  10. 10. Cloud Migration Project to upgrade from OpenCms 6 to OpenCms 7 (later upgraded to OpenCms 8) Original plan was to migrate hardware from physical to VMWare at our thrid party data center 2 months prior to completion the decision was made to migrate to Amazon Web Services (AWS)
  11. 11. Amazon Selection Factors Team familiarity with AWS Ease of beginning an engagement  No contract Cost  Limited payment options Large client base (Netflix, Instagram, …) Large selection of services beyond virtual computing resources
  12. 12. OpenCms on AWS S3 / EC2 Admin Tools CloudWatch & Simple Notification Service Simple Email Service (SES) Relational Database Service (RDS) ElastiCache
  13. 13. S3 / EC2 Overview What is S3 (Simple Storage Service)  File storage in the cloud  Our uses:  Backing up a SQL dump of our production database  Temporary storage of video content during transcoding process  Serving some static web content (CSS, JS, etc) What is EC2 (Elastic Compute Cloud)?  Web service that provides resizable compute capacity in the cloud. (ie. servers)
  14. 14. S3 / EC2 OpenCms Admin Tools S3 Admin Tools in OpenCms  Allows for browsing the contents of all our S3 buckets from within OpenCms  Can download content directly from admin tool EC2 Admin Tools in OpenCms  Lists ALL servers in our environment including:  Server status (running, stopped, etc)  Public DNS entry  Instance ID  Security Group  Link directly to Tomcat admin portal if server is running it
  15. 15. CloudWatch & Simple Notification Service (SNS) CloudWatch provides monitoring for AWS cloud resources and applications. Default metrics for all EC2 Instances  e.g. CPU, memory, disk IO  1 minute measurement interval Tracking custom metrics  OpenCms Publish Times, Java Heap Usage, Database Connections Alert Thresholds (SNS)  Text message, email, JSON posts
  16. 16. Simple Email Service (SES) Managed service for bulk email sending CloudWatch Integration  Monitor deliveries, bounces etc. from within AWS console Multiple endpoints  SMTP  Web Service Calls Uses  Currently send all website registration emails through SES  Goal is to replace Exact Target with SES for mass emails (newsletters, etc) to reduce costs
  17. 17. Relational Database Service (RDS) Managed Relational Database (PaaS)  MySQL, Oracle, MSSQL  We use MySQL Change to traditional database administration  No access to server console  No access to a true SA account Active / Passive Failover Configuration Easy server upgrades Automatic backups
  18. 18. ElastiCache Managed Memcached Service  Web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud. Integration of ElastiCache metrics with CloudWatch  Hit rates, Eviction Rates etc. Similar to Alkacon OCEE Accelerator cache  Memcached allows for one global cache copy as opposed to one on each app server Reduces hits to our database servers, faster page loads , increases ability to handle high traffic volumes, and gets us closer to goal of autoscaling.
  19. 19. Demo
  20. 20. The Future• Automatic Horizontal Scaling• CloudFront as a Content Delivery Network (CDN)• CloudSearch to replace Solr Search
  21. 21. Auto Scaling Issueswith CurrentArchitecture
  22. 22. Auto Scaling Issues with Current Architecture (cont.) Why autoscale?  Increased scalability and reliability  Reduced costs Issues with AutoScaling OpenCms  Sticky Sessions  In the cloud sticky sessions are a bad thing  Direct communication between web and application servers  Application server synchronization  Direction communication across application servers (Alkacon Cluster Module)
  23. 23. Automatic Horizontal Scaling - Solutions Memcached session management with Tomcat Implement SQS (Simple Queue Service) for communication between the workplace and the application servers for application server synchronization Allow web and application tiers to expand / contract independently of each other  Load balance traffic between web and application layers
  24. 24. Automatic HorizontalScaling - Solution
  25. 25. CloudFront as aContent Distribution Network (CDN) Currently use Limelight for video distribution Moving videos to CloudFront will be a big cost savings Front static content from OpenCms to decrease page load times  Images, CSS, JavaScript
  26. 26. CloudSearch Fully managed search service (PaaS) API similar to Solr API  Reduced features Replace our Solr implementation which removes the need to manage a separate server dedicated to search Improved Scalability / Autoscale
  27. 27. Questions?David Landreman Web Services IT Manager Email: dlandreman@dispatch.com Twitter: @GraphIt2000 LinkedIn: www.linkedin.com/in/davidlandremanAndrew Roth Senior Internet Development Engineer Email: aroth@dispatch.com Twitter: @guitarplayinpro LinkedIn: www.linkedin.com/in/rothandrew

×