Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dealing with High Demand--Building high-capacity mapping applications


Published on

Is your mapping application ready for the big time? Maybe a better way to ask this question is, how well will the computing infrastructure that supports your mapping application stand up to a huge spike in demand?

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Dealing with High Demand--Building high-capacity mapping applications

  1. 1. Dealing with High DemandBuilding high-capacity mapping applicationsBy Keith Mann, Esri Writer A Review of Amazon EC2 The strategy for adding capacity to the wildfi re app uses Amazon Web Services (AWS)—specifically, Amazon Elastic Compute Cloud (EC2) ( Amazon describes EC2 as “a web service that provides resizable compute capacity in the cloud.” Essentially, various components of Amazon’s compute infrastruc- ture can be rented to rapidly build a high-capacity support system. The strategy also uses ArcGIS for Server to create and manage the GIS services for the application. Esri provides ArcGIS for Server as an Amazon Machine Image (AMI) running on Amazon EC2. The ArcGIS for Server on Amazon EC2 AMI includes 100 gigabytes of The Public Information Map template can be configured to allow storage. These AMIs can be licensed in the same fashion that ArcGISthe user to add features to the map. for Server is licensed on a local machine. When you start an instance of ArcGIS for Server on Amazon EC2,I s your mapping application ready for the big time? Maybe a you choose the ArcGIS for Server AMI, the size and type of machine better way to ask this question is, how well will the computing it will run on, and other services and features such as Elastic Load infrastructure that supports your mapping application stand up Balancing and Auto a huge spike in demand? You can upload your data and application to the instance’s storage Wildfi res raging in Colorado in 2010 precipitated a real-life test area and configure them with the ArcGIS for Server instance so thatfor Esri. In response to the fi res, Esri published a public informa- you can publish services and add them to the mapping application.tion map (PIM) for wildfi res. The wildfi re app ran beautifully on a Detailed resources for accomplishing these tasks are listed in thesingle server machine and seemed to be handling the daily traffic online version of this article.with no problem until a major news organization provided a link tothe map in one of its stories. Immediately, demand went throughthe roof, and the server exceeded its load capacity. It shut down,and the wildfi re app went dark. The application was quickly reborn, but not on the single servermachine. Th is time, it was set up to run in the cloud. The secondincarnation of the wildfi re app used cloud services to distributedemand across multiple servers. Additionally, the base structure forthis type of system was established and configured so that servermachines could be added or removed as needed. In other words, a scalable system for supporting high-capacitymapping applications was created. Th is system is now used for all Server MachineEsri disaster response and current events maps ( Th is article provides a brief, high-level view of the thought process File ArcGIS for Geodatabase Application Serverused in building the infrastructure for a high-capacity mapping ap-plication. Although it’s certainly not the only way to do this, it is atried-and-true method that allows incremental system design while  Spikes in demand from the Internet can quickly overload a singlemaintaining manual control over scalability. server machine setup.38 au Spring 2012
  2. 2. Developer’s Corner  With two identical single server setups, one can function as the staging setup and the other Demand as the production setup. Staging Production Server Machine Server Machine File ArcGIS for File ArcGIS for Geodatabase Application Server Geodatabase Application ServerThe Single Server Setup The Staging Server andA common method for deploying a mapping Production Server Setupapplication is to put everything—the map- Another way to prepare for server overload the GIS server. Creating a custom AMI canping application, the GIS server, and the situations is to make it easier and faster to take 20 minutes to an hour. Once it’s cre-data—on one machine. spin up a new instance of your mapping ated, you can generate new instances from Th is method is often used for internal ap- application if it goes down. To accom- the custom AMI very quickly—in approxi-plications because machine usage is either plish this, fi rst create a single server setup. mately 15 to 20 minutes.known or can be safely predicted. However, Conceptually, you can think of this as your Having a staging setup lets you test yourwhen the application is exposed to the staging setup. Next, make a copy of your application and make edits before youInternet, it is more difficult to predict the staging setup and use the copy as your pro- deploy the production setup. If you want tomaximum load on the server. duction setup. End users only access the re- add functionality to the application or make Sudden spikes in demand can quickly sources of the production setup. If demand changes to the data and services, you can dooverload the processing capability of the overloads your production server, you make this without disturbing the end users whoserver machine and cause it to crash. During a new copy from the staging setup and use it are using the production setup. When you’rethe resultant downtime, your customers to replace the crashed production setup. ready to deploy the new setup, just kill offlose access to the application, and you end On Amazon EC2, making a copy simply the production instance and replace it withup spending hours getting the server up and means starting a new instance from a the updated production instance.running, only to have it crash again as soon custom AMI. To create the custom AMI, However, this framework doesn’t help youas users discover its availability. complete the staging setup and save it as achieve the capacity needed to support large One way to combat spikes in demand is to an AMI. The custom AMI encapsulates your spikes in demand. To do that, you’ll need twouse a server machine with more processing entire setup—the application, the data, and load-balanced production instances.power. The problem with this solution is thatyou end up paying for a server that is un-derutilized most of the time. Th is scenario istrue whether you’re deploying your applica-tion on local infrastructure or in the cloud. Demand A load balancing service can be used to distributedemand across the two production setups. Load Balancer Staging Production Production Server Machine Server Machine Server Machine File ArcGIS for File ArcGIS for File ArcGIS forGeodatabase Application Server Geodatabase Application Server Geodatabase Application Server Spring 2012 au 39
  3. 3.  If demand begins to exceed the capacity Demandof the two production AMIs, anotherproduction AMI can be added to the loadbalancer. Load Balancer Staging Production Production Production Server Machine Server Machine Server Machine Server Machine File ArcGIS for File ArcGIS for File ArcGIS for File ArcGIS forGeodatabase Application Server Geodatabase Application Server Geodatabase Application Server Geodatabase Application ServerSetting Up the Staging Server Multiple Staging Servers In this case, you make custom AMIs ofand Load-Balanced, Multiple and Load-Balanced, Multiple the application server setup and the GISProduction Servers Production Servers Setup server setup, create two new instances ofIn this scenario, you use the custom AMI With most mapping applications, you’ll each, and load balance the two productioncreated from the staging setup to generate want to make improvements, add func- application AMIs and the two GIS servertwo identical production instances. Next, tionality, or change the look and feel of the AMIs. Th is scenario provides scalability toyou employ a load balancing service (called app on a regular basis. Changes to data and your application as well as your GIS serverElastic Load Balancing on Amazon EC2) services usually occur much less frequently. and allows you to work on either while re-that will distribute demand across the pro- You can take advantage of this phenomenon ducing the impact on your end users. Bothduction instances. and make your high-capacity system more the application and the GIS server setups If something goes wrong with one of the efficient if you put the application on a dif- can be scaled up and down depending onproduction instances, the application con- ferent server from the GIS server and data. where the loads are greatest.tinues to work through the remaining pro-duction instance. Th is gives you time to starta new instance of the production instanceand configure it with the load balancer. You can also add additional machines  Splitting the application and the GIS server onto their own server setups Demandto the load balancing service. If demand improves scalability as well as updatability.increases to a critical threshold—say,80 percent on both production servers—youcan start a new production instance from Load Balancerthe custom AMI and add it to the load bal-ancer. When demand subsides to normallevels, you can kill off the third production Staging Application Production Application Production Application Server Machine Server Machine Server Machineinstance. In this scenario, you are achieving a level Application Application Applicationof scalability that allows you to manuallyrespond to spikes in demand that may affectthe performance or viability of your map- Demandping application. While the load balancercan be used to monitor whether the applica-tion is responding properly, it cannot checkthe fine-grained functionality of ArcGIS Load Balancerservices. To more closely monitor your ser-vices, use ServiceMonitor, a script providedby Esri that checks each service periodi- Staging GIS Production GIS Production GIS Server Machine Server Machine Server Machinecally and makes sure it is responding. You Filecan download the ServiceMonitor script at Geodatabase ArcGIS for Server File Geodatabase ArcGIS for Server File Geodatabase ArcGIS for au Spring 2012
  4. 4. Developer’s CornerIncluding an EnterpriseGeodatabase Server for FeatureEditingSome mapping applications allow the end instances in different zones within the same instances exceeds tolerances, another pro-user to perform feature editing such as region, you can have them all behind the duction instance can be manually or auto-adding a point, line, or polygon to the map. same load balancer. matically added to the load balancer.To have consistency on the production serv- If you anticipate frequent changes to theers running in the load balancer, editing Conclusion application, you may find it useful to put themust be done in an ArcSDE database. The To support a high-capacity mapping appli- application server and GIS server on sepa-ArcSDE database should reside on a sepa- cation, you will need a scalable system of rate tiers of the system. Likewise, if your ap-rate instance from the production instances. identical setups, each with a GIS server; an plication includes the ability to add or editThese features are stored in an enterprise application; and, possibly, data server ma- map features, you should put the requiredgeodatabase. chines. If you want to maintain your system enterprise geodatabase instance on a sepa- Esri also provides an enterprise geodata- in a manner that minimally impacts your rate tier.base AMI. Data for editable feature services, end users, consider creating a staging AMI These concepts and suggestions are notis stored in an instance created from this that you can work on while the production the only way to use Amazon EC2 to bringAMI. After putting the data in the enterprise AMIs serve your end users. Ideally, there high capacity to your mapping application,geodatabase, creating the feature service, should be two load-balanced production in- but you may find that this model is a prac-and configuring the service with the ap- stances in the base structure of the system tical approach to maintaining fine-grainedplication, you make a custom AMI of the so that if one fails, the application remains control over how and when your systemstaging data server setup and launch new viable. If demand on both production scales.production instances. In this example, one staging data serverand only one production data server areused to support the application. If possi-ble, use two production data servers, eachwith a mirrored enterprise geodatabase for  If the mapping application includes a feature editing capability, the data for those features Demandredundancy. However, currently, this is not should reside in an enterprise geodatabase.a simple, out-of-the-box solution providedwith the Esri enterprise geodatabase AMI—it requires a strong understanding of enter- Load Balancerprise database management.Where to Go from Here Staging Application Production Application Production Application Server Machine Server Machine Server MachineAmazon offers an Auto Scaling service thatyou can configure with the load balancing Application Application Applicationservice to automatically spin up new produc-tion instances when load thresholds reach aspecified level. However, you may find that Demandmaintaining manual control over scaling ofyour production instances is preferable. Amazon’s cloud infrastructure and data Load Balancercenters are available in a number of regionsaround the world. Each region has multi-ple Availability Zones, designed to protect Staging GIS Production GIS Production GIS Server Machineyour instances from failure. Instances can Server Machine Server Machinebe spread across Availability Zones within File Geodatabase ArcGIS for Server File Geodatabase ArcGIS for Server File ArcGIS for Geodatabase Servera Region so that if one data center goesdown, the other instances will continue tofunction. The Amazon EC2 Service Level Staging Data Production DataAgreement commitment is to provide 99.95 Server Machine Server Machinepercent availability for each Amazon EC2 Enterprise Geodatabase Enterprise GeodatabaseRegion. As long as you create production Spring 2012 au 41