Scalable Resilient Web Services In .Net


Published on

Web Architecture with AppFabric and WCF services in a hosting environment.

  • Be the first to comment

  • Be the first to like this

Scalable Resilient Web Services In .Net

  1. 1. Designing and Delivering Scalable and Resilient Web ServicesRon JacobsSr. Technical Evangelist, Microsoft
  2. 2. AgendaThe ProblemA Possible SolutionNew PossibilitiesWhat to do next
  3. 3. Simple Do the simplest thingthat will possibly work
  4. 4. ScalabilityAble to support therequired quality ofservice as the systemload increases -Wiktionary
  5. 5. Typical Web ArchitectureUsers • Need to get routed to same machine (i.e. sticky sessions)Web • Each machine round trips for data • Data that is cached is storedTier in the memory of one serverData • CPU and disk can get saturated due to trafficTier • Service access is slow
  6. 6. Web Explosion Web Site’s tooWhere did slow!! myshopping IIS/ASP.NET IIS/ASP.NET IIS/ASP.NETcart go? Application Application Application Servers are crashing Database Services Database is hot!! are slow
  7. 7. AgendaThe ProblemA Possible SolutionNew PossibilitiesWhat to do next
  8. 8. Data Near ProcessingCache Cache Smart Client Browser CacheCache Cache Database
  9. 9. Good but…• Cache is scoped to machine / process – Machines die – Processes recycle• Cache memory is limited
  10. 10. What if?• You could have as much cache as you wanted?• You could share a giant cache across servers, services and even clients?• What if this was something you could simply add to the platform for 1free? 1Some features may require certain editions of Windows Server
  11. 11. Windows Server AppFabric AppFabric CACHING WORKFLOW HOSTING MONITORING SERVICE HOSTINGHIGH AVAILABILITY SCALE OUT MANAGEMENT AppFabric Cache – Formerly known as Code Name “Velocity”
  12. 12. What is AppFabric Caching?• An explicit, distributed, in-memory application cache for all kinds of data Caching clients can be across machines or processes Clients Access the Cache as if it Unified Cache View was a large single cache Cache Layer distributes data across the various cache nodes
  13. 13. AppFabric Cache Users • No need for sticky sessions Web • Cached data available to all servers TierCaching • Multiple machines means scale and high-availability Tier • Lots of cache memory Data • Reduces load on database • Duplicate Service Calls Tier eliminated
  14. 14. Data Distribution - Partitioned Cache ASP.Net App Caching Client ASP.Net App Caching Client … ASP.Net App Caching Client Web Tier A B C D E F G H I Cache Service Cache Service Cache Service Cache D I C G A F BH E TierScale on Data Size - More machines => More memory to cacheScale on Cache Throughput - More machines => keys distributed acrossmore machines => better throughput
  15. 15. Scale Test Output Load Throughput 1 Cache Server As load increases, throughput fails to scale latency increases LatencyCaching Tier
  16. 16. Add Second Cache Server Load Load Max Throughput Throughput increases Latency decreases LatencyCaching Tier
  17. 17. Add Third Cache Server Load Throughput LatencyCaching Tier
  18. 18. Associated Press• Caches metadata and news• Serves 16 million hits per day• Increased the amount of cached data 6 times.
  19. 19. System.Web.Cache
  20. 20. AppFabric DataCache
  21. 21. Deployment Update Web.config Copy Client DLLs<hosts> <host name="BL1CDB8083714“ … cachePort="22233" cacheHostName="DistributedCacheService"/> .NET 3.5 SP1 OR .NET 4 Application Caching Access Layer Application Caching Access Layer Application Caching Access Layer ….. </hosts><localCache isEnabled=“true" ../><security … /> Caching Service Caching Service Caching Service Install Configure .NET 4 AppFabric Configuration Store
  22. 22. Usage Pattern – Cache Aside (Explicit Caching)// Read from CacheToy toyObj = (Toy) Application catalog.Get("toy-101"); Caching Access Layer// If Not present in the cacheif (toyObj == null) Caching Service{ // Read from backend.. toyObj = ReadFromDatabase(); // Populate Cache catalog.Put("toy-101", toyObj); Database return toyObj;}
  23. 23. Administration• PowerShell cmdlets are used to administer the cache cluster• Rich set of cmdlets for – Cache cluster management – Cache creation and monitoring
  24. 24. Hello AppFabric Cache
  25. 25. Using PowerShellRemember – PowerShell can also be called from .NET Code!
  26. 26. AppFabric Cache Codeplex Tool
  27. 27. Security• Domain Based Security Option – Domain Account / Local Account based Authentication – Only authorized servers can join the cluster – Only authorized clients can connect to the cluster grant-cacheallowedclientaccount RedDomainMachine1$ grant-cacheallowedclientaccount RedDomainJohn• Transport Level Security – Turn on/off Signing or Encryption• Can turn off Cache Security – Use Firewalls, IPSec, VLANs to protect cache
  28. 28. AppFabric Caching Logical AppFabric AppFabric AppFabric Hierarchy AppFabric Caching Caching Caching Caching Service Service Service Service Named Cache : Product Catalog Named Cache : Electronics Inventory Regions Key Payload Tags Region A 121 xxxx “Toy” “Child” 123 yyyy “Toy” “Chair”..Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects• Host • Cache Item – Physical processes hosting AppFabric – Key, Payload (Object ), Tags, TTL, Caching instance. Timestamps, Version• Named Caches • Regions – Can span across machines – Physically co-located Container of – Defined in the configuration file Cache Items – May be implicit or explicitly created
  29. 29. AppFabric Caching API // Create instance of cachefactory (reads appconfig) DataCacheFactory fac = new DataCacheFactory(); // Get a named cache from the factory DataCache catalog = fac.GetCache("catalogcache");// Simple Get/Putcatalog.Put("toy-101", new Toy("Puzzle", .,.));// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");// Region based Get/Putcatalog.CreateRegion("toyRegion");// Both toy and toyparts are put in the same regioncatalog.Put("toy-101", new Toy( .,.), “toyRegion”);Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");
  30. 30. Access APIs – Tagging ItemsTag hotItem = new Tag("hotItem");catalog.Put("toy-101", new Toy("Puzzle"), new Tag[]{hotItem}, “toyRegion”);catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);// From the same or a different clientList<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);
  31. 31. Types of DataReference Activity ResourcePrimary Read Only Read-Write Not shared Read-Write, SharedCatalog Data Shopping Cart Auction Data/Seat Assignment Grocery Shop Web Tier Shopping Cart Grocery Inventory Distributed Cache Grocery Catalog
  32. 32. Reference Data – Performance• Catalog data doesn’t change often• Unnecessary network cost to access from different machines• Solution – Local Cache Application Application Get(K2) Get(K2) Put(K2, v3) AppFabric Caching Client AppFabric Caching Client Local Cache K2, V2 Routing Table Routing Table Cache1 Cache2 Cache3 Primary for K1,V1 Primary for K2,V2 Primary for K3,V3 K1, V1 K2, V2 V3 K3, V3
  33. 33. Reference Data – Bulk Get• Bulk Fetch from region – 200-300k ops per second – Fewer network callsCatalog.BulkGet( new List<string>(){“toy-101”, “toy-102”} , “toyRegion”);
  34. 34. Activity Data – Session Integration Load Balance Requests No more sticky routing<sessionState mode="Custom“ customProvider="SessionStoreProvider"><providers> Drop in AppFabric <add name="SessionStoreProvider" type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider, Application Application … Application Caching SessionStoreProvider ClientLibrary“ Access Layer Caching Caching Access Layer Caching Access Layer cacheName="<YourNamedCache>"/></providers> Session State</sessionState> stored in AppFabricAllows session state to be shared amongst multiple applications Caching Scale your Session Store Caching Service Caching Service Cache Service Dynamically Highly Available
  35. 35. Activity Data - Availability Application (K2, V2) Application PUT Get(K2) AppFabric Caching Client Routing Table AppFabric Caching Client Routing TableCache1 Cache2 Cache3Primary for Primary for Primary for K2, V2 Replication K1, V1 K2, V2 Agent K3, V3 K2, V2Secondary for Secondary for Secondary for K2, V2 K3, V3 K1, V1
  36. 36. Resource Data - Optimistic Locking • GetCacheItem returns a version object • Every update to an object internally increments its version • Supply the version obtained along with the Put/Remove • Put/Remove will succeed only if the passed in version matches the version in the cache Version Based UpdateTime Client1 Client2 (Different Thread or process) Two clients access the sameT0 CacheItem item = CacheItem item = item catalog.GetCacheItem(“PlayerRegion”, catalog.GetCacheItem(“PlayerRegion”, ”Zune”); ”Zune”); Both update the itemT1 ((ZuneObject)item.Object).inventory --; ((ZuneObject)item.Object).inventory--;T2 catalog.Put(“PlayerRegion”, “Zune”, Second Client gets in first; item.Object, item.Version); put succeeds because item version matches; atomicallyT3 catalog.Put(“PlayerRegion”, “Zune”, increments the version item.Object, item.Version); // Version mismatch First client tries put; // Client must retry again Fails because the versions don’t match
  37. 37. Resource Data - Pessimistic Locking Client1: Client2: Client3: GetAndLock ("k1") GetAndLock ("k1") Get ("k1") GetAndLock gets lock handle Regular Get Other GetAndLock succeeds on same item fails K1• Take locks on non-existent keys – Allows you to co-ordinate calls for data
  38. 38. Data RaceGET GET GET
  39. 39. Lock Non-Existent KeyGET/LOCK GET/LOCK GET/LOCK Caching Service Cache Service Cache Service
  40. 40. Cache Service CALLCache Service WAITCaching Service WAIT Composite Race
  41. 41. PUTCache Service UNLOCKCache Service GETCaching Service GET Composite Race
  42. 42. Resource/Activity Data – Tracking Changes • Cache Event notifications • Register on any client to notify changes • Batched NotificationsDataCache.RegisterCacheLevelCallback( int filter, DataCacheChangeCallback delegate);DataCache.RegisterRegionLevelCallback( String region, int filter, DataCacheChangeCallback delegate);DataCache.RegisterKeyLevelCallback( String region, String key, int filter, DataCacheChangeCallback delegate);
  43. 43. Scalable Notifications Register Notification for Key “K3"Call Delegate ApplicationStore Last LSN Map Keys AppFabric Caching Client to Partition Routing Table Partition: P2 Poll Last LSN: 19 Required Nodes Nodes Return List of Changes LSN Order Cache1 Cache2 Cache3 Primary for Primary for Primary for Change Log Change Log K1, V1 Change Log K2, V2 (Partition P2) K3, V3 Partition P1 33 Add K1 1 Add K2 18 Del K32 34 Del K22 2 Del K32 19 Del K43
  44. 44. AgendaThe ProblemA Possible SolutionNew PossibilitiesWhat to do next
  45. 45. Pre-FetchHospitalData Center
  46. 46. Pre-FetchRemote Clinic Hospital Slow!! Data Center WAN
  47. 47. Pre-FetchRemote Clinic Hospital Cache Service Data Center WAN
  48. 48. AgendaThe ProblemA Possible SolutionNew PossibilitiesWhat to do next
  49. 49. Web Platform Installer
  50. 50. Select Enterprise
  51. 51. Install AppFabric
  52. 52. Install AppFabric
  53. 53.
  54. 54. AppFabric on MSDN