Presentation delivered at JavaOne 2010. Talks about how to use Java to build highly scalable and reliable applications on Windows Azure.

1. Building Highly Scalable Java Applications on Windows Azure David Chou david.chou@microsoft.com blogs.msdn.com/dachou

2. >Introduction Agenda Overview of Windows Azure Java How-to Architecting for Scale What’s Next

3. >Azure Overview What is Windows Azure? A cloud computing platform(as-a-service) on-demand application platform capabilities geo-distributed Microsoft data centers automated, model-driven services provisioning and management You manage code, data, content, policies, service models, etc. not servers (unless you want to) We manage the platform application containers and services, distributed storage systems service lifecycle, data replication and synchronization server operating system, patching, monitoring, management physical infrastructure, virtualization networking security “fabric controller” (automated, distributed service management system)

4. >Azure Overview How this may be interesting to you Not managing and interacting with server OS less work for you don’t have to care it is “Windows Server” (you can if you want to) but have to live with some limits and constraints Some level of control process isolation (runs inside your own VM/guest OS) service and data geo-location allocated capacity, scale on-demand full spectrum of application architectures and programming models You can run Java! plus PHP, Python, Ruby, MySQL, memcached, etc. and eventually anything that runs on Windows

5. > Azure Overview >Anatomy of a Windows Azure instance Compute – instance types: Web Role & Worker Role. Windows Azure applications are built with web role instances, worker role instances, or a combination of both. Storage – distributed storage systems that are highly consistent, reliable, and scalable. Anatomy of a Windows Azure instance HTTP/HTTPS Each instance runs on its own VM (virtual machine) and local transient storage; replicated as needed Guest VM Guest VM Guest VM Host VM Maintenance OS, Hardware-optimized hypervisor The Fabric Controller communicates with every server within the Fabric. It manages Windows Azure, monitors every application, decides where new applications should run – optimizing hardware utilization.

6. >Java How-To Java and Windows Azure Provide your JVM any version or flavor that runs on Windows Provide your code no programming constraints (e.g., whitelisting libraries, execution time limit, multi-threading, etc.) use existing frameworks use your preferred tools (Eclipse, emacs, etc.) File-based deployment no OS-level installation(conceptually extracting a tar/zip with run.bat) Windows Azure “Worker Role” sandbox standard user (non-admin privileges; “full trust” environment) native code execution (via launching sub-processes) service end points (behind VIPs and load balancers)

7. > Java How-To > Boot-strapping Some boot-strapping in C# Kick-off process in WorkerRole.run() get environment info (assigned end point ports, file locations) set up local storage (if needed; for configuration, temp files, etc.) configure diagnostics (Windows Server logging subsystem for monitoring) launch sub-process(es) to run executable (launch the JVM) Additional hooks (optional) Manage role lifecycle Handle dynamic configuration changes Free tools Visual Studio Express Windows Azure Tools for Visual Studio Windows Azure SDK

8. > Java How-To > Tomcat Running Tomcat in Windows Azure Service Instance listen port(x) Service Instance Worker Role Sub-Process Tomcat server.xml Catalina index.jsp new Process() RoleEntry Point bind port(x) get runtime info SQL Database JVM http://instance:x http://instance:y Service Bus Access Control http://app:80 Fabric Controller Load Balancer Table Storage Blob Storage Queue

9. > Java How-To > Jetty Running Jetty in Windows Azure Boot-strapping code in WorkerRole.run() Service end point(s) in ServiceDefinition.csdef string response = ""; try{ System.IO.StreamReadersr; string port = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["HttpIn"].IPEndpoint.Port.ToString(); stringroleRoot = Environment.GetEnvironmentVariable("RoleRoot"); stringjettyHome = roleRoot + @"pprootppetty7"; stringjreHome = roleRoot + @"pprootppre6"; Processproc = newProcess(); proc.StartInfo.UseShellExecute = false; proc.StartInfo.RedirectStandardOutput = true; proc.StartInfo.FileName = String.Format("quot;{0}binjava.exequot;", jreHome); proc.StartInfo.Arguments = String.Format("-Djetty.port={0} -Djetty.home=quot;{1}quot; -jar quot;{1}start.jarquot;", port, jettyHome); proc.EnableRaisingEvents = false; proc.Start(); sr = proc.StandardOutput; response = sr.ReadToEnd();} catch(Exception ex) { response = ex.Message; Trace.TraceError(response); } <Endpoints> <InputEndpointname="HttpIn"port="80"protocol="tcp" /></Endpoints>

10. > Java How-To > Limitations Current constraints Platform Dynamic networking <your app>.cloudapp.net no naked domain CNAME re-direct from custom domain sending traffic to loopback addresses not allowed and cannot open arbitrary ports No OS-level access Non-persistent local file system allocate local storage directory read-only: Windows directory, machine configuration files, service configuration files Available registry resources read-only: HKEY_CLASSES_ROOT, HKEY_LOCAL_MACHINE, HKEY_USERS, HKEY_CURRENT_CONFIG full access: HKEY_CURRENT_USER Java Sandboxed networking NIO (java.nio) not supported engine and host-level clustering JNDI, JMS, JMX, RMI, etc. need to configure networking Non-persistent local file system logging, configuration, etc. REST-based APIs to services Table Storage – schema-less (noSQL) Blob Storage – large files (<200GB block blobs; <1TB page blobs) Queues Service Bus Access Control

12. burst & overflow capacity

14. mobile/client connected services

16. distributed processing

17. distributed data

19. content transcoding

21. common data repositories

23. B2B & e-commerce

24. supply chain management

25. health & life sciences

27. > Architecting for Scale > Vertical Scaling Traditional scale-up architecture Common characteristics synchronous processes sequential units of work tight coupling stateful pessimistic concurrency clustering for HA vertical scaling units of work app server web data store app server web data store

28. > Architecting for Scale >Vertical Scaling Traditional scale-up architecture To scale, get bigger servers expensive has scaling limits inefficient use of resources app server web data store app server web

29. > Architecting for Scale >Vertical Scaling Traditional scale-up architecture When problems occur bigger failure impact data store app server web app server web

30. > Architecting for Scale >Vertical Scaling Traditional scale-up architecture When problems occur bigger failure impact more complex recovery app server web data store web

34. > Architecting for Scale > Horizontal scaling Scale-out architecture To scale, add more servers not bigger servers app server web data store app server web data store app server web data store app server web data store app server web data store app server web data store

35. > Architecting for Scale > Horizontal scaling Scale-out architecture When problems occur smaller failure impact higher perceived availability app server web data store app server web data store app server web data store app server web data store app server web data store app server web data store

36. > Architecting for Scale > Horizontal scaling Scale-out architecture When problems occur smaller failure impact higher perceived availability simpler recovery app server web data store app server web data store web app server data store web data store app server web data store app server web data store

37. > Architecting for Scale > Horizontal scaling Scale-out architecture + distributed computing parallel tasks Scalable performance at extreme scale asynchronous processes parallelization smaller footprint optimized resource usage reduced response time improved throughput app server web data store app server web data store web app server data store app server web data store perceived response time app server web data store app server web data store async tasks

38. > Architecting for Scale > Horizontal scaling Scale-out architecture + distributed computing When problems occur smaller units of work decoupling shields impact app server web data store app server web data store web app server data store app server web data store app server web data store app server web data store

39. > Architecting for Scale > Horizontal scaling Scale-out architecture + distributed computing When problems occur smaller units of work decoupling shields impact even simpler recovery app server web data store app server web data store web app server data store app server web data store app server web data store web data store

40. > Architecting for Scale >Cloud Architecture Patterns Live Journal (from Brad Fitzpatrick, then Founder at Live Journal, 2007) Web Frontend Apps & Services Partitioned Data Distributed Cache Distributed Storage

41. > Architecting for Scale >Cloud Architecture Patterns Flickr (from Cal Henderson, then Director of Engineering at Yahoo, 2007) Web Frontend Apps & Services Distributed Storage Distributed Cache Partitioned Data

42. > Architecting for Scale >Cloud Architecture Patterns SlideShare(from John Boutelle, CTO at Slideshare, 2008) Web Frontend Apps & Services Distributed Cache Partitioned Data Distributed Storage

43. > Architecting for Scale >Cloud Architecture Patterns Twitter (from John Adams, Ops Engineer at Twitter, 2010) Web Frontend Apps & Services Partitioned Data Queues Async Processes Distributed Cache Distributed Storage

44. > Architecting for Scale >Cloud Architecture Patterns Distributed Storage Facebook (from Jeff Rothschild, VP Technology at Facebook, 2009) 2010 stats (Source: http://www.facebook.com/press/info.php?statistics) People +500M active users 50% of active users log on in any given day people spend +700B minutes /month Activity on Facebook +900M objects that people interact with +30B pieces of content shared /month Global Reach +70 translations available on the site ~70% of users outside the US +300K users helped translate the site through the translations application Platform +1M developers from +180 countries +70% of users engage with applications /month +550K active applications +1M websites have integrated with Facebook Platform +150M people engage with Facebook on external websites /month Web Frontend Apps & Services Distributed Cache Parallel Processes Partitioned Data Async Processes

45. >Architecting for Scale Fundamental concepts Vertical scaling still works

46. >Architecting for Scale Fundamental concepts Horizontal scaling for cloud computing Small pieces, loosely coupled Distributed computing best practices asynchronous processes (event-driven design) parallelization idempotent operations (handle duplicity) de-normalized, partitioned data (sharding) shared nothing architecture optimistic concurrency fault-tolerance by redundancy and replication etc.

47. > Architecting for Scale >Fundamental Concepts Asynchronous processes & parallelization Defer work as late as possible return to user as quickly as possible event-driven design (instead of request-driven) Cloud computing friendly distributes work to more servers (divide & conquer) smaller resource usage/footprint smaller failure surface decouples process dependencies Windows Azure platform services Queue Service AppFabric Service Bus inter-node communication Worker Role Web Role Queues Service Bus Web Role Web Role Web Role Worker Role Worker Role Worker Role

48. > Architecting for Scale >Fundamental Concepts Partitioned data Shared nothing architecture transaction locality (partition based on an entity that is the “atomic” target of majority of transactional processing) loosened referential integrity (avoid distributed transactions across shard and entity boundaries) design for dynamic redistribution and growth of data (elasticity) Cloud computing friendly divide & conquer size growth with virtually no limits smaller failure surface Windows Azure platform services Table Storage Service SQL Azure read Web Role Queues Web Role Web Role Worker Role Relational Database Relational Database Relational Database Web Role write

49. > Architecting for Scale >Fundamental Concepts Idempotent operations Repeatable processes allow duplicates (additive) allow re-tries (overwrite) reject duplicates (optimistic locking) stateless design Cloud computing friendly resiliency Windows Azure platform services Queue Service AppFabric Service Bus Worker Role Service Bus Worker Role Worker Role

51. Single site, cluster database, LDAP, xFS file system, etc.

54. Distributed cache, DNS, etc.

55. Optimistic locking, expiration/leases, etc.“Towards Robust Distributed Systems”, Dr. Eric A. Brewer, UC Berkeley

56. > Architecting for Scale >Fundamental Concepts Hybrid architectures Scale-out (horizontal) BASE: Basically Available, Soft state, Eventually consistent focus on “commit” conservative (pessimistic) shared nothing favor extreme size e.g., user requests, data collection & processing, etc. Scale-up (vertical) ACID: Atomicity, Consistency, Isolation, Durability availability first; best effort aggressive (optimistic) transactional favor accuracy/consistency e.g., BI & analytics, financial processing, etc. Most distributed systems employ both approaches

57. Thank you! David Chou david.chou@microsoft.com blogs.msdn.com/dachou © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

58. > Introduction Cloud computing Characteristics On-demand self-service Broad network access Resource pooling Rapid elasticity Measured service Service models Software as a service Platform as a service Infrastructure as a service Deployment models Private cloud Community cloud Public cloud Hybrid cloud “Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models.” Source: The NIST Definition of Cloud Computing, Version 15, 2009.10.07, Peter Mell and Tim Grance http://csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.doc

59. Your Own Data Center Someone Else’s Data Center Use (services, information, etc.) Build (applications, data, etc.) Host (software, database, etc.)

60. Private Cloud Public Cloud Service Delivery Models Software(as-a-service) Platform(as-a-service) Infrastructure(as-a-service) Community Dedicated Hybrid Cloud Cloud Deployment Models

61. > Introduction Service delivery models (On-Premise) Infrastructure (as a Service) Platform (as a Service) Software (as a Service) You manage Applications Applications Applications Applications You manage Data Data Data Data Runtime Runtime Runtime Runtime Managed by vendor Middleware Middleware Middleware Middleware You manage Managed by vendor O/S O/S O/S O/S Managed by vendor Virtualization Virtualization Virtualization Virtualization Servers Servers Servers Servers Storage Storage Storage Storage Networking Networking Networking Networking

62. Globally Distributed Data Centers Quincy, WA Chicago, IL San Antonio, TX Dublin, Ireland Generation 4 DCs

63. Cloud Web Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Cloud Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data

64. Composite Services Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data

65. Cloud Agent Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data

66. B2B Integration Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data

67. Grid / Parallel Computing Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data

68. Hybrid Enterprise Application User Silverlight Application Web Browser Mobile Browser WPF Application ASP.NET (Web Role) Web Svc (Web Role) Jobs (Worker Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Private Cloud Public Services ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) ASP.NET (Web Role) Enterprise Application Application Service Enterprise Web Svc Data Service Table Storage Service Blob Storage Service Queue Service Enterprise Data Storage Service Identity Service Enterprise Identity Service Bus Access Control Service Workflow Service User Data Application Data Reference Data