Recommended
More Related Content
What's hot
What's hot (20)
Similar to Java Colombo: Developing Highly Scalable Apps
Similar to Java Colombo: Developing Highly Scalable Apps (20)
More from Afkham Azeez
More from Afkham Azeez (20)
Recently uploaded
Recently uploaded (20)
Java Colombo: Developing Highly Scalable Apps
- 1. Developing highly scalable applications By Afkham Azeez twitter.com/afkham_azeez blog.afkham.org
- 2. About Me • PMC member Apache Axis, Apache Stratos, Committer Synapse & Web Services • Member, Apache Software Foundation • Co-author, Axis2 Web Services • Director of Architecture, WSO2 Inc • Electronics, Arduino, 4x4 hobbyist • Blog: http://blog.afkham.org • Twitter: afkham_azeez 2
- 3. Agenda • Some core concepts • Vertical/horizontal scaling techniques • Capacity planning • Show me the code! • Q&A, Discussion
- 4. Scalability "The ability of the of a system to continue to operate correctly even when it is scaled to a larger size”
- 7. High Availability A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly. High Availability: The proportion of time during which the service is accessible with reasonable response times should be close to 100%.
- 8. How to decide required scale (capacity) & availability? • Average throughput (TPS) • Max throughput (TPS) • Monetary value of a transaction • Average loss & max loss per second of downtime • Decide on how much to invest based on cost vs. benefit tradeoff 8
- 9. Vertical Scaling • Get the maximum out of each allocated JVM or resource • Increase CPU size • Increase memory
- 11. Load Balancing • Load balancing algorithms • Round robin • Weighted • Response based • Health check • Failover-only
- 13. Clustering for availability Group Communication Channel/State replication 13
- 18. Thread Pooling
- 19. ThreadLocal
- 21. ThreadPooling + ThreadLocal CAUTION: Make sure that ThreadLocal variables are reset prior to Threads being returned to the pool
- 22. Object Pooling • Some objects would be expensive to create • e.g. Connection objects • Reuse objects • State stored in those objects need to be cleared before being returned to the pool
- 23. Lazy Loading 1. Lazy Initialization A null field indicates that there is no data. When the value is requested, a null check is performed to see if the actual data needs to be loaded. If(this.value== null) this.value = loadValue(); return this.value;
- 24. 2. Virtual Proxy Lazy Loading The virtual proxy implements the same interface as the real object, and when called for the very first time, it loads the real object & delegates to that object. class VirtualProxy implements Interface { private RealObject real; private RealObject getReal(){ if(real == null) real = new RealObject (); return real; } public void justDoIt(){ getReal().justDoIt() } } class RealObject implements Interface { public void justDoIt(){ System.out.println(“DONE!”); } }
- 25. Lazy Loading 3. Value Holder A value holder is an object with a getValue method, which the clients will invoke in order to obtain a reference to the real object. Note that the method may not necessarily be named getValue. private ValueHolder<Widget> valueHolder; public Widget getWidget { return valueHolder.getValue(); }
- 26. Lazy Loading 4. Ghost The real object without any data. The data is loaded as and when required. UUID id = UUID.getUuid(); String name; Connection dbConn; … public Connection getDbConnection(){ If(dbConn == null) dbConn = loadDatabaseConnection(); }
- 27. Lazy Loading - performance
- 28. Asynchrony • Callback • Dual channel • Queing
- 31. Capacity Planning Throughput = number of completed requests / time to complete the requests No. of servers = (projected max load * 1.3) / max throughput of one server max throughput of one server = max throughput of that server for the slowest scenario in the set of use cases
- 32. Static membership • Predefined members • Other (non-predefined) nodes cannot join Static group M1 M2 M3 N M4 32
- 33. Dynamic membership • No predefined members • Nodes can join & leave Dynamic group M1 M2 N Join M3 M4 33
- 34. Hybrid membership • Some predefined (well-known) members, and some dynamic members • Nodes can join & leave • Membership revolves around the static members Hybrid group Dynamic members M6 M5 M7 N Static members M1 M2 M3 Join (IP, Port) M4 34
- 35. Multicast based membership management M4 N M1 Join (IP, Port) M2 M3 35
- 36. Well-known Address (WKA) based membership management Hybrid group Dynamic members Static members M6 N WK1 M5 WK2 Notify M7 Join (IP, Port) WK3 WK4 36
- 37. Multicast vs. WKA Multicast WKA All nodes should be in the same subnet Nodes can be in different networks All nodes should be in the same multicast domain No multicasting requirement Multicasting should not be blocked No fixed IP addresses or hosts required At least one well-known IP address or host required Failure of any member does not affect membership discovery New members can join with some WKA nodes down, but not if all WKA nodes are down Does not work on IaaSs such as Amazon EC2 IaaS-friendly Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure 37
- 38. Auto-scaling • Scale-out when load increases • Scale-in when load decreases • Always use the optimum amount of resources • Try out • AWS ELB • Apache Stratos Load Balancer • WSO2 ELB
- 39. Auto-scaling – steady load
- 40. Auto-scaling – load increasing
- 41. Auto-scaling – load increasing
- 42. Auto-scaling – steady load
- 43. Auto-scaling – decreased load
- 44. Auto-scaling – decreased load
- 45. Auto-scaling – steady load
- 46. Autoscaling - Analysis & Results
- 47. Autoscaling - Analysis & Results
- 51. Active cluster, multiple LB HIGHEST Cost LOWEST Availability keepalived Active Active Active 51
- 52. Multi-zone or multi-datacenter Deployment HIGHEST Cloud Controller Zone 1 Cost LOWEST Availability Zone 2 Region X 52
- 53. Multi-region deployment HIGHEST Zone 1 Zone 2 Region X Zone 2 Cost LOWEST Availability Zone 1 Region Y 53
- 54. Multiple IaaS (hybrid) Deployment HIGHEST Zone 1 Private cloud (data center) Zone 2 Zone 1 Zone 2 Amazon EC2 Cost LOWEST Availability Zone 1 Zone 2 Rackspace Cloud 54
- 55. Multi-region Multi-IaaS Multi-zone Multi-node active cluster - Single zone Primary-Secondary, with multiple LBs Primary-Secondary, single LB Single Node Cost of Availability 55
- 56. Hazelcast Clustering and highly scalable data distribution platform for Java, which is: Lightning-fast; thousands of operations/sec. Fail-safe; no losing data after crashes. Dynamically scales as new servers added. Open source
- 57. Hazelcast – In-memory data grid
- 58. Hazelcast – Management Center
- 60. JSR 107 – JCache (javax.cache.*) CacheManager cacheManager = Caching.getCacheManagerFactory().getCacheManager(“manger"); Cache<String, Integer> cache = cacheManager.getCache("sampleCache"); int value1 = 9876; cache.put(key, value1); int value = cache.get(key).intValue()
- 61. JSR 107 – JCache (javax.cache.*) CacheManager cacheManager Caching.getCacheManagerFactory().getCacheManager("test"); String cacheName = "cacheXXX"; Cache<String,Integer> cache = cacheManager.<String, Integer>createCacheBuilder(cacheName). setExpiry(CacheConfiguration.ExpiryType.MODIFIED, new CacheConfiguration.Duration(TimeUnit.SECONDS, 10)). setStoreByValue(false).build(); int value = 9876; cache.put(key, value); assertEquals(cache.get(key).intValue(), value);
- 63. DISCUSSION
- 64. Thanks!