Writing Scalable Software in Java
Upcoming SlideShare
Loading in...5
×
 

Writing Scalable Software in Java

on

  • 12,963 views

Writing Scalable Software in Java - From multi-core to grid-computing

Writing Scalable Software in Java - From multi-core to grid-computing

Statistics

Views

Total Views
12,963
Views on SlideShare
11,156
Embed Views
1,807

Actions

Likes
16
Downloads
442
Comments
4

18 Embeds 1,807

http://www.zonaj.org 1528
http://jug.pt 150
http://zonaj.org 36
http://www.linkedin.com 15
http://ptjug.wordpress.com 15
http://www.slideshare.net 12
http://107.170.176.40 11
http://lanyrd.com 11
http://feeds.feedburner.com 8
http://zonaj.dreamhosters.com 6
http://rubenbadaro.com 5
http://infoblogs.com.br 4
http://www.joanabotto.com 1
http://www.rubenbadaro.com 1
http://webcache.googleusercontent.com 1
https://www.linkedin.com 1
http://grumpy.junta.com.au 1
http://coderwall.com 1
More...

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Thanks Ruben.
    Are you sure you want to
    Your message goes here
    Processing…
  • Quite helpful for basic understanding..
    Are you sure you want to
    Your message goes here
    Processing…
  • Thanks Ruben Badaró
    Are you sure you want to
    Your message goes here
    Processing…
  • Very nice diagrams.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Writing Scalable Software in Java Writing Scalable Software in Java Presentation Transcript

  • Writing Scalable Software in Java From multi-core to grid-computing
  • Me • Ruben Badaró • Dev Expert at Changingworlds/Amdocs • PT.JUG Leader • http://www.zonaj.org
  • What this talk is not about • Sales pitch • Cloud Computing • Service Oriented Architectures • Java EE • How to write multi-threaded code
  • Summary • Define Performance and Scalability • Vertical Scalability - scaling up • Horizontal Scalability - scaling out • Q&A
  • Performance != Scalability
  • Performance Amount of useful work accomplished by a computer system compared to the time and resources used
  • Scalability Capability of a system to increase the amount of useful work as resources and load are added to the system
  • Scalability • A system that performs fast with 10 users might not do so with 1000 - it doesn’t scale • Designing for scalability always decreases performance
  • Linear Scalability Throughput Resources
  • Reality is sub-linear Throughput Resources
  • Amdahl’s Law
  • Scalability is about parallelizing • Parallel decomposition allows division of work • Parallelizing might mean more work • There’s almost always a part of serial computation
  • Vertical Scalability
  • Vertical Scalability Somewhat hard
  • Vertical Scalability Scale Up • Bigger, meaner machines - More cores (and more powerful) - More memory - Faster local storage • Limited - Technical constraints - Cost - big machines get exponentially expensive
  • Shared State • Need to use those cores • Java - shared-state concurrency - Mutable state protected with locks - Hard to get right - Most developers don’t have experience writing multithreaded code
  • This is how they look like public static synchronized SomeObject getInstance() { return instance; } public SomeObject doConcurrentThingy() { synchronized(this) { //... } return ..; }
  • Single vs Multi-threaded • Single-threaded - No scheduling cost - No synchronization cost • Multi-threaded - Context Switching (high cost) - Memory Synchronization (memory barriers) - Blocking
  • Lock Contention Little’s Law The average number of customers in a stable system is equal to their average arrival rate multiplied by their average time in the system
  • Reducing Contention • Reduce lock duration • Reduce frequency with which locks are requested (stripping) • Replace exclusive locks with other mechanisms - Concurrent Collections - ReadWriteLocks - Atomic Variables - Immutable Objects
  • Concurrent Collections • Use lock stripping • Includes putIfAbsent() and replace() methods • ConcurrentHashMap has 16 separate locks by default • Don’t reinvent the wheel
  • ReadWriteLocks • Pair of locks • Read lock can be held by multiple threads if there are no writers • Write lock is exclusive • Good improvements if object as fewer writers
  • Atomic Variables • Allow to make check-update type of operations atomically • Without locks - use low-level CPU instructions • It’s volatile on steroids (visibility + atomicity)
  • Immutable Objects • Immutability makes concurrency simple - thread- safety guaranteed • An immutable object is: - final - fields are final and private - Constructor constructs the object completely - No state changing methods - Copy internal mutable objects when receiving or returning
  • JVM issues • Caching is useful - storing stuff in memory • Larger JVM heap size means longer garbage collection times • Not acceptable to have long pauses • Solutions - Maximum size for heap 2GB/4GB - Multiple JVMs per machine - Better garbage collectors: G1 might help
  • Scaling Up: Other Approaches • Change the paradigm - Actors (Erlang and Scala) - Dataflow programming (GParallelizer) - Software Transactional Memory (Pastrami) - Functional languages, such as Clojure
  • Scaling Up: Other Approaches • Dedicated JVM-friendly hardware - Azul Systems is amazing - Hundreds of cores - Enormous heap sizes with negligible gc pauses - HTM included - Built-in lock elision mechanism
  • Horizontal Scalability
  • Horizontal Scalability The hard part
  • Horizontal Scalability Scale Out • Big machines are expensive - 1 x 32 core normally much more expensive than 4 x 8 core • Increase throughput by adding more machines • Distributed Systems research revisited - not new
  • Requirements • Scalability • Availability • Reliability • Performance
  • Typical Server Architecture
  • ... # of users increases
  • ... and increases
  • ... too much load
  • ... and we loose availability
  • ... so we add servers
  • ... and a load balancer
  • ... and another one rides the bus
  • ... we create a DB cluster
  • ... and we cache wherever we can Cache Cache
  • Challenges • How do we route requests to servers? • How do distribute data between servers? • How do we handle failures? • How do we keep our cache consistent? • How do we handle load peaks?
  • Technique #1: Partitioning A F K P U ... ... ... ... ... E J O T Z Users
  • Technique #1: Partitioning • Each server handles a subset of data • Improves scalability by parallelizing • Requires predictable routing • Introduces problems with locality • Move work to where the data is!
  • Technique #2: Replication Active Backup
  • Technique #2: Replication • Keep copies of data/state in multiple servers • Used for fail-over - increases availability • Requires more cold hardware • Overhead of replicating might reduce performance
  • Technique #3: Messaging
  • Technique #3: Messaging • Use message passing, queues and pub/sub models - JMS • Improves reliability easily • Helps deal with peaks - The queue keeps filling - If it gets too big, extra requests are rejected
  • Solution #1: De- normalize DB • Faster queries • Additional work to generate tables • Less space efficiency • Harder to maintain consistency
  • Solution #2: Non-SQL Database • Why not remove the relational part altogether • Bad for complex queries • Berkeley DB is a prime example
  • Solution #3: Distributed Key/Value Stores • Highly scalable - used in the largest websites in the world, based on Amazon’s Dynamo and Google’s BigTable • Mostly open source • Partitioned • Replicated • Versioned • No SPOF • Voldemort (LinkedIn), Cassandra (Facebook) and HBase are written in Java
  • Solution #4: MapReduce Map...
  • Solution #4: MapReduce Map...
  • Solution #4: MapReduce Divide Work Map...
  • Solution #4: MapReduce Divide Work Map...
  • Solution #4: MapReduce Divide Work Map...
  • Solution #4: MapReduce Map...
  • Solution #4: MapReduce Compute Map...
  • Solution #4: MapReduce Return and aggregate Reduce...
  • Solution #4: MapReduce Return and aggregate Reduce...
  • Solution #4: MapReduce Return and aggregate Reduce...
  • Solution #4: MapReduce • Google’s algorithm to split work, process it and reduce to an answer • Used for offline processing of large amounts of data • Hadoop is used everywhere! Other options such as GridGain exist
  • Solution #5: Data Grid • Data (and computations) • In-memory - low response times • Database back-end (SQL or not) • Partitioned - operations on data executed in specific partition • Replicated - handles failover automatically • Transactional
  • Solution #5: Data Grid • It’s a distributed cache + computational engine • Can be used as a cache with JPA and the like • Oracle Coherence is very good. • Terracotta, Gridgain, Gemfire, Gigaspaces, Velocity (Microsoft) and Websphere extreme scale (IBM)
  • Retrospective • You need to scale up and out • Write code thinking of hundreds of cores • Relational might not be the way to go • Cache whenever you can • Be aware of data locality
  • Q &A Thanks for listening! Ruben Badaró http://www.zonaj.org