Shopzilla On Concurrency

Shopzilla on Concurrency Concurrency as Shopzilla's performance building block Rod Barlow, Architect 3/2/2010

Agenda
Introduce Shopzilla
History of Java Concurrency
Java 5 Concurrency Features
Concurrency in Frameworks
Concurrency @ Shopzilla
Future

Shopzilla, Inc. - Online Shopping Network 100M impressions/day 20-29M UV’s per Month 8,000+ searches per second 100M+ Products

Concurrent code pre Java 1.5 was difficult and error prone.
Unintended side effects
Doug Lea's concurrent package (circa 1998)
Java users need to write reliable multi-threaded software!
JSR-133 Java Memory Model (threads, locks, volatiles, ...)
JSR-166 Concurrency Utilities
Expert groups consisting of Bloch, Goetz, and Lea
History of Java Concurrency

Where is Concurrent Code?
Concurrent code is in our application containers
org.apache.tomcat.util.threads.ThreadPool
"A thread pool that is trying to copy the apache process management. Should we remove this in favor of Doug Lea's thread package?"
http://tomcat.apache.org/tomcat-6.0-doc/api/org/apache/tomcat/util/threads/ThreadPool.html

Where is Concurrent Code?
Concurrent code is in the frameworks we all use
Concurrent code is increasingly in our application code
Concurrent code @ Shopzilla!

Immutability
Immutability = a class whose instances can't be modified
Eg; String, boxed primitives, BigDecimal, BigInteger
Joshua Bloch's Effective Java sets forth guidelines
Eliminate mutators
Eliminate extensibility
All fields final
Exclusivity for mutable components
Further, Bloch's Effective Java reminds us
Immutable objects are inherently thread safe.
Immutable objects require no synchronization
Immutable objects can be shared freely

Immutability - Guava
Formerly Google Collections, now Guava - http://code.google.com/p/guava-libraries/
http://www.infoq.com/news/2010/01/google_collections_10
“ Immutability” guarantees no other actor in the system can change the state of the collection
“ Unmodifiable” (java.util.Collections.unmodifiable... factory methods) only guarantees that the client of the collection – the user – can never change the collection
An instance of ImmutableList contains its own private data and will never change

Atomic References
Immune to deadlock and other liveness issues
Offer non-blocking synchronization of single variables
Offer lower scheduling overhead than traditional synchronization techniques
Immune to deadlock and other liveness issues
Effectively volatile variables with extra features
Modern hardware support through compare-and-swap processor instructions

Atomic References – Unique ID
We needed an unique ID value
Unique across multiple data-centers, and silos
Configuration elements prime the singleton IdGenerator for distributed uniqueness
A portion is based on a time value, e.g.: the seconds since the start of the month
An additional portion provides uniqueness within a single JVM, using an Atomic Reference

Atomic References – Parent Node
AtomicReference's compareAndSet()
Used for visibility
Used to enforce data integrity constraints
Note ImmutableList

Atomic References – Takeaways
Volatiles suffice where atomic check-then-act is overkill
Some atomic nonblocking algorithms involve looping for a failed compareAndSet()
During high thread contention this could actually mean inefficiency
Most real-world threads have far more to do than mere lock contention though

Blocking Queues
Acts as a thread-safe implementation of a producer / consumer pattern
JMS queue, though not distributed
Insertion blocks until there is space available (for bounded queues)

Blocking Queues – Data Publish
A very large (50GB) flat-file
Consumers send data to a remote grid cache
Multiple queue consumers increased throughput

Locks – ReadWriteLock
Allows for multiple concurrent read locks
No new read locks once a write lock is placed
Write lock blocks until read locks complete
Lock modes; non-fair (default), fair
Reentrancy
Downgrading

Distributed Cached Data Snapshot
Multiple clients -> Multiple HTTP requests
Multiple load balanced JVMs
Distributed data grid cache
Need to stream a data snapshot to n clients

Coherence partitioned cache supports cluster wide key-based lock
Locked objects can still be read by other cluster threads without a lock
Locks are unaffected by server failure (and will failover to a backup server.) Locks are immediately released when the lock owner (client) fails.
Lock timeouts (-1, 0, 1+)

Hazelcast
http://www.hazelcast.com/
Open source clustering and highly scalable data distribution platform for Java
Distributed data structures
Queue / Topic
Map, MultiMap, Set, List
Lock
Effectively distributed java.util.concurrent
Uses TCP/IP
Cluster wide ID generators
Distributed executor services

Apache Zookeeper
http://hadoop.apache.org/zookeeper/
Terracotta
http://www.terracotta.org/

Distributed competition for the publishing privilege
Computation of completeness
Communicate completeness
Other threads in other JVMs happily polling for State.DONE

n number of clients
n number of HTTP requests across 6 load balanced Tomcat JVMs
Threads failing to acquire lock immediately start shipping data
What about the thread obtaining the lock?

Hibernate Core 3.5.0
CountDownLatch
Need a thread to wait until some number of events have occurred
Constructed with the count of the # events which must occur before release
Callable, ExecutorService, ReentrantLock, AtomicReference

Spring Framework 3.0.1
TaskExecutor
Spring 2.0 supported Java 1.4
TaskExecutor did not implement Executor
In Spring 3.0 TaskExecutor extends Executor
TaskExecutor sees wide use within Spring framework
Quartz
Message Driven POJO
Spring Enterprise Recipies – Josh Long, Gar Mak

Shopzilla's Website Concurrency
Needed sub 650ms server side response time
Simplify the layers
Decompose architecture
Functionally separate, individually testable, loosely coupled web-services
Define SLAs

How to invoke 30+ web-services and ship a page in <650ms?
Concurrency! Our pages today ship within 250ms

Shopzilla's Website Concurrency Pods & Service Calls

Started simple
Implement only the concurrency features required
Concurrency isolated to pods
Pods responsible for fetching data
We're using simple building blocks
Incremental implementation based solely on requirements
Haven't seen deadlocks

Thread longevity configured at the HTTP connection level
HTTPClient connectionTimeout
Spring wired HTTPClient implementation
Ability to add a pod to a controller

FuturePodResult implements the PodResult interface
Abstracts the details of the future
PodCallable types the pod and command

Need to execute pods
Configurable ExecutorService
Backed with a queue
Naming of threads proved useful in initial testing (JMX)

Once the concept was proven, interesting feature requests materializing
Product Review pod
Distilled, 2 pods needed to share a single result
Added ServiceInvocation concept

Pods now have access to a Service Invocation Map
get() blocks on the result of the service invocation
A single service invocation result can be shared between two pods

Now we were sharing results, we were done, right?
Product Review information was now required in-line in the product pod
Still needed the special Product Review pod too!
Dependent Service Invocations

Service Invocations can now depend on results of others
Dependent Callable is configured with two callbacks;
A callback whose result is blocked for
A callback which is invoked once the blocking result arrives

Future
More use of distributed data structures
Spring 3.0
@Async
@Scheduled
JSR-315 Servlets 3.0
AsyncContext
More parallelism

Resources, Books
Java Concurrency in Practice (Goetz)
Effective Java (Bloch)
Spring Enterprise Recipes (Long, Mak)
http://jcp.org/en/jsr/detail?id=133
http://jcp.org/en/jsr/detail?id=166
Spring 3.0

For more info…go to:
http://tech.shopzilla.com
http://rodneybarlow.org

Shopzilla On Concurrency

by Rod Barlow on Mar 05, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 4

Statistics

Shopzilla On Concurrency — Presentation Transcript