The document discusses the importance of concurrency in improving Shopzilla's performance, highlighting the evolution of Java concurrency features and their application within the company's architecture. It outlines the use of immutability, atomic references, and distributed data structures to enhance thread safety and efficiency in processing numerous web service requests. Future efforts are focused on expanding the use of concurrency and implementing distributed systems to further optimize response times.

1. Shopzilla on Concurrency Concurrency as Shopzilla's performance building block Rod Barlow, Architect 3/2/2010

2. Agenda Introduce Shopzilla History of Java Concurrency Java 5 Concurrency Features Concurrency in Frameworks Concurrency @ Shopzilla Future

3. Shopzilla, Inc. - Online Shopping Network 100M impressions/day 20-29M UV’s per Month 8,000+ searches per second 100M+ Products

4. Concurrent code pre Java 1.5 was difficult and error prone. Unintended side effects Doug Lea's concurrent package (circa 1998) Java users need to write reliable multi-threaded software! JSR-133 Java Memory Model (threads, locks, volatiles, ...) JSR-166 Concurrency Utilities Expert groups consisting of Bloch, Goetz, and Lea History of Java Concurrency

5. Where is Concurrent Code? Concurrent code is in our application containers org.apache.tomcat.util.threads.ThreadPool "A thread pool that is trying to copy the apache process management. Should we remove this in favor of Doug Lea's thread package?" http://tomcat.apache.org/tomcat-6.0-doc/api/org/apache/tomcat/util/threads/ThreadPool.html

6. Where is Concurrent Code? Concurrent code is in the frameworks we all use Concurrent code is increasingly in our application code Concurrent code @ Shopzilla!

7. Immutability Immutability = a class whose instances can't be modified Eg; String, boxed primitives, BigDecimal, BigInteger Joshua Bloch's Effective Java sets forth guidelines Eliminate mutators Eliminate extensibility All fields final Exclusivity for mutable components Further, Bloch's Effective Java reminds us Immutable objects are inherently thread safe. Immutable objects require no synchronization Immutable objects can be shared freely

8. Immutability - Guava Formerly Google Collections, now Guava - http://code.google.com/p/guava-libraries/ http://www.infoq.com/news/2010/01/google_collections_10 “ Immutability” guarantees no other actor in the system can change the state of the collection “ Unmodifiable” (java.util.Collections.unmodifiable... factory methods) only guarantees that the client of the collection – the user – can never change the collection An instance of ImmutableList contains its own private data and will never change

9. Atomic References Immune to deadlock and other liveness issues Offer non-blocking synchronization of single variables Offer lower scheduling overhead than traditional synchronization techniques Immune to deadlock and other liveness issues Effectively volatile variables with extra features Modern hardware support through compare-and-swap processor instructions

10. Atomic References – Unique ID We needed an unique ID value Unique across multiple data-centers, and silos Configuration elements prime the singleton IdGenerator for distributed uniqueness A portion is based on a time value, e.g.: the seconds since the start of the month An additional portion provides uniqueness within a single JVM, using an Atomic Reference

11. Atomic References – Parent Node AtomicReference's compareAndSet() Used for visibility Used to enforce data integrity constraints Note ImmutableList

12. Atomic References – Takeaways Volatiles suffice where atomic check-then-act is overkill Some atomic nonblocking algorithms involve looping for a failed compareAndSet() During high thread contention this could actually mean inefficiency Most real-world threads have far more to do than mere lock contention though

13. Blocking Queues Acts as a thread-safe implementation of a producer / consumer pattern JMS queue, though not distributed Insertion blocks until there is space available (for bounded queues)

14. Blocking Queues – Data Publish A very large (50GB) flat-file Consumers send data to a remote grid cache Multiple queue consumers increased throughput

15. Locks – ReadWriteLock Allows for multiple concurrent read locks No new read locks once a write lock is placed Write lock blocks until read locks complete Lock modes; non-fair (default), fair Reentrancy Downgrading

16. Distributed Cached Data Snapshot Multiple clients -> Multiple HTTP requests Multiple load balanced JVMs Distributed data grid cache Need to stream a data snapshot to n clients

17. Distributed Cached Data Snapshot Coherence partitioned cache supports cluster wide key-based lock Locked objects can still be read by other cluster threads without a lock Locks are unaffected by server failure (and will failover to a backup server.) Locks are immediately released when the lock owner (client) fails. Lock timeouts (-1, 0, 1+)

18. Hazelcast http://www.hazelcast.com/ Open source clustering and highly scalable data distribution platform for Java Distributed data structures Queue / Topic Map, MultiMap, Set, List Lock Effectively distributed java.util.concurrent Uses TCP/IP Cluster wide ID generators Distributed executor services Distributed Cached Data Snapshot

19. Distributed Cached Data Snapshot Apache Zookeeper http://hadoop.apache.org/zookeeper/ Terracotta http://www.terracotta.org/

20. Distributed Cached Data Snapshot Distributed competition for the publishing privilege Computation of completeness Communicate completeness Other threads in other JVMs happily polling for State.DONE

21. Distributed Cached Data Snapshot n number of clients n number of HTTP requests across 6 load balanced Tomcat JVMs Threads failing to acquire lock immediately start shipping data What about the thread obtaining the lock?

22. Concurrency in Frameworks Hibernate Core 3.5.0 CountDownLatch Need a thread to wait until some number of events have occurred Constructed with the count of the # events which must occur before release Callable, ExecutorService, ReentrantLock, AtomicReference

23. Concurrency in Frameworks Spring Framework 3.0.1 TaskExecutor Spring 2.0 supported Java 1.4 TaskExecutor did not implement Executor In Spring 3.0 TaskExecutor extends Executor TaskExecutor sees wide use within Spring framework Quartz Message Driven POJO Spring Enterprise Recipies – Josh Long, Gar Mak

24. Shopzilla's Website Concurrency Needed sub 650ms server side response time Simplify the layers Decompose architecture Functionally separate, individually testable, loosely coupled web-services Define SLAs

25. Shopzilla's Website Concurrency How to invoke 30+ web-services and ship a page in <650ms? Concurrency! Our pages today ship within 250ms

26. Shopzilla's Website Concurrency Pods & Service Calls

27. Shopzilla's Website Concurrency Started simple Implement only the concurrency features required Concurrency isolated to pods Pods responsible for fetching data We're using simple building blocks Incremental implementation based solely on requirements Haven't seen deadlocks

28. Shopzilla's Website Concurrency Thread longevity configured at the HTTP connection level HTTPClient connectionTimeout Spring wired HTTPClient implementation Ability to add a pod to a controller

29. Shopzilla's Website Concurrency FuturePodResult implements the PodResult interface Abstracts the details of the future PodCallable types the pod and command

30. Shopzilla's Website Concurrency Need to execute pods Configurable ExecutorService Backed with a queue Naming of threads proved useful in initial testing (JMX)

31. Shopzilla's Website Concurrency Once the concept was proven, interesting feature requests materializing Product Review pod Distilled, 2 pods needed to share a single result Added ServiceInvocation concept

32. Shopzilla's Website Concurrency Pods now have access to a Service Invocation Map get() blocks on the result of the service invocation A single service invocation result can be shared between two pods

33. Shopzilla's Website Concurrency Now we were sharing results, we were done, right? Product Review information was now required in-line in the product pod Still needed the special Product Review pod too! Dependent Service Invocations

34. Shopzilla's Website Concurrency Service Invocations can now depend on results of others Dependent Callable is configured with two callbacks; A callback whose result is blocked for A callback which is invoked once the blocking result arrives

35. Future More use of distributed data structures Spring 3.0 @Async @Scheduled JSR-315 Servlets 3.0 AsyncContext More parallelism

36. Resources, Books Java Concurrency in Practice (Goetz) Effective Java (Bloch) Spring Enterprise Recipes (Long, Mak) http://jcp.org/en/jsr/detail?id=133 http://jcp.org/en/jsr/detail?id=166 Spring 3.0

37. For more info…go to: http://tech.shopzilla.com http://rodneybarlow.org