The document discusses an architecture for scaling to handle very large datasets and complex requirements. It proposes scaling in three dimensions: 1) scaling hardware through techniques like parallel processing and optimized components; 2) scaling server instances using a stateless and shared-nothing approach for easy load balancing; and 3) scaling client computers by leveraging client-side storage, caching, and processing to reduce server load. The proposed architecture aims to meet the needs of a use case involving personalized online ordering and pickup of store-specific products from hundreds of stores with millions of products.

1. Architecture for 10^X
3d-scaling for real world problems
Donnerstag, 26. Mai 2011

2. Who we are
Markus Wolf, @KnisterPeter Meno Abels, @fastandfearless
Senior Developer, SinnerSchrader Director, SinnerSchrader
2
Donnerstag, 26. Mai 2011

3. Architecture is important
Choosing an architecture has impact in several dimensions
Donnerstag, 26. Mai 2011

4. Development
Donnerstag, 26. Mai 2011

5. Operation
Donnerstag, 26. Mai 2011

6. Maintaince
Maintenance
Donnerstag, 26. Mai 2011

7. Our Briefing from
We want to sell our store-specific range of goods to our
customers online
Customers should be able to collect these items within a
appointed timeframe
In other words: “Click & Collect”
Sounds easy.
Donnerstag, 26. Mai 2011

8. Wait: store-specific?
Donnerstag, 26. Mai 2011

9. What else?
2*10^4 products
4*10^2 shops (potential for up to 3*10^3)
... and adding:
_unique prices and product-attributes per shop
_subset catalogs for specific product lines
(e.g. Bio/Rewe-Feine-Welt/Angebote)
_personalized catalog
We need to have as much webshops as real shops providing
this service.
Donnerstag, 26. Mai 2011

10. Personalized Catalog for each customer?
Donnerstag, 26. Mai 2011

11. The Establishment will fail
No standard shop software will meet these requirements
Donnerstag, 26. Mai 2011

12. We have a mission
Accept the challenge and think yourself
Donnerstag, 26. Mai 2011

13. Scaling in 1D
Hardware
Use memory extensively
=> around 10^6 faster than disks
Donnerstag, 26. Mai 2011

14. Scaling in 1D
Hardware
Use parallel algorithms
=> leverage multicore cpu
Donnerstag, 26. Mai 2011

15. Scaling in 1D
Hardware
Use stateless components
=> less error prone multithreading
Donnerstag, 26. Mai 2011

16. Scaling in 1D
Hardware
Use view optimized Business Objects
=> no transformation for view rendering
Donnerstag, 26. Mai 2011

17. Scaling in 1D
Hardware
=> more requests per Server Instance (approx. 100 per second)
Donnerstag, 26. Mai 2011

18. Scaling in 2D
Server Instances
Donnerstag, 26. Mai 2011

19. Scaling in 2D
Server Instances
“Stateless Statefullness”
_No server-side state
_All state on client
Donnerstag, 26. Mai 2011

20. Scaling in 2D
Server Instances
Shared Nothing
„Kind of“ SOA
This means easy load-balancing
=> weʼre cloud-ready!
Donnerstag, 26. Mai 2011

21. Scaling in 3D
Client Computer
Donnerstag, 26. Mai 2011

22. Scaling in 3D
Client Computer
Use client storage
_Persist private data into browser
_Caching of product data
Donnerstag, 26. Mai 2011

23. Scaling in 3D
Client Computer
Donnerstag, 26. Mai 2011

24. Scaling in 3D
Client Computer
Less server cycles
_Gaining performance independent from server side
_Green-IT
Donnerstag, 26. Mai 2011

25. Scaling in 3D
Client Computer
=> Less requests, faster user experience, happy users!
Donnerstag, 26. Mai 2011

26. Scaling in Reality
Simple Case
Donnerstag, 26. Mai 2011

27. Enterprise toolchain
Donnerstag, 26. Mai 2011

28. Future Proof
Donnerstag, 26. Mai 2011

29. The Result
Server-Software lines of code <2*10^4
(common > 5*10^4)
Client-Software lines of code < 10^4
(no one should need more)
Development Effort approx. (5*10^1) %
(compared to traditional approach)
Deployment 3 Systems*
*Cost of operations per year is C.H.E.A.P.
Cost per user/year is roughly about:
Donnerstag, 26. Mai 2011

30. The Result
Donnerstag, 26. Mai 2011

31. Be careful!
Thank you.
Donnerstag, 26. Mai 2011