October 2, 2010                                          1
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                          2
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
IBC SOLAR: Selling                  857 million turnover
     everything from a                   in 2009
     solar modul...
„Build yet another Shop
                       System for TYPO3 – are you
                                 insane?“
      ...
Previous system based on Commerce
        • scalability issues

     tt_products? Hell no! ;-)
     Specialized functional...
October 2, 2010                                          6
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                          7
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                          8
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                          9
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
All data comes from the company‘s ERP system
     Custom import functionality for complete orders
     Individual prices f...
standing on the shoulders of giants




October 2, 2010                                          11
Clemens Kalb, Lienhart...
How do you design a generic domain model for a
                    shop system?




October 2, 2010                       ...
You don‘t …
                                (unless you have too much time on your hands)




October 2, 2010             ...
…because it already exists:




October 2, 2010                                                              14
Clemens Ka...
BMEcat

October 2, 2010                                                  15
Clemens Kalb, Lienhart Woitok, T3CON10        ...
Proven & open standard for product catalogues
     Very well documented (specification >600 pages)
     Originally an XML ...
The Extbase kickstarter:
            a great way to create lots of model objects




October 2, 2010                      ...
October 2, 2010                                         18
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                         19
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
The kickstarter is not an editor!




October 2, 2010                                                              20
Clem...
Using SCM to cherry-pick changes:
1. Save in kickstarter
2. svn revert <fileThatShouldNotBeTouched>
3. Eclipse diff view t...
Apache Solr as a Storage Backend




October 2, 2010                                         22
Clemens Kalb, Lienhart Woi...
Java based search
                                         server
                                         Searches millio...
The naive way:
     writing direct querys to the Solr server from the
                     application code




October 2,...
NO
                                           Faceted       Extbase
     Repository                             Query
    ...
Tx_Extbase_Persistence_Query
     $query->equals(‘foobar’, $foobar)
     $query->logicalAnd($statement1,
     $statement2)...
Tx_Nxsolrbackend_Persistence_Backend_
SolrBackend
     foobar:$foobar
     ($statement1 AND $statement2)
     foobar:($ite...
Tx_Nxsolrbackend_Persistence_FacetedQuery
extends Tx_Extbase_Persistence_Query
     $query->facet($facetConfiguration)
   ...
Structural differences: database vs. index
     Data duplication: database and Solr index
     Extbase knows only one Stor...
Pentaho Data Integration




October 2, 2010                                         30
Clemens Kalb, Lienhart Woitok, T3C...
How to feed data into the database
                          from external sources?




October 2, 2010                   ...
(Photo courtesy of trenttsd/flickr)
October 2, 2010                                                                       ...
Looking for the right tool:
     Integrate different data sources homogenously
     Source data from Microsoft Dynamics NA...
Pentaho Data Integration
     Extract, Transform and Load
     JDBC + plugins for different data sources/targets




Octob...
Configurability: import into different sysfolders,
     configure parameters and default values
     Where needed: Use JS ...
How fast is Extbase?




October 2, 2010                                         36
Clemens Kalb, Lienhart Woitok, T3CON10...
First implementations were painfully slow due to…
     Working with thousands of objects
     Bug in Extbase: most objects...
Build first, optimize later!
              (Might be scary at times, but the approach does actually work…)




October 2, ...
Optimization #1:
     Make heavy use of lazy loading, load objects only
     when needed




October 2, 2010              ...
Optimization #2:
     Fix Extbase: Objects no longer dirty ;-)




October 2, 2010                                        ...
Optimization #3:
     Not everything needs to be an object:
     If you don‘t need domain logic,
     you might just rende...
Optimization #4:
     Use the good old TYPO3 caching mechanisms,
     but be especially careful




October 2, 2010       ...
October 2, 2010                                         43
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
October 2, 2010                                         44
Clemens Kalb, Lienhart Woitok, T3CON10   media.netlogix.de
Upcoming SlideShare
Loading in …5
×

Building a large eCommerce application with Extbase, Fluid and Apache Solr

6,496 views

Published on

  • Aron, on the one hand the slides might come across a little theoretical, but on the other hand, this wasn’t meant to be a 45-minute tutorial on Extbase either. Yes, of course we did dirty our hands. The slides relate to an actual project that has been online since a couple of months. We are currently in the process of open sourcing the components that make up the system, step by step. Extbase might not be complete in some aspects and it might still have a long way to go until it fits every project in the TYPO3 world - but in this specific case, we couldn’t have been happier with the results.

    (Clemens Kalb @ netlogix Media speaking ;-)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • It is too theoritical! Did you dirty your hand with actual project? Extbase still has a long way to go, yes, it will be awesome when ready.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Great project based on a great technology!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Building a large eCommerce application with Extbase, Fluid and Apache Solr

  1. October 2, 2010 1 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  2. October 2, 2010 2 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  3. IBC SOLAR: Selling 857 million turnover everything from a in 2009 solar module to 4-figure number of complete solar plants products x product Working with retail variants x retail partners in Germany partners and throughout the world October 2, 2010 3 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  4. „Build yet another Shop System for TYPO3 – are you insane?“ All of us & the customer (before the start of the project) October 2, 2010 4 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  5. Previous system based on Commerce • scalability issues tt_products? Hell no! ;-) Specialized functionality lacking in all systems Magento looked fairly nice – but not integrated in TYPO3 October 2, 2010 5 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  6. October 2, 2010 6 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  7. October 2, 2010 7 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  8. October 2, 2010 8 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  9. October 2, 2010 9 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  10. All data comes from the company‘s ERP system Custom import functionality for complete orders Individual prices for customers No need for retail functions like online payment processing October 2, 2010 10 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  11. standing on the shoulders of giants October 2, 2010 11 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  12. How do you design a generic domain model for a shop system? October 2, 2010 12 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  13. You don‘t … (unless you have too much time on your hands) October 2, 2010 13 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  14. …because it already exists: October 2, 2010 14 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  15. BMEcat October 2, 2010 15 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  16. Proven & open standard for product catalogues Very well documented (specification >600 pages) Originally an XML data exchange format October 2, 2010 16 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  17. The Extbase kickstarter: a great way to create lots of model objects October 2, 2010 17 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  18. October 2, 2010 18 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  19. October 2, 2010 19 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  20. The kickstarter is not an editor! October 2, 2010 20 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  21. Using SCM to cherry-pick changes: 1. Save in kickstarter 2. svn revert <fileThatShouldNotBeTouched> 3. Eclipse diff view to manually merge files October 2, 2010 21 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  22. Apache Solr as a Storage Backend October 2, 2010 22 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  23. Java based search server Searches millions of documents (products, text entries, prices…) in very little time Provides faceted search facilities October 2, 2010 23 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  24. The naive way: writing direct querys to the Solr server from the application code October 2, 2010 24 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  25. NO Faceted Extbase Repository Query FACETED persistence Query QUERIES layer Background storage Solr storage Storage Backend backend (DB, Solr…) October 2, 2010 25 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  26. Tx_Extbase_Persistence_Query $query->equals(‘foobar’, $foobar) $query->logicalAnd($statement1, $statement2) $query->in(‘foobar’, array($item1, $item2)) October 2, 2010 26 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  27. Tx_Nxsolrbackend_Persistence_Backend_ SolrBackend foobar:$foobar ($statement1 AND $statement2) foobar:($item1 $item2) October 2, 2010 27 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  28. Tx_Nxsolrbackend_Persistence_FacetedQuery extends Tx_Extbase_Persistence_Query $query->facet($facetConfiguration) $query->execute(): Tx_Nxsolrbackend_Persistence_ FacetedQueryResult $facetedQueryResult->getObjects() $facetedQueryResult->getFacets() October 2, 2010 28 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  29. Structural differences: database vs. index Data duplication: database and Solr index Extbase knows only one StorageBackend October 2, 2010 29 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  30. Pentaho Data Integration October 2, 2010 30 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  31. How to feed data into the database from external sources? October 2, 2010 31 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  32. (Photo courtesy of trenttsd/flickr) October 2, 2010 32 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  33. Looking for the right tool: Integrate different data sources homogenously Source data from Microsoft Dynamics NAV (MSSQL) CSV file to specify additional files for products like datasheets and the like Need to understand it 3 years later October 2, 2010 33 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  34. Pentaho Data Integration Extract, Transform and Load JDBC + plugins for different data sources/targets October 2, 2010 34 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  35. Configurability: import into different sysfolders, configure parameters and default values Where needed: Use JS and external code for special transformations October 2, 2010 35 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  36. How fast is Extbase? October 2, 2010 36 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  37. First implementations were painfully slow due to… Working with thousands of objects Bug in Extbase: most objects considered dirty at any time October 2, 2010 37 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  38. Build first, optimize later! (Might be scary at times, but the approach does actually work…) October 2, 2010 38 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  39. Optimization #1: Make heavy use of lazy loading, load objects only when needed October 2, 2010 39 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  40. Optimization #2: Fix Extbase: Objects no longer dirty ;-) October 2, 2010 40 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  41. Optimization #3: Not everything needs to be an object: If you don‘t need domain logic, you might just render raw query results October 2, 2010 41 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  42. Optimization #4: Use the good old TYPO3 caching mechanisms, but be especially careful October 2, 2010 42 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  43. October 2, 2010 43 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de
  44. October 2, 2010 44 Clemens Kalb, Lienhart Woitok, T3CON10 media.netlogix.de

×