Using Document Databases with TYPO3 Flow


Published on

Presentation given during Inspiring Flow 2013 in Kolbermoor

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Using Document Databases with TYPO3 Flow

  1. 1. Using DocumentDatabases with TYPO3 Flow Inspiring people to share
  2. 2. Karsten Dambekalns TYPO3 Neos and Flow developer 35 years old lives in Lübeck, Germany 1 wife, 3 sons 1 espresso machine likes canoeing & climbing
  3. 3. Persistence Basics in Flow
  4. 4. Flow uses Doctrine 2 ORM TYPO3 Flow Doctrine ORM Doctrine DBAL Native and PDO DB drivers MySQL PostgreSQL SQLite Oracle
  5. 5. Flow provides a centralizedPersistenceManager Repository DoctrineRepository DoctrinePersistenceManager Doctrine ORM Doctrine DBAL
  6. 6. Repositories encapsulatepersistence concerns Your Application Code Repository DoctrineRepository DoctrinePersistenceManager
  7. 7. Models do not know about persistence
  8. 8. Models do not know (a lot) about persistence (internals)
  9. 9. Properties/** * @FlowEntity */class Organisation { /** * The name * @var string * @FlowValidate(type="NotEmpty") * @FlowValidate(type="StringLength", options={"maximum"=40}) * @ORMColumn(length=40) */ protected $name;
  10. 10. Relations/** * @FlowEntity */class Organisation { /** * @var DoctrineCommonCollectionsCollection <AcmeDemoDomainModelAlias> * @ORMOneToMany(mappedBy="organisation",cascade={"persist"}) * @ORMOrderBy({"name" = "ASC"}) * @FlowLazy */ protected $aliases;
  11. 11. Document Databases Provide some powerful advantages• schema-less storage• usually scale well• really good for non-relational data No more Object-Relational Mapping, now you have Object- Document Mapping Queries, but not as you know them• Most document databases do not allow queries on arbitary properties
  12. 12. Which Database to Choose Many different document databases exist Which one is right for your project depends on many factors• MongoDB: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back.• CouchDB: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. A nice overview with much more can be found on
  13. 13. CouchDB Is one of the more widespread products Stores documents in JSON Provides a REST interface Provides master-master replication and MVCC
  14. 14. Futon on CouchDB
  15. 15. CouchDB REST API Create a database curl -X PUT {"ok":true} Create documents curl -H Content-Type: application/json -X POST -d {"company": "Example, Inc."} {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8", "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
  16. 16. CouchDB Basics Fetch a document curl -X GET 8843faaf0b831d364278331bc3001bd8 {"_id":"8843faaf0b831d364278331bc3001bd8", "_rev":"1-33b9fbce46930280dab37d672bbc8bb9", "company":"Example, Inc."}
  17. 17. CouchDB Basics Create a database using Futon or with curl -X PUT Create documents using Futon or with curl -H Content-Type: application/json -X POST -d {"company": "Example, Inc."} Fetch documents using Futon If you do manual work on the shell, try HTTPie instead of cURL Read and
  18. 18. CouchDB Basics Read and Learn about views and map/reduce
  19. 19. Hint: A better cURL If you do manual work on the shell, try HTTPie instead of cURL
  20. 20. TYPO3.CouchDB Developed for Rossmann by networkteam Fully replaces the ORM persistence Model annotations stay the same Provides basic QOM-to-View mapping Note: Not working with TYPO3 Flow 2.0 right now
  21. 21. Installation No composer package, so just clone itcd Packages/Applicationgit clone git://
  22. 22. ConfigurationTYPO3: FLOW3: persistence: # Options for the CouchDB backend backendOptions: # database: database_name dataSourceName: username: ~ password: ~ enableCouchdbLucene: no driver: ~ path: ~
  23. 23. Usage – Repositoryclass TestEntityRepository extends Repository { /** * @param string $name * @return TYPO3FLOW3PersistenceQueryResultInterface */ public function findByNameLike($name) { $query = $this->testIndex->createQuery(); $query->matching( $query->like(name, $name) ); return $query->execute(); }}
  24. 24. Usage – LuceneIndexclass MyIndex extends TYPO3CouchDBDomainIndexLuceneIndex { /** * Configure the index * * @return void */ public function configure() { $this->forEntity(AcmeDemoDomainModelSomeEntity) ->indexProperty(name) ->indexProperty(relatedValueObject.color); }}
  25. 25. Usage – Views & Design Docs Views can be defined by• implementing TYPO3CouchDBViewInterface• configuring TYPO3CouchDBQueryView Design documents can be defined by extending TYPO3CouchDBDesignDocument
  26. 26. Now for some bad news
  27. 27. Only CouchDB is supportedKristina Alexanderson CC BY-NC-SA 2.0
  28. 28. One Backend at a Time Repository DoctrineRepository DoctrinePersistenceManager Doctrine ORM Doctrine DBAL
  29. 29. One Backend at a Time Repository DoctrineRepository DoctrinePersistenceManager Doctrine ORM Doctrine DBAL
  30. 30. You can work around this! In your repositories you can basically do what you want• Open a direct database connection• Read CSV files• Connect to a Solr server• Instantiate another Doctrine EntityManager Do something more advanced, let Radmiraal.CouchDB inspire you As long as you encapsulate nicely!
  31. 31. And now some good news
  32. 32. Radmiraal.CouchDB
  33. 33. Radmiraal.CouchDB Developed for some projects having similar needs Started out as an extension to TYPO3.CouchDB Since then refactored & now based on Doctrine 2 CouchDB ODM Can be used alongside the usual ORM
  34. 34. Radmiraal.CouchDB
  35. 35. Radmiraal.CouchDB
  36. 36. Installation"repositories": [ { "type": "git", "url": "" }],"require": { "radmiraal/couchdb": "dev-doctrine",}
  37. 37. ConfigurationTYPO3: Flow: persistence: backendOptions: … as usualRadmiraal: CouchDB: persistence: backendOptions: databaseName: mycouchdb username: mycouchuser password: mycouchpassw0rd
  38. 38. Usage – Modelsuse DoctrineODMCouchDBMappingAnnotations as ODM;/** * @ODMDocument(indexed=true) */class DataSet { /** * @var string * @ODMId(type="string") */ protected $id; /** * @var AcmeDemoDomainModelOrganisation
  39. 39. Usage – Models /** * @var AcmeDemoDomainModelOrganisation * @ODMField(type="string") * @ODMIndex */ protected $organisation; /** * @var DateTime * @ODMField(type="datetime") * @ODMIndex */ protected $creationDateTime;
  40. 40. Usage – Repositoryclass OrganisationBoundaryRepository extends RadmiraalCouchDBPersistenceAbstractRepository { public function findAll() { return $this->createQuery( organisationboundaries, by_organisation_id) ->execute(); }}
  41. 41. Usage – Repositoryclass DataSetRepository extends RadmiraalCouchDBPersistenceAbstractRepository { public function findByQuux(Quux $quux) { return $this->createQuery(datasets, most_recent, native) ->setStartKey(array($this->getQueryMatchValue($quux))) ->setGroup(TRUE) ->setGroupLevel(1) ->setLimit(1) ->execute()}
  42. 42. Usage – Views & Design Docs
  43. 43. And now someeven better news
  44. 44. Multiple backends in parallel Doctrine Doctrine MongoDB Repository Repository Repository PersistenceManager Doctrine ORM MongoDB ODM Doctrine DBAL Doctrine MongoDB
  45. 45. The Future TYPO3 Flow will support multiple persistence backends in parallel:• of the same type: 2 MySQL and 1 PostgreSQL databases• of different types: MySQL and MongoDB
  46. 46. The Future Configuration extended but backwards compatible Consistent use independent of backend used: Annotations, result interfaces & queries stay consistent References across persistence backends? Eventually, but not right away…
  47. 47. You can help, too… The work on a proper ODM integration so far has been done and made possible by (applause!):• Beech IT• BKWI• Rens Admiraal• myself More work is needed, so if you think this is the way to go… contact me!
  48. 48. yourquestions now!
  49. 49. Thank You! These slides can be found at: Give me feedback: | Book me: Follow me on twitter: @kdambekalns Support me using My Amazon wishlist: