Presentation given during Inspiring Flow 2013 in Kolbermoor

1. Using Document
Databases with
TYPO3 Flow
Inspiring people to
share

2. Karsten Dambekalns
TYPO3 Neos and Flow developer
35 years old
lives in Lübeck, Germany
1 wife, 3 sons
1 espresso machine
likes canoeing & climbing

3. Persistence Basics
in Flow

4. Flow uses Doctrine 2 ORM
TYPO3 Flow
Doctrine ORM
Doctrine DBAL
Native and PDO DB drivers
MySQL PostgreSQL SQLite Oracle

5. Flow provides a centralized
PersistenceManager
Repository DoctrineRepository
DoctrinePersistenceManager
Doctrine ORM
Doctrine DBAL

6. Repositories encapsulate
persistence concerns
Your Application Code
Repository DoctrineRepository
DoctrinePersistenceManager

7. Models do not know
about persistence

8. Models do not know
(a lot)
about persistence
(internals)

9. Properties
/**
* @FlowEntity
*/
class Organisation {
/**
* The name
* @var string
* @FlowValidate(type="NotEmpty")
* @FlowValidate(type="StringLength", options={"maximum"=40})
* @ORMColumn(length=40)
*/
protected $name;

10. Relations
/**
* @FlowEntity
*/
class Organisation {
/**
* @var DoctrineCommonCollectionsCollection
<AcmeDemoDomainModelAlias>
* @ORMOneToMany(mappedBy="organisation",cascade={"persist"})
* @ORMOrderBy({"name" = "ASC"})
* @FlowLazy
*/
protected $aliases;

11. Document Databases
Provide some powerful advantages
• schema-less storage
• usually scale well
• really good for non-relational data
No more Object-Relational Mapping, now you have Object-
Document Mapping
Queries, but not as you know them
• Most document databases do not allow queries on arbitary
properties

12. Which Database to Choose
Many different document databases exist
Which one is right for your project depends on many factors
• MongoDB: For most things that you would do with MySQL or
PostgreSQL, but having predefined columns really holds you back.
• CouchDB: For accumulating, occasionally changing data, on which
pre-defined queries are to be run. Places where versioning is
important.
A nice overview with much more can be found on
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

13. CouchDB
Is one of the more widespread products
Stores documents in JSON
Provides a REST interface
Provides master-master replication and MVCC

14. Futon on CouchDB

15. CouchDB REST API
Create a database
curl -X PUT http://127.0.0.1:5984/demo
{"ok":true}
Create documents
curl -H 'Content-Type: application/json'
-X POST http://127.0.0.1:5984/demo
-d '{"company": "Example, Inc."}'
{"ok":true,"id":"8843faaf0b831d364278331bc3001bd8",
"rev":"1-33b9fbce46930280dab37d672bbc8bb9"}

16. CouchDB Basics
Fetch a document
curl -X GET http://127.0.0.1:5984/demo/
8843faaf0b831d364278331bc3001bd8
{"_id":"8843faaf0b831d364278331bc3001bd8",
"_rev":"1-33b9fbce46930280dab37d672bbc8bb9",
"company":"Example, Inc."}

17. CouchDB Basics
Create a database using Futon or with
curl -X PUT http://127.0.0.1:5984/demo
Create documents using Futon or with
curl -H 'Content-Type: application/json'
-X POST http://127.0.0.1:5984/demo
-d '{"company": "Example, Inc."}'
Fetch documents using Futon
If you do manual work on the shell, try HTTPie instead of cURL
Read guide.couchdb.org and docs.couchdb.org

18. CouchDB Basics
Read guide.couchdb.org and docs.couchdb.org
Learn about views and map/reduce

19. Hint: A better cURL
If you do manual work on the shell, try HTTPie instead of cURL

20. TYPO3.CouchDB
Developed for Rossmann by networkteam
Fully replaces the ORM persistence
Model annotations stay the same
Provides basic QOM-to-View mapping
Note: Not working with TYPO3 Flow 2.0 right now

21. Installation
No composer package, so just clone it
cd Packages/Application
git clone git://git.typo3.org/FLOW3/Packages/TYPO3.CouchDB.git

22. Configuration
TYPO3:
FLOW3:
persistence:
# Options for the CouchDB backend
backendOptions:
# database: 'database_name'
dataSourceName: 'http://127.0.0.1:5984'
username: ~
password: ~
enableCouchdbLucene: no
driver: ~
path: ~

23. Usage – Repository
class TestEntityRepository extends Repository {
/**
* @param string $name
* @return TYPO3FLOW3PersistenceQueryResultInterface
*/
public function findByNameLike($name) {
$query = $this->testIndex->createQuery();
$query->matching(
$query->like('name', $name)
);
return $query->execute();
}
}

24. Usage – LuceneIndex
class MyIndex extends TYPO3CouchDBDomainIndexLuceneIndex {
/**
* Configure the index
*
* @return void
*/
public function configure() {
$this->forEntity('AcmeDemoDomainModelSomeEntity')
->indexProperty('name')
->indexProperty('relatedValueObject.color');
}
}

25. Usage – Views & Design Docs
Views can be defined by
• implementing TYPO3CouchDBViewInterface
• configuring TYPO3CouchDBQueryView
Design documents can be defined by extending
TYPO3CouchDBDesignDocument

26. Now for some
bad news

27. Only CouchDB is supported
Kristina Alexanderson
CC BY-NC-SA 2.0

28. One Backend at a Time
Repository DoctrineRepository
DoctrinePersistenceManager
Doctrine ORM
Doctrine DBAL

29. One Backend at a Time
Repository DoctrineRepository
DoctrinePersistenceManager
Doctrine ORM
Doctrine DBAL

30. You can work around this!
In your repositories you can basically do what you want
• Open a direct database connection
• Read CSV files
• Connect to a Solr server
• Instantiate another Doctrine EntityManager
Do something more advanced, let Radmiraal.CouchDB inspire you
As long as you encapsulate nicely!

31. And now some
good news

32. Radmiraal.CouchDB

33. Radmiraal.CouchDB
Developed for some projects having similar needs
Started out as an extension to TYPO3.CouchDB
Since then refactored & now based on Doctrine 2
CouchDB ODM
Can be used alongside the usual ORM

34. Radmiraal.CouchDB

35. Radmiraal.CouchDB

36. Installation
"repositories": [
{
"type": "git",
"url": "https://github.com/radmiraal/Radmiraal.CouchDB.git"
}
],
"require": {
"radmiraal/couchdb": "dev-doctrine",
}

37. Configuration
TYPO3:
Flow:
persistence:
backendOptions:
… as usual
Radmiraal:
CouchDB:
persistence:
backendOptions:
databaseName: 'mycouchdb'
username: 'mycouchuser'
password: 'mycouchpassw0rd'

38. Usage – Models
use DoctrineODMCouchDBMappingAnnotations as ODM;
/**
* @ODMDocument(indexed=true)
*/
class DataSet {
/**
* @var string
* @ODMId(type="string")
*/
protected $id;
/**
* @var AcmeDemoDomainModelOrganisation

39. Usage – Models
/**
* @var AcmeDemoDomainModelOrganisation
* @ODMField(type="string")
* @ODMIndex
*/
protected $organisation;
/**
* @var DateTime
* @ODMField(type="datetime")
* @ODMIndex
*/
protected $creationDateTime;

40. Usage – Repository
class OrganisationBoundaryRepository
extends RadmiraalCouchDBPersistenceAbstractRepository {
public function findAll() {
return $this->createQuery(
'organisationboundaries', 'by_organisation_id')
->execute();
}
}

41. Usage – Repository
class DataSetRepository
extends RadmiraalCouchDBPersistenceAbstractRepository {
public function findByQuux(Quux $quux) {
return
$this->createQuery('datasets', 'most_recent', 'native')
->setStartKey(array($this->getQueryMatchValue($quux)))
->setGroup(TRUE)
->setGroupLevel(1)
->setLimit(1)
->execute()
}

42. Usage – Views & Design Docs

43. And now some
even better news

44. Multiple backends in parallel
Doctrine Doctrine MongoDB
Repository Repository Repository
PersistenceManager
Doctrine ORM MongoDB ODM
Doctrine DBAL Doctrine MongoDB

45. The Future
TYPO3 Flow will support multiple persistence backends in
parallel:
• of the same type: 2 MySQL and 1 PostgreSQL databases
• of different types: MySQL and MongoDB

46. The Future
Configuration extended but backwards compatible
Consistent use independent of backend used:
Annotations, result interfaces & queries stay consistent
References across persistence backends? Eventually, but not
right away…

47. You can help, too…
The work on a proper ODM integration so far has been
done and made possible by (applause!):
• Beech IT
• BKWI
• Rens Admiraal
• myself
More work is needed, so if you think this is the way to
go… contact me!

48. your
questions
now!

49. Thank You!
These slides can be found at:
http://speakerdeck.com/kdambekalns
http://slideshare.net/kfish
Give me feedback: karsten@typo3.org | karsten@dambekalns.de
Book me: karsten@dambekalns.de
Follow me on twitter: @kdambekalns
Support me using
My Amazon wishlist: http://bit.ly/kdambekalns-wishlist