Doctrine NoSQL
Benjamin Eberlei (SimpleThings GmBH)
About me
 Benjamin Eberlei
 Working at SimpleThings GmBH
   http://www.simplethings.de
 Open Source contributor
   Doctrine2, Symfony2
   (Zeta Components, PHPUnit, ...)
 Twitter @beberlei
 Blog http://www.whitewashing.de
The Doctrine Project
www.doctrine-project.org

The Doctrine Project is the home of a selected
set of PHP libraries primarily focused on
providing persistence services and related
functionality.
Doctrine Subprojects
 DBAL and ORM
 Document Mapper (MongoDB, CouchDB, PHPCR)
 Annotations
 XML
 What is next?
Doctrine Philosophy
       Separating Persistance and Objects
Doctrine Philosophy
    Small common API - Similar look and feel
Doctrine Philosphy
            Embrace Differences
Why NoSQL Mapper?
Schemaless storage allows storing:

  Arbitrary associations
  Embedded objects
  Lists and Associative Arrays

No duplicate schema-maintenance!
Doctrine NoSQL History
 MongoDB Mapper early 2010 (OpenSky)
 CouchDB Mapper started in October 2010 (Liip)
 PHPCR ODM started in early 2011 (Liip)
 APIs heavily inspired from ORM
SQL and NoSQL Similarities
 Extracted common persistence interfaces
 Covering roughly 10-20% of the use-cases
   Simple Finder Methods
   Insert/Update/Delete
   Metadata API
 Support for Annotations/XML/YAML/PHP Mapping
Persistence Interfaces
<?php
interface ObjectManager
{
    function find($class, $id);
    function getReference($class, $id);
    function persist($object);
    function remove($object);
    function flush();

    function getClassMetadata($class);
    function getRepository($class);
}
Persistence Interfaces
<?php
interface ObjectRepository
{
    function find($id);
    function findAll();
    function findBy(array $criteria,
        $orderBy = null,
        $limit = null,
        $offset = null
    )
    function findOneBy(array $criteria);
}
Sample Document
<?php
/** @Document */
class Message
{
    /** @Id */
    public $id;
    /** @Field(type="string") */
    public $text;
}

$message = new Message();
$message->setText("Hello World!");
NoSQL benefits
<?php
/** @Document */
class Product
{
    /** other fields */
    /** @Field(type="array") */
    public $attributes;
    /** @Field(type="array") */
    public $translations;
}

$product->attributes["isbn"] = "A-B-C-D";
$product->translations["de"]["name"] = "Ein Produkt";
Working with Objects 1
Creating a new document:

<?php
/** @var $dm DocumentManager */
$message = new Message();
$message->setText("I am new!");

$dm->persist($message);
$dm->flush();

echo "ID: " . $message->getId();
Working with Objects 2
Find and update document:

<?php
/** @var $dm DocumentManager */
$message = $dm->find("Message", 1);
$message->setText("New Message");
$dm->flush();
Working with Objects 3
Find and remove documents:

<?php
/** @var $dm DocumentManager */
$repository = $dm->getRepository("User");
$criteria = array("status" => "inactive");
$users = $repository->findBy($criteria);

foreach ($users AS $user) {
    $dm->remove($user);
}
$dm->flush();
Persistence API Use-Cases
 No query abstraction
 Focus on "in memory" object workflows
 Specialized reusable Modules for Frameworks
 Symfony2:
   User Management
   Comment
   Admin Generators
   lichess
Associations in NoSQL
Cons
 No referential integrity
 No support for transactions

Pros
 Embedded Documents
 Adhoc references of arbitrary types
Association Mappings
<?php
/** @Document */
class Blog
{
    /** @ReferenceMany */
    private $articles;
    /** @ReferenceOne(targetDocument="User") */
    private $owner;
}
Association keys
<?php
$id = "1";
$articleSlug = "hello-world";

$blog = $dm->find("Blog", $id);
$blog->articles[$articleSlug]->getHeadline();
Embedded Mappings
<?php
/** @Document */
class User
{
    /** @EmbedMany */
    private $phonenumbers;
    /** @EmbedOne(targetDocument="Address") */
    private $address;
}
CouchDB and Doctrine
 JSON Datastorage
 HTTP/REST API
 MVCC, eventually consistent (Conflicts)
 Replication
 Attachments
 Views and Map/Reduce in Javascript
 CouchDB Lucene
 Doctrine CouchDB 1.0 Alpha 1
JSON Document
{
    "_id": "716104ac33c797b12d50c0a6483f1661",
    "_rev": "1-32db404b78f130fd8f7575905859e19b",
    "doctrine_metadata":
    {
        "type": "MyProject.Document.Message",
        "associations":
        {
            "user": "055fe8a3ab06c3998d27b6d99f5a9bdd"
        }
    },
    "message": "I am a message"
}
Document Version
Implement Optimistic-Locking

<?php
class Article
{
    /** @Version */
    private $version;
}

$article = $dm->find(
    "Article", $id, $expectedVersion
);
Attachments
 CouchDB supports Attachments to documents
 Doctrine converts into Attachment object
 Lazy Load binary data from the server
 Stream support planned

<?php
class Article
{
    /** @Attachments */
    public $attachments = array();
}
Attachments 2
<?php
use DoctrineCouchDBAttachment;

$article = $dm->find("Article", 1);
$data = $article->attachments["teaser.jpg"]->getContent();

$a = Attachment::createFromBase64data($data, "image/jpg");
$article->attachments["author.jpg"] = $a;

$dm->flush();
Views
Doctrine CouchDB maps filesystem to design document:

application/
    couchdb/
        views/
             username/
                 map.js
                 reduce.js

Use javascript syntax higlighting in your IDE/Editor.
Views 2
<?php
use DoctrineCouchDBViewFileFolderDesignDocument;

$path = "/path/application/couchdb";
$designDoc = new FileFolderDesignDocument($path);

/* @doc $couch CouchClient */
$docName = "myapp";
$couch->createDesignDoc($docName, $designDoc);
Query Views
<?php
/* @var $dm DocumentManager */
$query = $dm->createQuery("myapp", "username");
$result = $query->setStartKey("b")
                ->setEndKey("c")
                ->setLimit(10)
                ->setSkip(10)
                ->includeDocs(true)
                ->execute();

Using include docs creates PHP instances for you.
Lucene Queries
Support for the CouchDB Lucene extension:

<?php
$query = $dm->createLuceneQuery("lucenedoc", "users");
$result = $query->setQuery('"John Galt" OR "John Wayne"')
                ->setLimit(10)
                ->setSkip(10)
                ->includeDocs(true)
                ->execute();
MongoDB and Doctrine
 Indexing and on the fly
 queries
 Very fast
 In-Place Updates²
 GridFS, Geolocation
 Sharding
 Doctrine MongoDB 1.0 Beta2
Complex Associations
<?php
class User
{
    /**
      * @ReferenceMany(
      *    targetDocument="Comment",
      *    mappedBy="blogPost",
      *    sort={"date"="desc"},
      *    limit=5)
      */
    private $last5Comments;
}
Query API
<?php
$qb = $dm->createQueryBuilder('User')
      ->field('groups')
      ->all(array('Group 1', 'Group 2'))
      ->sort("username", "asc")
      ->limit(10)
      ->skip(10)
      ->execute();
Map/Reduce
<?php
$qb = $dm->createQueryBuilder('DocumentsUser')
    ->field('type')->equals('sale')
    ->map('function() { emit(this.user.$id, 1); }')
    ->reduce('function(k, vals) {
         var sum = 0;
         for (var i in vals) {
             sum += vals[i];
         }
         return sum;
    }');
Geospatial Queries
<?php
/** @Document @Index(keys={"coordinates"="2d"}) */
class City
{
    /** @EmbedOne(targetDocument="Coordinates") */
    public $coordinates;
    /** @Distance */
    public $distance;
}
class Coordinates
{
    public $lat;
    public $long;
}
Geospatial Queries 2
Execute a Geospatial query and find locations near a point:

<?php
/* @var $dm DocumentManager */
$cities = $dm->createQuery('City')
    ->field('coordinates')->near(50, 60)
    ->execute();
Eventual Migration
Handle simple and complex schema refactorings

<?php
class Person
{
    public $id;

     /** @AlsoLoad("name") */
     public $fullName;
}
More of Doctrine MongoDB
 Support for Trees
 Support for Files in MongoGridFS
 Capped Collections
 Tailable Cursors
PHPCR ODM
PHP Content Repository Project
Port of the Java Content Repository API
PHP API for Apache Jackrabbit
Doctrine PHPCR ODMs creates objects from Content Repositories.
Object to XML Mapper
 Convert objects to XML documents and back

<?php
$user = new User();
$user->setFirstName('John');
$user->setLastName('Doe');
$user->setAddress(new Address('123 Street', 'New Haven'));
$user->addContact(new CustomerContact('no@way.com'));

$xml = $marshaller->marshalToString($user);
$user = $marshaller->unmarshalFromString($xml);
Using Doctrine 2.0.x ORM?
Please checkout 2.1 BETA1
 It is backwards compatible
 You win a present if you can prove otherwise.
Thank you!
Rate this talk:

http://joind.in/talk/view/3515

Doctrine for NoSQL

  • 1.
  • 2.
    About me BenjaminEberlei Working at SimpleThings GmBH http://www.simplethings.de Open Source contributor Doctrine2, Symfony2 (Zeta Components, PHPUnit, ...) Twitter @beberlei Blog http://www.whitewashing.de
  • 3.
    The Doctrine Project www.doctrine-project.org TheDoctrine Project is the home of a selected set of PHP libraries primarily focused on providing persistence services and related functionality.
  • 4.
    Doctrine Subprojects DBALand ORM Document Mapper (MongoDB, CouchDB, PHPCR) Annotations XML What is next?
  • 5.
    Doctrine Philosophy Separating Persistance and Objects
  • 6.
    Doctrine Philosophy Small common API - Similar look and feel
  • 7.
    Doctrine Philosphy Embrace Differences
  • 8.
    Why NoSQL Mapper? Schemalessstorage allows storing: Arbitrary associations Embedded objects Lists and Associative Arrays No duplicate schema-maintenance!
  • 9.
    Doctrine NoSQL History MongoDB Mapper early 2010 (OpenSky) CouchDB Mapper started in October 2010 (Liip) PHPCR ODM started in early 2011 (Liip) APIs heavily inspired from ORM
  • 10.
    SQL and NoSQLSimilarities Extracted common persistence interfaces Covering roughly 10-20% of the use-cases Simple Finder Methods Insert/Update/Delete Metadata API Support for Annotations/XML/YAML/PHP Mapping
  • 11.
    Persistence Interfaces <?php interface ObjectManager { function find($class, $id); function getReference($class, $id); function persist($object); function remove($object); function flush(); function getClassMetadata($class); function getRepository($class); }
  • 12.
    Persistence Interfaces <?php interface ObjectRepository { function find($id); function findAll(); function findBy(array $criteria, $orderBy = null, $limit = null, $offset = null ) function findOneBy(array $criteria); }
  • 13.
    Sample Document <?php /** @Document*/ class Message { /** @Id */ public $id; /** @Field(type="string") */ public $text; } $message = new Message(); $message->setText("Hello World!");
  • 14.
    NoSQL benefits <?php /** @Document*/ class Product { /** other fields */ /** @Field(type="array") */ public $attributes; /** @Field(type="array") */ public $translations; } $product->attributes["isbn"] = "A-B-C-D"; $product->translations["de"]["name"] = "Ein Produkt";
  • 15.
    Working with Objects1 Creating a new document: <?php /** @var $dm DocumentManager */ $message = new Message(); $message->setText("I am new!"); $dm->persist($message); $dm->flush(); echo "ID: " . $message->getId();
  • 16.
    Working with Objects2 Find and update document: <?php /** @var $dm DocumentManager */ $message = $dm->find("Message", 1); $message->setText("New Message"); $dm->flush();
  • 17.
    Working with Objects3 Find and remove documents: <?php /** @var $dm DocumentManager */ $repository = $dm->getRepository("User"); $criteria = array("status" => "inactive"); $users = $repository->findBy($criteria); foreach ($users AS $user) { $dm->remove($user); } $dm->flush();
  • 18.
    Persistence API Use-Cases No query abstraction Focus on "in memory" object workflows Specialized reusable Modules for Frameworks Symfony2: User Management Comment Admin Generators lichess
  • 19.
    Associations in NoSQL Cons No referential integrity No support for transactions Pros Embedded Documents Adhoc references of arbitrary types
  • 20.
    Association Mappings <?php /** @Document*/ class Blog { /** @ReferenceMany */ private $articles; /** @ReferenceOne(targetDocument="User") */ private $owner; }
  • 21.
    Association keys <?php $id ="1"; $articleSlug = "hello-world"; $blog = $dm->find("Blog", $id); $blog->articles[$articleSlug]->getHeadline();
  • 22.
    Embedded Mappings <?php /** @Document*/ class User { /** @EmbedMany */ private $phonenumbers; /** @EmbedOne(targetDocument="Address") */ private $address; }
  • 23.
    CouchDB and Doctrine JSON Datastorage HTTP/REST API MVCC, eventually consistent (Conflicts) Replication Attachments Views and Map/Reduce in Javascript CouchDB Lucene Doctrine CouchDB 1.0 Alpha 1
  • 24.
    JSON Document { "_id": "716104ac33c797b12d50c0a6483f1661", "_rev": "1-32db404b78f130fd8f7575905859e19b", "doctrine_metadata": { "type": "MyProject.Document.Message", "associations": { "user": "055fe8a3ab06c3998d27b6d99f5a9bdd" } }, "message": "I am a message" }
  • 25.
    Document Version Implement Optimistic-Locking <?php classArticle { /** @Version */ private $version; } $article = $dm->find( "Article", $id, $expectedVersion );
  • 26.
    Attachments CouchDB supportsAttachments to documents Doctrine converts into Attachment object Lazy Load binary data from the server Stream support planned <?php class Article { /** @Attachments */ public $attachments = array(); }
  • 27.
    Attachments 2 <?php use DoctrineCouchDBAttachment; $article= $dm->find("Article", 1); $data = $article->attachments["teaser.jpg"]->getContent(); $a = Attachment::createFromBase64data($data, "image/jpg"); $article->attachments["author.jpg"] = $a; $dm->flush();
  • 28.
    Views Doctrine CouchDB mapsfilesystem to design document: application/ couchdb/ views/ username/ map.js reduce.js Use javascript syntax higlighting in your IDE/Editor.
  • 29.
    Views 2 <?php use DoctrineCouchDBViewFileFolderDesignDocument; $path= "/path/application/couchdb"; $designDoc = new FileFolderDesignDocument($path); /* @doc $couch CouchClient */ $docName = "myapp"; $couch->createDesignDoc($docName, $designDoc);
  • 30.
    Query Views <?php /* @var$dm DocumentManager */ $query = $dm->createQuery("myapp", "username"); $result = $query->setStartKey("b") ->setEndKey("c") ->setLimit(10) ->setSkip(10) ->includeDocs(true) ->execute(); Using include docs creates PHP instances for you.
  • 31.
    Lucene Queries Support forthe CouchDB Lucene extension: <?php $query = $dm->createLuceneQuery("lucenedoc", "users"); $result = $query->setQuery('"John Galt" OR "John Wayne"') ->setLimit(10) ->setSkip(10) ->includeDocs(true) ->execute();
  • 32.
    MongoDB and Doctrine Indexing and on the fly queries Very fast In-Place Updates² GridFS, Geolocation Sharding Doctrine MongoDB 1.0 Beta2
  • 33.
    Complex Associations <?php class User { /** * @ReferenceMany( * targetDocument="Comment", * mappedBy="blogPost", * sort={"date"="desc"}, * limit=5) */ private $last5Comments; }
  • 34.
    Query API <?php $qb =$dm->createQueryBuilder('User') ->field('groups') ->all(array('Group 1', 'Group 2')) ->sort("username", "asc") ->limit(10) ->skip(10) ->execute();
  • 35.
    Map/Reduce <?php $qb = $dm->createQueryBuilder('DocumentsUser') ->field('type')->equals('sale') ->map('function() { emit(this.user.$id, 1); }') ->reduce('function(k, vals) { var sum = 0; for (var i in vals) { sum += vals[i]; } return sum; }');
  • 36.
    Geospatial Queries <?php /** @Document@Index(keys={"coordinates"="2d"}) */ class City { /** @EmbedOne(targetDocument="Coordinates") */ public $coordinates; /** @Distance */ public $distance; } class Coordinates { public $lat; public $long; }
  • 37.
    Geospatial Queries 2 Executea Geospatial query and find locations near a point: <?php /* @var $dm DocumentManager */ $cities = $dm->createQuery('City') ->field('coordinates')->near(50, 60) ->execute();
  • 38.
    Eventual Migration Handle simpleand complex schema refactorings <?php class Person { public $id; /** @AlsoLoad("name") */ public $fullName; }
  • 39.
    More of DoctrineMongoDB Support for Trees Support for Files in MongoGridFS Capped Collections Tailable Cursors
  • 40.
    PHPCR ODM PHP ContentRepository Project Port of the Java Content Repository API PHP API for Apache Jackrabbit Doctrine PHPCR ODMs creates objects from Content Repositories.
  • 41.
    Object to XMLMapper Convert objects to XML documents and back <?php $user = new User(); $user->setFirstName('John'); $user->setLastName('Doe'); $user->setAddress(new Address('123 Street', 'New Haven')); $user->addContact(new CustomerContact('no@way.com')); $xml = $marshaller->marshalToString($user); $user = $marshaller->unmarshalFromString($xml);
  • 42.
    Using Doctrine 2.0.xORM? Please checkout 2.1 BETA1 It is backwards compatible You win a present if you can prove otherwise.
  • 43.
    Thank you! Rate thistalk: http://joind.in/talk/view/3515