Doctrine NoSQL
Benjamin Eberlei (SimpleThings GmbH)
About me
 Benjamin Eberlei
 Working at SimpleThings GmbH
   http://www.simplethings.de
 Open Source contributor
   Doctrine2, Symfony2
   (Zeta Components, PHPUnit, ...)
 Twitter @beberlei
 Blog http://www.whitewashing.de
The Doctrine Project
www.doctrine-project.org

The Doctrine Project is the home of a selected
set of PHP libraries primarily focused on
providing persistence services and related
functionality.
Doctrine Subprojects
 DBAL and ORM
 Document Mapper (MongoDB, CouchDB, PHPCR)
 Annotations
 XML
 What is next?
Doctrine Philosophy
        Separate Persistence and Model
Doctrine Philosophy
            Similar look and feel
Doctrine Philosophy
            Embrace Differences
Why NoSQL Mapper?
Schemaless storage allows:

  Arbitrary associations
  Embedded objects
  Lists and Associative Arrays

No duplicate schema-maintenance!
Doctrine NoSQL History
 MongoDB Mapper early 2010 (OpenSky)
 CouchDB Mapper started in October 2010 (Liip)
 PHPCR ODM started in early 2011 (Liip)
 APIs heavily inspired from ORM
SQL and NoSQL Similarities
 Extracted common persistence interfaces
 Covering roughly 10-20% of the use-cases
   Simple Finder Methods
   Insert/Update/Delete
   Metadata API
 Support for Annotations/XML/YAML/PHP Mapping
Persistence Interfaces
<?php
interface ObjectManager
{
    function find($class, $id);
    function getReference($class, $id);
    function persist($object);
    function remove($object);
    function flush();

    function getClassMetadata($class);
    function getRepository($class);
}
Persistence Interfaces
<?php
interface ObjectRepository
{
    function find($id);
    function findAll();
    function findBy(array $criteria,
        $orderBy = null,
        $limit = null,
        $offset = null
    )
    function findOneBy(array $criteria);
}
Sample Document
<?php
/** @Document */
class Message
{
    /** @Id */
    public $id;
    /** @Field(type="string") */
    public $text;
}

$message = new Message();
$message->setText("Hello World!");
NoSQL benefits
<?php
/** @Document */
class Product
{
    /** other fields */
    /** @Field(type="array") */
    public $attributes;
    /** @Field(type="array") */
    public $translations;
}

$product->attributes["isbn"] = "A-B-C-D";
$product->translations["de"]["name"] = "Ein Produkt";
Working with Objects 1
Creating a new document:

<?php
/** @var $dm DocumentManager */
$message = new Message();
$message->setText("I am new!");

$dm->persist($message);
$dm->flush();

echo "ID: " . $message->getId();
Working with Objects 2
Find and update document:

<?php
/** @var $dm DocumentManager */
$message = $dm->find("Message", 1);
$message->setText("New Message");
$dm->flush();
Working with Objects 3
Find and remove documents:

<?php
/** @var $dm DocumentManager */
$repository = $dm->getRepository("User");
$criteria = array("status" => "inactive");
$users = $repository->findBy($criteria);

foreach ($users AS $user) {
    $dm->remove($user);
}
$dm->flush();
Persistence API Use-Cases
 Focus on "in memory" object workflows
 Specialized reusable Modules
 Symfony2:
   User Management
   Comment
   Admin Generators
   lichess.org
Associations in NoSQL
Pros
 Embedded Documents
 References between arbitrary types

Cons
 No referential integrity
 No support for transactions
Association Mappings
<?php
/** @Document */
class Blog
{
    /** @ReferenceMany */
    private $articles;
    /** @ReferenceOne(targetDocument="User") */
    private $owner;
}
Association keys
<?php
$id = "1";
$articleSlug = "hello-world";

$blog = $dm->find("Blog", $id);
$blog->articles[$articleSlug]->getHeadline();
Embedded Mappings
<?php
/** @Document */
class User
{
    /** @EmbedMany */
    private $phonenumbers;
    /** @EmbedOne(targetDocument="Address") */
    private $address;
}
CouchDB and Doctrine
 JSON Datastorage
 HTTP/REST API
 MVCC, eventually consistent (Conflicts)
 Replication
 Attachments
 Views and Map/Reduce in Javascript
 CouchDB Lucene
 Doctrine CouchDB 1.0 Alpha 1
JSON Document
{
    "_id": "716104ac33c797b12d50c0a6483f1661",
    "_rev": "1-32db404b78f130fd8f7575905859e19b",
    "doctrine_metadata":
    {
        "type": "MyProject.Document.Message",
        "associations":
        {
            "user": "055fe8a3ab06c3998d27b6d99f5a9bdd"
        }
    },
    "message": "I am a message"
}
Document Version
Implement Optimistic-Locking

<?php
class Article
{
    /** @Version */
    private $version;
}

$article = $dm->find(
    "Article", $id, $expectedVersion
);
Attachments
 CouchDB supports Attachments to documents
 Doctrine converts into Attachment object
 Lazy Load binary data from the server
 Stream support planned

<?php
class Article
{
    /** @Attachments */
    public $attachments = array();
}
Attachments 2
<?php
use DoctrineCouchDBAttachment;

$article = $dm->find("Article", 1);
$data = $article->attachments["teaser.jpg"]->getContent();

$a = Attachment::createFromBase64data($data, "image/jpg");
$article->attachments["author.jpg"] = $a;

$dm->flush();
Views
Doctrine CouchDB maps filesystem to design document:

application/
    couchdb/
        views/
             username/
                 map.js
                 reduce.js

Use javascript syntax highlighting in your IDE/Editor.
Views 2
<?php
use DoctrineCouchDBViewFileFolderDesignDocument;

$path = "/path/application/couchdb";
$designDoc = new FileFolderDesignDocument($path);

/* @doc $couch CouchClient */
$docName = "myapp";
$couch->createDesignDoc($docName, $designDoc);
Query Views
<?php
/* @var $dm DocumentManager */
$query = $dm->createQuery("myapp", "username");
$result = $query->setStartKey("b")
                ->setEndKey("c")
                ->setLimit(10)
                ->setSkip(10)
                ->includeDocs(true)
                ->execute();

Using include docs creates PHP instances for you.
Lucene Queries
Support for the CouchDB Lucene extension:

<?php
$query = $dm->createLuceneQuery("lucenedoc", "users");
$result = $query->setQuery('"John Galt" OR "John Wayne"')
                ->setLimit(10)
                ->setSkip(10)
                ->includeDocs(true)
                ->execute();
MongoDB and Doctrine
 Indexing and on the fly
 queries
 Very fast
 In-Place Updates²
 GridFS, Geolocation
 Sharding
 Doctrine MongoDB 1.0 Beta2
Complex Associations
<?php
class User
{
    /**
      * @ReferenceMany(
      *    targetDocument="Comment",
      *    mappedBy="blogPost",
      *    sort={"date"="desc"},
      *    limit=5)
      */
    private $last5Comments;
}
Query API
<?php
$qb = $dm->createQueryBuilder('User')
      ->field('groups')
      ->all(array('Group 1', 'Group 2'))
      ->sort("username", "asc")
      ->limit(10)
      ->skip(10)
      ->execute();
Map/Reduce
<?php
$qb = $dm->createQueryBuilder('DocumentsUser')
    ->field('type')->equals('sale')
    ->map('function() { emit(this.user.$id, 1); }')
    ->reduce('function(k, vals) {
         var sum = 0;
         for (var i in vals) {
             sum += vals[i];
         }
         return sum;
    }');
Geospatial Queries
<?php
/** @Document @Index(keys={"coordinates"="2d"}) */
class City
{
    /** @EmbedOne(targetDocument="Coordinates") */
    public $coordinates;
    /** @Distance */
    public $distance;
}
class Coordinates
{
    public $lat;
    public $long;
}
Geospatial Queries 2
Execute a Geospatial query and find locations near a point:

<?php
/* @var $dm DocumentManager */
$cities = $dm->createQuery('City')
    ->field('coordinates')->near(50, 60)
    ->execute();
Eventual Migration
Handle simple and complex schema refactorings

<?php
/** @Document */
class Person
{
    public $id;

     public $name; // old

     /** @AlsoLoad("name") */
     public $fullName;
}
More of Doctrine MongoDB
 Support for Trees
 Support for Files in MongoGridFS
 Capped Collections
 Tailable Cursors
PHPCR ODM
PHPCR: Port of the Java Content Repository API
Jackalope: Access to Apache Jackrabbit in PHP
Doctrine PHPCR ODM: PHP objects from PHP Content Repositories
Object to XML Mapper
Convert objects to XML documents and back using metadata

<?php
$user = new User();
$user->setFirstName('John');
$user->setLastName('Doe');
$user->setAddress(new Address('123 Street', 'New Haven'));
$user->addContact(new CustomerContact('no@way.com'));

$xml = $marshaller->marshalToString($user);
$user = $marshaller->unmarshalFromString($xml);
Using Doctrine 2.0.x ORM?
Please checkout 2.1 BETA1
 Backwards compatible!
 You win a present if you can prove otherwise.
Thank you!
Rate this talk:

http://joind.in/talk/view/3515

Doctrine and NoSQL

  • 1.
  • 2.
    About me BenjaminEberlei Working at SimpleThings GmbH http://www.simplethings.de Open Source contributor Doctrine2, Symfony2 (Zeta Components, PHPUnit, ...) Twitter @beberlei Blog http://www.whitewashing.de
  • 3.
    The Doctrine Project www.doctrine-project.org TheDoctrine Project is the home of a selected set of PHP libraries primarily focused on providing persistence services and related functionality.
  • 4.
    Doctrine Subprojects DBALand ORM Document Mapper (MongoDB, CouchDB, PHPCR) Annotations XML What is next?
  • 5.
    Doctrine Philosophy Separate Persistence and Model
  • 6.
    Doctrine Philosophy Similar look and feel
  • 7.
    Doctrine Philosophy Embrace Differences
  • 8.
    Why NoSQL Mapper? Schemalessstorage allows: Arbitrary associations Embedded objects Lists and Associative Arrays No duplicate schema-maintenance!
  • 9.
    Doctrine NoSQL History MongoDB Mapper early 2010 (OpenSky) CouchDB Mapper started in October 2010 (Liip) PHPCR ODM started in early 2011 (Liip) APIs heavily inspired from ORM
  • 10.
    SQL and NoSQLSimilarities Extracted common persistence interfaces Covering roughly 10-20% of the use-cases Simple Finder Methods Insert/Update/Delete Metadata API Support for Annotations/XML/YAML/PHP Mapping
  • 11.
    Persistence Interfaces <?php interface ObjectManager { function find($class, $id); function getReference($class, $id); function persist($object); function remove($object); function flush(); function getClassMetadata($class); function getRepository($class); }
  • 12.
    Persistence Interfaces <?php interface ObjectRepository { function find($id); function findAll(); function findBy(array $criteria, $orderBy = null, $limit = null, $offset = null ) function findOneBy(array $criteria); }
  • 13.
    Sample Document <?php /** @Document*/ class Message { /** @Id */ public $id; /** @Field(type="string") */ public $text; } $message = new Message(); $message->setText("Hello World!");
  • 14.
    NoSQL benefits <?php /** @Document*/ class Product { /** other fields */ /** @Field(type="array") */ public $attributes; /** @Field(type="array") */ public $translations; } $product->attributes["isbn"] = "A-B-C-D"; $product->translations["de"]["name"] = "Ein Produkt";
  • 15.
    Working with Objects1 Creating a new document: <?php /** @var $dm DocumentManager */ $message = new Message(); $message->setText("I am new!"); $dm->persist($message); $dm->flush(); echo "ID: " . $message->getId();
  • 16.
    Working with Objects2 Find and update document: <?php /** @var $dm DocumentManager */ $message = $dm->find("Message", 1); $message->setText("New Message"); $dm->flush();
  • 17.
    Working with Objects3 Find and remove documents: <?php /** @var $dm DocumentManager */ $repository = $dm->getRepository("User"); $criteria = array("status" => "inactive"); $users = $repository->findBy($criteria); foreach ($users AS $user) { $dm->remove($user); } $dm->flush();
  • 18.
    Persistence API Use-Cases Focus on "in memory" object workflows Specialized reusable Modules Symfony2: User Management Comment Admin Generators lichess.org
  • 19.
    Associations in NoSQL Pros Embedded Documents References between arbitrary types Cons No referential integrity No support for transactions
  • 20.
    Association Mappings <?php /** @Document*/ class Blog { /** @ReferenceMany */ private $articles; /** @ReferenceOne(targetDocument="User") */ private $owner; }
  • 21.
    Association keys <?php $id ="1"; $articleSlug = "hello-world"; $blog = $dm->find("Blog", $id); $blog->articles[$articleSlug]->getHeadline();
  • 22.
    Embedded Mappings <?php /** @Document*/ class User { /** @EmbedMany */ private $phonenumbers; /** @EmbedOne(targetDocument="Address") */ private $address; }
  • 23.
    CouchDB and Doctrine JSON Datastorage HTTP/REST API MVCC, eventually consistent (Conflicts) Replication Attachments Views and Map/Reduce in Javascript CouchDB Lucene Doctrine CouchDB 1.0 Alpha 1
  • 24.
    JSON Document { "_id": "716104ac33c797b12d50c0a6483f1661", "_rev": "1-32db404b78f130fd8f7575905859e19b", "doctrine_metadata": { "type": "MyProject.Document.Message", "associations": { "user": "055fe8a3ab06c3998d27b6d99f5a9bdd" } }, "message": "I am a message" }
  • 25.
    Document Version Implement Optimistic-Locking <?php classArticle { /** @Version */ private $version; } $article = $dm->find( "Article", $id, $expectedVersion );
  • 26.
    Attachments CouchDB supportsAttachments to documents Doctrine converts into Attachment object Lazy Load binary data from the server Stream support planned <?php class Article { /** @Attachments */ public $attachments = array(); }
  • 27.
    Attachments 2 <?php use DoctrineCouchDBAttachment; $article= $dm->find("Article", 1); $data = $article->attachments["teaser.jpg"]->getContent(); $a = Attachment::createFromBase64data($data, "image/jpg"); $article->attachments["author.jpg"] = $a; $dm->flush();
  • 28.
    Views Doctrine CouchDB mapsfilesystem to design document: application/ couchdb/ views/ username/ map.js reduce.js Use javascript syntax highlighting in your IDE/Editor.
  • 29.
    Views 2 <?php use DoctrineCouchDBViewFileFolderDesignDocument; $path= "/path/application/couchdb"; $designDoc = new FileFolderDesignDocument($path); /* @doc $couch CouchClient */ $docName = "myapp"; $couch->createDesignDoc($docName, $designDoc);
  • 30.
    Query Views <?php /* @var$dm DocumentManager */ $query = $dm->createQuery("myapp", "username"); $result = $query->setStartKey("b") ->setEndKey("c") ->setLimit(10) ->setSkip(10) ->includeDocs(true) ->execute(); Using include docs creates PHP instances for you.
  • 31.
    Lucene Queries Support forthe CouchDB Lucene extension: <?php $query = $dm->createLuceneQuery("lucenedoc", "users"); $result = $query->setQuery('"John Galt" OR "John Wayne"') ->setLimit(10) ->setSkip(10) ->includeDocs(true) ->execute();
  • 32.
    MongoDB and Doctrine Indexing and on the fly queries Very fast In-Place Updates² GridFS, Geolocation Sharding Doctrine MongoDB 1.0 Beta2
  • 33.
    Complex Associations <?php class User { /** * @ReferenceMany( * targetDocument="Comment", * mappedBy="blogPost", * sort={"date"="desc"}, * limit=5) */ private $last5Comments; }
  • 34.
    Query API <?php $qb =$dm->createQueryBuilder('User') ->field('groups') ->all(array('Group 1', 'Group 2')) ->sort("username", "asc") ->limit(10) ->skip(10) ->execute();
  • 35.
    Map/Reduce <?php $qb = $dm->createQueryBuilder('DocumentsUser') ->field('type')->equals('sale') ->map('function() { emit(this.user.$id, 1); }') ->reduce('function(k, vals) { var sum = 0; for (var i in vals) { sum += vals[i]; } return sum; }');
  • 36.
    Geospatial Queries <?php /** @Document@Index(keys={"coordinates"="2d"}) */ class City { /** @EmbedOne(targetDocument="Coordinates") */ public $coordinates; /** @Distance */ public $distance; } class Coordinates { public $lat; public $long; }
  • 37.
    Geospatial Queries 2 Executea Geospatial query and find locations near a point: <?php /* @var $dm DocumentManager */ $cities = $dm->createQuery('City') ->field('coordinates')->near(50, 60) ->execute();
  • 38.
    Eventual Migration Handle simpleand complex schema refactorings <?php /** @Document */ class Person { public $id; public $name; // old /** @AlsoLoad("name") */ public $fullName; }
  • 39.
    More of DoctrineMongoDB Support for Trees Support for Files in MongoGridFS Capped Collections Tailable Cursors
  • 40.
    PHPCR ODM PHPCR: Portof the Java Content Repository API Jackalope: Access to Apache Jackrabbit in PHP Doctrine PHPCR ODM: PHP objects from PHP Content Repositories
  • 41.
    Object to XMLMapper Convert objects to XML documents and back using metadata <?php $user = new User(); $user->setFirstName('John'); $user->setLastName('Doe'); $user->setAddress(new Address('123 Street', 'New Haven')); $user->addContact(new CustomerContact('no@way.com')); $xml = $marshaller->marshalToString($user); $user = $marshaller->unmarshalFromString($xml);
  • 42.
    Using Doctrine 2.0.xORM? Please checkout 2.1 BETA1 Backwards compatible! You win a present if you can prove otherwise.
  • 43.
    Thank you! Rate thistalk: http://joind.in/talk/view/3515