OVERVIEW

Igor Moochnick
Director, Cloud Platforms
BlueMetal Architects
igorm@bluemetal.com
Blog: igorshare.wordpress.com
BlueMetal Capabilities
            FOCUS AREAS                          PLATFORMS                              APPROACH


            Creative & Interactive Services




                                                                                        Deep Discovery & Business Alignment
                                                  Apple, Amazon and Leading Platforms




                                                                                        Agile, Small Teams with High Velocity
            Mobile Applications
            Web & RIA Clients




                                                                                        Integrated Creative & Engineering
            Enterprise Collaboration
            Social Platforms
            Information Management




                                                  Open Source Software
                                                  Microsoft Platforms
            Application Modernization
            Server-side Application Components
            Private, Public or Hybrid Cloud

            Relational Database Engines
            Business Analytics and Insights
            NoSQL Data Platforms
Why work with us?
• Our model is based on a deep connection to our customer’s true needs
     ̵   We are a relationship-driven partner not a transactional services firm
     ̵   Our relationships are based on high integrity, ethics & mutual success
     ̵   Deep discovery “Seek first to understand” – listen, then synthesize
     ̵   We never lead with a solution - first understand the true business problem then
         design the solution that exactly meets the customer’s needs


• Our capabilities approach is unique to marketplace:
     ̵   Senior level, extremely talented individuals that can operate at high velocity
     ̵   Team model where we operate at a high rate of speed: force multiplier
     ̵   Integrated Platform approach where we offer end to end solutions
     ̵   Differentiated Creative connected to Engineering resulting in shipping what we
         design
Sweet Spots

 Web Related Data, such as user sessions, shopping cart, etc.
 Dynamic Entities, such as user-customizable entities, entities
  with a large number of optional fields, etc.
 Persisted View Models

 Large Data Sets - known to scale in excess of 1
  terabyte (on a single machine with thousands writes/sec)
  and trivial to shard the database across multiple machines
Modeling

 Stop thinking relational

 Start thinking about how your data will be used
     Usage scenarios
     Optimize for reads? Writes?

 Think about your domain objects and business logic in native .Net
  (POCO) classes

 Deformalize if needed

 Reference entities to other entities, or collections of them

 Identify aggregate root(s)
Modes of operations

 Windows Service
 Console App (debug mode)
 Embedded
 IIS App (ASP.NET Service)
 On the cloud (Azure, RavenNest)
RavenDB Design Guidelines

 Self optimizing, zero admin
 Eventual consistency
 Extensible
 Fluent simple API
Client API

 Lightweight
 Silverlight
 Embedded
 RavenLight
Sessions

 Manages UoW (Units of Work)
 Cheap to create
 Transactional
 Manages IDs, tracking
Don’t shoot yourself in the foot

 Will get you first 128 results (unbounded result set
  protection)
 Don’t forget to commit your changes by calling
  SaveChanges()
 30 operations per session (client/server chatter protection)
 Automatic batching
 No locking
Advanced API

 2nd level – Session level
 3rd level – DatabaseCommands
  Document store level

  Thin wrapper around HTTP API

  NOT Transactional
Queries

 Strongly typed
 Linq
 Works against Lucene indexes
 Feeds back into index updates
 By design, results may be stale, but you’ll know about it
Indexes

 Dynamic, created on-the-fly - temporary
 Becomes static, if you insist (suggested for production)
 Rich Actions:
    Map
    Reduce
    Transform (live projections)
    Field
    Paging

 Spatial index
Linq Queries
 // Filter by property value
 var posts = from post in session.Query<BlogPost>()
         where post.Subject == "NoSQL Update"
         select post;

 // Filter by range
 var posts = from post in session.Query<BlogPost>()
         where post.NumberOfWords > 50
         select post;

 // Filter by nested (calculated) property
 var posts = from post in session.Query<BlogPost>()
         where post.Comments.Count > 2
         select post;
Queries

 // Filter by property value
 var posts = session.Query<BlogPost>()
         .Where(post => post.Subject == "NoSQL Update");

 // Filter by range
 var posts = session.Query<BlogPost>()
         .Where(post => post.NumberOfWords > 50);

 // Filter by nested (calculated) property
 var posts = session.Query<BlogPost>()
         .Where(post => post.Comments.Count > 2);
Paging

 You can skip and pick any number of entities from the
  returned Query result
 Use Skip() and Take() to control the size of the result set

  // If a Page has 10 records, to get the 4th page -
  var posts = session.Query<BlogPost>()
          .Skip(30)           // Skip 3 pages worth of posts
          .Take(10)           // Select next page
          .ToArray();         // Execute query
Efficient loads

 Improves normalized data access
 Include relevant information with your query results



 // Request to include Author with the Post result
 var post = session.Include<BlogPost>(post => post.Author.Id)
         .Load ("BlogPosts/34");

 var author = session.Load<Author>(post.Author.Id);
BLOBs / Attachments

 Can store and manage large binary BLOBs



  // Store BLOB/Attachment
  byte[] data = new byte[] { ... };
  documentStore.DatabaseCommands.PutAttachment("videos/2",
    null, data, new RavenJObject {{"Title", "Birthday song"}});

  // Get BLOB/Attachment
  var attachment =
     documentStore.DatabaseCommands.GetAttachment("videos/1");
Polymorphism                      For more watch webcast …


 No required inheritance association




// Query against unified index
var result = session.Query<dynamic,
    EverytingByName>();

foreach (dynamic animal in result)
    Console.WriteLine(animal.Name);
Extensibility

 MEF (Managed Extensibility Framework)

 Triggers
     PUT triggers
     DELETE triggers
     Read triggers
     Index update triggers

 Request Responders

 Custom Serialization/Deserialization
Plugins

  // RavenDB plugin
  public class DocSize : RequestResponder
  {
     public override void Respond(IHttpContext context)
     { ... }

      public override string UrlPattern
      { get { return "/blobsize/(.+)"; } }

      public override string[] SupportedVerbs
      { get { return new[] {"GET"}; } }
  }
Bundles

 Drop-in plugins (MEF, .NET)
 Write your own
 Available
    Authentication
    Authorization
    Cascade delete
    Expiration
    Quotas
    Replication
    Versioning
Replication Between Servers

  Implemented as a plug-in (Raven.Bundles.Replication.dll)
   Tracks the server the document was originally written on.
   The replication bundle uses this information to determine if
    a replicated document is conflicting with the existing
    document.
  Supported by the client API
   Detects that an instance is replicating to another set of instances.
   When that instance is down, will automatically shift to the other
    instances.
Replication to Relational DB
   Given this document…




   And this index…




    Gives this table output



                               http://ravendb.net/bundles/index-replication
Sharding


 Sharding refers to horizontal partitioning
  of data across multiple machines.
 The idea is to split the load across many commodity machines,
  instead of buying huge expensive machines.
 Raven has full support for sharding, and you can utilize
  sharding out of the box.
Advanced

 Single document patch
 Attachments (BLOBs)
 Transactions (transaction scope)
 Batching
 Lucene Query (parser syntax)
 Query statistics
Learn More!
 Raven DB Home Page
   http://ravendb.net/

 Raven DB: An Introduction
   http://www.codeproject.com/KB/cs/RavenDBIntro.aspx

 Herding Code 83: Ayende Rahien on RavenDB
   http://herdingcode.com/?p=255

 Raven posts from Ayende Rahien
   http://ayende.com/Blog/category/564.aspx

 Raven posts from Rob Ashton
    http://codeofrob.com/category/13.aspx

 RavenDB - The Lost Session

RavenDB overview

  • 1.
    OVERVIEW Igor Moochnick Director, CloudPlatforms BlueMetal Architects igorm@bluemetal.com Blog: igorshare.wordpress.com
  • 2.
    BlueMetal Capabilities FOCUS AREAS PLATFORMS APPROACH Creative & Interactive Services Deep Discovery & Business Alignment Apple, Amazon and Leading Platforms Agile, Small Teams with High Velocity Mobile Applications Web & RIA Clients Integrated Creative & Engineering Enterprise Collaboration Social Platforms Information Management Open Source Software Microsoft Platforms Application Modernization Server-side Application Components Private, Public or Hybrid Cloud Relational Database Engines Business Analytics and Insights NoSQL Data Platforms
  • 3.
    Why work withus? • Our model is based on a deep connection to our customer’s true needs ̵ We are a relationship-driven partner not a transactional services firm ̵ Our relationships are based on high integrity, ethics & mutual success ̵ Deep discovery “Seek first to understand” – listen, then synthesize ̵ We never lead with a solution - first understand the true business problem then design the solution that exactly meets the customer’s needs • Our capabilities approach is unique to marketplace: ̵ Senior level, extremely talented individuals that can operate at high velocity ̵ Team model where we operate at a high rate of speed: force multiplier ̵ Integrated Platform approach where we offer end to end solutions ̵ Differentiated Creative connected to Engineering resulting in shipping what we design
  • 4.
    Sweet Spots  WebRelated Data, such as user sessions, shopping cart, etc.  Dynamic Entities, such as user-customizable entities, entities with a large number of optional fields, etc.  Persisted View Models  Large Data Sets - known to scale in excess of 1 terabyte (on a single machine with thousands writes/sec) and trivial to shard the database across multiple machines
  • 5.
    Modeling  Stop thinkingrelational  Start thinking about how your data will be used  Usage scenarios  Optimize for reads? Writes?  Think about your domain objects and business logic in native .Net (POCO) classes  Deformalize if needed  Reference entities to other entities, or collections of them  Identify aggregate root(s)
  • 6.
    Modes of operations Windows Service  Console App (debug mode)  Embedded  IIS App (ASP.NET Service)  On the cloud (Azure, RavenNest)
  • 7.
    RavenDB Design Guidelines Self optimizing, zero admin  Eventual consistency  Extensible  Fluent simple API
  • 8.
    Client API  Lightweight Silverlight  Embedded  RavenLight
  • 9.
    Sessions  Manages UoW(Units of Work)  Cheap to create  Transactional  Manages IDs, tracking
  • 10.
    Don’t shoot yourselfin the foot  Will get you first 128 results (unbounded result set protection)  Don’t forget to commit your changes by calling SaveChanges()  30 operations per session (client/server chatter protection)  Automatic batching  No locking
  • 11.
    Advanced API  2ndlevel – Session level  3rd level – DatabaseCommands  Document store level  Thin wrapper around HTTP API  NOT Transactional
  • 12.
    Queries  Strongly typed Linq  Works against Lucene indexes  Feeds back into index updates  By design, results may be stale, but you’ll know about it
  • 13.
    Indexes  Dynamic, createdon-the-fly - temporary  Becomes static, if you insist (suggested for production)  Rich Actions:  Map  Reduce  Transform (live projections)  Field  Paging  Spatial index
  • 14.
    Linq Queries //Filter by property value var posts = from post in session.Query<BlogPost>() where post.Subject == "NoSQL Update" select post; // Filter by range var posts = from post in session.Query<BlogPost>() where post.NumberOfWords > 50 select post; // Filter by nested (calculated) property var posts = from post in session.Query<BlogPost>() where post.Comments.Count > 2 select post;
  • 15.
    Queries // Filterby property value var posts = session.Query<BlogPost>() .Where(post => post.Subject == "NoSQL Update"); // Filter by range var posts = session.Query<BlogPost>() .Where(post => post.NumberOfWords > 50); // Filter by nested (calculated) property var posts = session.Query<BlogPost>() .Where(post => post.Comments.Count > 2);
  • 16.
    Paging  You canskip and pick any number of entities from the returned Query result  Use Skip() and Take() to control the size of the result set // If a Page has 10 records, to get the 4th page - var posts = session.Query<BlogPost>() .Skip(30) // Skip 3 pages worth of posts .Take(10) // Select next page .ToArray(); // Execute query
  • 17.
    Efficient loads  Improvesnormalized data access  Include relevant information with your query results // Request to include Author with the Post result var post = session.Include<BlogPost>(post => post.Author.Id) .Load ("BlogPosts/34"); var author = session.Load<Author>(post.Author.Id);
  • 18.
    BLOBs / Attachments Can store and manage large binary BLOBs // Store BLOB/Attachment byte[] data = new byte[] { ... }; documentStore.DatabaseCommands.PutAttachment("videos/2", null, data, new RavenJObject {{"Title", "Birthday song"}}); // Get BLOB/Attachment var attachment = documentStore.DatabaseCommands.GetAttachment("videos/1");
  • 19.
    Polymorphism For more watch webcast …  No required inheritance association // Query against unified index var result = session.Query<dynamic, EverytingByName>(); foreach (dynamic animal in result) Console.WriteLine(animal.Name);
  • 20.
    Extensibility  MEF (ManagedExtensibility Framework)  Triggers  PUT triggers  DELETE triggers  Read triggers  Index update triggers  Request Responders  Custom Serialization/Deserialization
  • 21.
    Plugins //RavenDB plugin public class DocSize : RequestResponder { public override void Respond(IHttpContext context) { ... } public override string UrlPattern { get { return "/blobsize/(.+)"; } } public override string[] SupportedVerbs { get { return new[] {"GET"}; } } }
  • 22.
    Bundles  Drop-in plugins(MEF, .NET)  Write your own  Available  Authentication  Authorization  Cascade delete  Expiration  Quotas  Replication  Versioning
  • 23.
    Replication Between Servers  Implemented as a plug-in (Raven.Bundles.Replication.dll)  Tracks the server the document was originally written on.  The replication bundle uses this information to determine if a replicated document is conflicting with the existing document.  Supported by the client API  Detects that an instance is replicating to another set of instances.  When that instance is down, will automatically shift to the other instances.
  • 24.
    Replication to RelationalDB Given this document… And this index… Gives this table output http://ravendb.net/bundles/index-replication
  • 25.
    Sharding  Sharding refersto horizontal partitioning of data across multiple machines.  The idea is to split the load across many commodity machines, instead of buying huge expensive machines.  Raven has full support for sharding, and you can utilize sharding out of the box.
  • 26.
    Advanced  Single documentpatch  Attachments (BLOBs)  Transactions (transaction scope)  Batching  Lucene Query (parser syntax)  Query statistics
  • 27.
    Learn More!  RavenDB Home Page http://ravendb.net/  Raven DB: An Introduction http://www.codeproject.com/KB/cs/RavenDBIntro.aspx  Herding Code 83: Ayende Rahien on RavenDB http://herdingcode.com/?p=255  Raven posts from Ayende Rahien http://ayende.com/Blog/category/564.aspx  Raven posts from Rob Ashton http://codeofrob.com/category/13.aspx  RavenDB - The Lost Session

Editor's Notes

  • #5 http://ravendb.net/
  • #14 curl -X PUT http://localhost:8080/indexes/byEmail -d &quot;{ Map: &apos;from doc in docs where doc.email != null select new {doc.email, count = 1 };&apos; , Reduce: &apos;from result in results group result by result.email into g select new { email = g.Key, count = g.Sum(x=&gt;x.count) } &apos;}&quot;
  • #20 http://www.youtube.com/watch?v=uk2TVs-d6sg
  • #24 {   &quot;Destinations&quot;: [     {       &quot;Url&quot;: &quot;http://raven_two:8080/&quot;     },     {       &quot;Url&quot;: &quot;http://raven_three:8080/&quot;     },   ] }
  • #26 http://old.ravendb.net/documentation/docs-shardingvar shards = new Shards{ new DocumentStore { Identifier = &quot;Users&quot;,Url = &quot;http://localhost:8081&quot;, Conventions = {GenerateDocumentKey = user =&gt; &quot;users/&quot; + ((User) user).Name } }, new DocumentStore {Identifier = &quot;Blogs&quot;, Url = &quot;http://localhost:8082&quot;}, new DocumentStore {Identifier = &quot;Posts #1&quot;, Url = &quot;http://localhost:8083&quot;}, new DocumentStore {Identifier = &quot;Posts #2&quot;, Url = &quot;http://localhost:8084&quot;}, new DocumentStore {Identifier = &quot;Posts #3&quot;, Url = &quot;http://localhost:8085&quot;}};varshardStrategy = new ShardStrategy{ShardAccessStrategy = new ParallelShardAccessStrategy(),ShardSelectionStrategy = new BlogShardSelectionStrategy(3),ShardResolutionStrategy = new BlogShardResolutionStrategy(3),};vardocumentStore = new ShardedDocumentStore(shardStrategy, shards);documentStore.Initialize();