Being RDBMS Free
DAVID HOERSTER
2014
About Me
C# MVP (Since April 2011)
Sr. Director of Web Solutions at RGP
Conference Director for Pittsburgh TechFest
Co-Fou...
Traditional Architecture
Data persistence is central to application
Generally monolithic
Jack of all trades; master of none
Traditional Architecture
Client
Web
Server
App
Server
Data
Repository
Data Tables
Session
Cache (?)
FT Search
What if…
We could break some pieces out
◦ Flatten structures for querying
◦ Highly efficient search services
◦ Pub/sub hub...
Search
How do you search?
◦ LIKE ‘%blah%’ ?
◦ Dynamic SQL
◦ Full-Text
LIKE and Dynamic SQL can be quick to create
◦ Tough ...
Search
Number of search services out there like
◦ Lucene
◦ Solr
Lucene is a search engine
◦ Embed in apps
◦ .NET port (Luc...
Searching with Solr
Disconnected from your application
Search content via HTTP REST calls
Can use SolrNet as a client
◦ ht...
Searching with Solr
private readonly ISolrOperations<T> _solr;
public SolrSearchProvider(ISolrOperations<T> solr) { _solr ...
Evolving Architecture
Client
Web
Server
App
Server
Data
Repository
Data Tables
Session
Cache (?)
Search
Service
Query
Write
Data Storage
Typically, RDBMS is the de facto standard
◦ SQL Server
◦ MySQL
◦ PostgreSQL
◦ Oracle (Yikes!!)
But do you rea...
Data Storage
Get all the orders for user ‘David’ in last 30 days
SELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOr...
Data Storage
Wouldn’t it be great if it were something like this?
SELECT FirstName, MiddleName, LastName, SalesOrderID, Or...
Data Storage
Maybe a document database can be of use
Number out there
◦ MongoDB
◦ RavenDB
◦ Couchbase
Flattened structures...
Looking at MongoDB
Server can have databases
Databases contain collections (like a table)
Collections contain documents (l...
Working with Mongo’s C# Client
public class MongoContext<T> : IContext<T> where T : class, new() {
private IDictionary<Str...
Working with Mongo’s C# Client
Encapsulate my queries and commands
public class FindPageById : ICriteria<Page> {
private r...
Working with Mongo’s C# Client
Invoke my query/command
public class TemplateController : IportalBaseController {
private r...
Working with Mongo’s C# Client
Writing to Mongo is just as simple...
[HttpPost]
public async Task<Boolean> Post(Page page)...
Evolving Architecture
Client
Web
Server
App
Server
Data
Repository
Some data (?)
Session
Cache (?)
Search
Service
Query
Wr...
Session and Cache Data
Generally short-lived for users
Fairly static for cached data
Key/value stores can serve us well he...
Using Redis
public class RedisSessionManager : ISessionManager {
private static ConnectionMultiplexer _redis = null;
priva...
Using Redis
public async Task<Boolean> ExtendSession(String portalId) {
var timeout = _config.ContainsKey("session.timeout...
Using Redis
At login:
await Session.CreateSession(portalid, UserHandle, fullName);
Upon log out:
await Session.ExpireSessi...
Evolving Architecture
Client
Web
Server
App
Server
Data
Repository
Some data (?)
Search
Service
Query
Write
Document
Repos...
Why Data Store
We’re left with a database with not much use
◦ Transactional data in document store
◦ Search documents in S...
Evolving Architecture
Client
Web
Server
App
Server
Event Store
2-3 flat tables
Event data
Search
Service
Query
Write
Docum...
(D)Evolved Architecture
Client
Web
Server
App
Server
Event
Store
Search
Service
Query
Write
Doc
Repo
Write
Query
Session/
...
(D)Evolved Architecture
Pick and choose what components work best
Don’t use them just to use them
Proof-of-Concept / Proto...
Why look to be RDBMS free
Searching
◦ More than just full-text needs
Data
◦ Choose a system that you can model the busines...
Upcoming SlideShare
Loading in …5
×

Freeing Yourself from an RDBMS Architecture

777 views

Published on

Explore how we can begin to move functionality from a typical RDBMS application to one that uses tools and frameworks like MongoDB, Solr and Redis. At the end, the architecture we've evolved looks similar to.........

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
777
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Freeing Yourself from an RDBMS Architecture

  1. 1. Being RDBMS Free DAVID HOERSTER 2014
  2. 2. About Me C# MVP (Since April 2011) Sr. Director of Web Solutions at RGP Conference Director for Pittsburgh TechFest Co-Founder of BrainCredits (braincredits.com) Past President of Pittsburgh .NET Users Group and organizer of recent Pittsburgh Code Camps and other Tech Events Twitter - @DavidHoerster Blog – http://geekswithblogs.net/DavidHoerster Email – david@agileways.com
  3. 3. Traditional Architecture Data persistence is central to application Generally monolithic Jack of all trades; master of none
  4. 4. Traditional Architecture Client Web Server App Server Data Repository Data Tables Session Cache (?) FT Search
  5. 5. What if… We could break some pieces out ◦ Flatten structures for querying ◦ Highly efficient search services ◦ Pub/sub hubs ◦ Remote caching with excellent performance ◦ Session management outside a DB for load balanced environments How would app then be architected?
  6. 6. Search How do you search? ◦ LIKE ‘%blah%’ ? ◦ Dynamic SQL ◦ Full-Text LIKE and Dynamic SQL can be quick to create ◦ Tough to maintain Full-Text gives power ◦ Limited in search options
  7. 7. Search Number of search services out there like ◦ Lucene ◦ Solr Lucene is a search engine ◦ Embed in apps ◦ .NET port (Lucene.NET) Solr is search service ◦ Built on Lucene ◦ Connect apps to it
  8. 8. Searching with Solr Disconnected from your application Search content via HTTP REST calls Can use SolrNet as a client ◦ https://github.com/mausch/SolrNet Document-based
  9. 9. Searching with Solr private readonly ISolrOperations<T> _solr; public SolrSearchProvider(ISolrOperations<T> solr) { _solr = solr; } public IEnumerable<T> Query(String searchString) { var options = new QueryOptions() { Fields = new List<String> {"title", "body", "lastModified" }.ToArray(), Highlight = new HighlightingParameters() { BeforeTerm = "<strong><em>", AfterTerm = "</em></strong>", Fields = new List<String> { "title", "body" }.ToArray(), Fragsize = 100 } }; var results = _solr.Query(new SolrQuery(searchString), options); return results; }
  10. 10. Evolving Architecture Client Web Server App Server Data Repository Data Tables Session Cache (?) Search Service Query Write
  11. 11. Data Storage Typically, RDBMS is the de facto standard ◦ SQL Server ◦ MySQL ◦ PostgreSQL ◦ Oracle (Yikes!!) But do you really need it?
  12. 12. Data Storage Get all the orders for user ‘David’ in last 30 days SELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOrderID, soh.OrderDate, sod.UnitPrice, sod.OrderQty, sod.LineTotal, p.Name as 'ProductName', p.Color, p.ProductNumber, pm.Name as 'ProductModel', pc.Name as 'ProductCategory', pcParent.Name as 'ProductParentCategory' FROM SalesLT.Customer c INNER JOIN SalesLT.SalesOrderHeader soh ON c.CustomerID = soh.CustomerID INNER JOIN SalesLT.SalesOrderDetail sod ON soh.SalesOrderID = sod.SalesOrderID INNER JOIN SalesLT.Product p ON sod.ProductID = p.ProductID INNER JOIN SalesLT.ProductModel pm ON p.ProductModelID = pm.ProductModelID INNER JOIN SalesLT.ProductCategory pc ON p.ProductCategoryID = pc.ProductCategoryID INNER JOIN SalesLT.ProductCategory pcParent ON pc.ParentProductCategoryID = pcParent.ProductCategoryID WHERE c.FirstName = 'David' AND soh.OrderDate > (GETDATE()-30)
  13. 13. Data Storage Wouldn’t it be great if it were something like this? SELECT FirstName, MiddleName, LastName, SalesOrderID, OrderDate, UnitPrice, OrderQty, LineTotal, ProductName, Color, ProductNumber, ProductModel, ProductCategory, ProductParentCategory FROM CustomerSales WHERE FirstName = 'David' AND OrderDate > (GETDATE()-30)
  14. 14. Data Storage Maybe a document database can be of use Number out there ◦ MongoDB ◦ RavenDB ◦ Couchbase Flattened structures without relational ties to other collections Essentially object databases
  15. 15. Looking at MongoDB Server can have databases Databases contain collections (like a table) Collections contain documents (like rows) Documents can be structured, have constraints, have primary key
  16. 16. Working with Mongo’s C# Client public class MongoContext<T> : IContext<T> where T : class, new() { private IDictionary<String, String> _config; private readonly MongoCollection<T> _coll; public MongoContext(IDictionary<String, String> config) { _config = config; var client = new MongoClient(config["mongo.serverUrl"]); var server = client.GetServer(); var database = server.GetDatabase(config["mongo.database"]); _coll = database.GetCollection<T>(config["mongo.collection"]); } public IQueryable<T> Items { get { return _coll.FindAll().AsQueryable(); } } }
  17. 17. Working with Mongo’s C# Client Encapsulate my queries and commands public class FindPageById : ICriteria<Page> { private readonly String _id; public FindPageById(String pageId) { _id = pageId; } public IEnumerable<Page> Execute(IContext<Page> ctx) { return ctx.Items.Where(p => p.Id == _id).AsEnumerable(); } }
  18. 18. Working with Mongo’s C# Client Invoke my query/command public class TemplateController : IportalBaseController { private readonly IContext<Page> _pages; public TemplateController(IContext<Page> ctx) : base() { _pages = ctx; } [HttpGet] public async Task<IportalPageMetadata> Section(String cat, String page) { var id = String.Format("{0}/{1}", cat, page); var thePage = new FindPageById(id) .Execute(_pages) .FirstOrDefault(); ... } }
  19. 19. Working with Mongo’s C# Client Writing to Mongo is just as simple... [HttpPost] public async Task<Boolean> Post(Page page) { var userId = await GetUserId(); new CreatePage(page, userId) .Execute(_pages); _searchPage.Insert(page); return true; }
  20. 20. Evolving Architecture Client Web Server App Server Data Repository Some data (?) Session Cache (?) Search Service Query Write Document Repository Write Query
  21. 21. Session and Cache Data Generally short-lived for users Fairly static for cached data Key/value stores can serve us well here ◦ Redis Redis has two good .NET client libraries ◦ StackExchange.Redis ◦ ServiceStack.Redis
  22. 22. Using Redis public class RedisSessionManager : ISessionManager { private static ConnectionMultiplexer _redis = null; private readonly IDictionary<String, String> _config; public RedisSessionManager(IDictionary<String, String> config) { if (_redis == null) { _redis = ConnectionMultiplexer.Connect(config["session.serverUrl"].ToString()); } _config = config; } public async Task<Boolean> CreateSession(String portalId, String userId, String fullName) { var time = DateTime.UtcNow.ToString(); var timeout = _config.ContainsKey("session.timeout"); var vals = new HashEntry[] { new HashEntry("userid", userId), new HashEntry("login", time), new HashEntry("lastAction", time), new HashEntry("fullName", fullName) }; await RedisDatabase.HashSetAsync(portalId, vals); return await RedisDatabase.KeyExpireAsync(portalId, TimeSpan.FromMinutes(timeout)); } }
  23. 23. Using Redis public async Task<Boolean> ExtendSession(String portalId) { var timeout = _config.ContainsKey("session.timeout"); await RedisDatabase.HashSetAsync(portalId, "lastAction", DateTime.UtcNow.ToString()); return await RedisDatabase.KeyExpireAsync(portalId, TimeSpan.FromMinutes(timeout)); } public async Task<Boolean> ExpireSession(String portalId) { return await RedisDatabase.KeyDeleteAsync(portalId); }
  24. 24. Using Redis At login: await Session.CreateSession(portalid, UserHandle, fullName); Upon log out: await Session.ExpireSession(portalCookie.Value);
  25. 25. Evolving Architecture Client Web Server App Server Data Repository Some data (?) Search Service Query Write Document Repository Write Query Session/ Cache Service
  26. 26. Why Data Store We’re left with a database with not much use ◦ Transactional data in document store ◦ Search documents in Solr ◦ Session, caching, etc. in key/value or caching service like Redis What it probably ends up acting as is…
  27. 27. Evolving Architecture Client Web Server App Server Event Store 2-3 flat tables Event data Search Service Query Write Document Repository Write Query Session/ Cache Service Queue?
  28. 28. (D)Evolved Architecture Client Web Server App Server Event Store Search Service Query Write Doc Repo Write Query Session/ Cache Service Queue?
  29. 29. (D)Evolved Architecture Pick and choose what components work best Don’t use them just to use them Proof-of-Concept / Prototype
  30. 30. Why look to be RDBMS free Searching ◦ More than just full-text needs Data ◦ Choose a system that you can model the business ◦ Not the other way around Caching / Session Values / PubSub ◦ Offload necessary? ◦ Ensure performance Maintenance and support big factors to consider

×