Freeing Yourself from an RDBMS Architecture
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Freeing Yourself from an RDBMS Architecture

  • 402 views
Uploaded on

Explore how we can begin to move functionality from a typical RDBMS application to one that uses tools and frameworks like MongoDB, Solr and Redis. At the end, the architecture we've evolved looks......

Explore how we can begin to move functionality from a typical RDBMS application to one that uses tools and frameworks like MongoDB, Solr and Redis. At the end, the architecture we've evolved looks similar to.........

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
402
On Slideshare
387
From Embeds
15
Number of Embeds
1

Actions

Shares
Downloads
6
Comments
0
Likes
1

Embeds 15

https://twitter.com 15

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Being RDBMS Free DAVID HOERSTER 2014
  • 2. About Me C# MVP (Since April 2011) Sr. Director of Web Solutions at RGP Conference Director for Pittsburgh TechFest Co-Founder of BrainCredits (braincredits.com) Past President of Pittsburgh .NET Users Group and organizer of recent Pittsburgh Code Camps and other Tech Events Twitter - @DavidHoerster Blog – http://geekswithblogs.net/DavidHoerster Email – david@agileways.com
  • 3. Traditional Architecture Data persistence is central to application Generally monolithic Jack of all trades; master of none
  • 4. Traditional Architecture Client Web Server App Server Data Repository Data Tables Session Cache (?) FT Search
  • 5. What if… We could break some pieces out ◦ Flatten structures for querying ◦ Highly efficient search services ◦ Pub/sub hubs ◦ Remote caching with excellent performance ◦ Session management outside a DB for load balanced environments How would app then be architected?
  • 6. Search How do you search? ◦ LIKE ‘%blah%’ ? ◦ Dynamic SQL ◦ Full-Text LIKE and Dynamic SQL can be quick to create ◦ Tough to maintain Full-Text gives power ◦ Limited in search options
  • 7. Search Number of search services out there like ◦ Lucene ◦ Solr Lucene is a search engine ◦ Embed in apps ◦ .NET port (Lucene.NET) Solr is search service ◦ Built on Lucene ◦ Connect apps to it
  • 8. Searching with Solr Disconnected from your application Search content via HTTP REST calls Can use SolrNet as a client ◦ https://github.com/mausch/SolrNet Document-based
  • 9. Searching with Solr private readonly ISolrOperations<T> _solr; public SolrSearchProvider(ISolrOperations<T> solr) { _solr = solr; } public IEnumerable<T> Query(String searchString) { var options = new QueryOptions() { Fields = new List<String> {"title", "body", "lastModified" }.ToArray(), Highlight = new HighlightingParameters() { BeforeTerm = "<strong><em>", AfterTerm = "</em></strong>", Fields = new List<String> { "title", "body" }.ToArray(), Fragsize = 100 } }; var results = _solr.Query(new SolrQuery(searchString), options); return results; }
  • 10. Evolving Architecture Client Web Server App Server Data Repository Data Tables Session Cache (?) Search Service Query Write
  • 11. Data Storage Typically, RDBMS is the de facto standard ◦ SQL Server ◦ MySQL ◦ PostgreSQL ◦ Oracle (Yikes!!) But do you really need it?
  • 12. Data Storage Get all the orders for user ‘David’ in last 30 days SELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOrderID, soh.OrderDate, sod.UnitPrice, sod.OrderQty, sod.LineTotal, p.Name as 'ProductName', p.Color, p.ProductNumber, pm.Name as 'ProductModel', pc.Name as 'ProductCategory', pcParent.Name as 'ProductParentCategory' FROM SalesLT.Customer c INNER JOIN SalesLT.SalesOrderHeader soh ON c.CustomerID = soh.CustomerID INNER JOIN SalesLT.SalesOrderDetail sod ON soh.SalesOrderID = sod.SalesOrderID INNER JOIN SalesLT.Product p ON sod.ProductID = p.ProductID INNER JOIN SalesLT.ProductModel pm ON p.ProductModelID = pm.ProductModelID INNER JOIN SalesLT.ProductCategory pc ON p.ProductCategoryID = pc.ProductCategoryID INNER JOIN SalesLT.ProductCategory pcParent ON pc.ParentProductCategoryID = pcParent.ProductCategoryID WHERE c.FirstName = 'David' AND soh.OrderDate > (GETDATE()-30)
  • 13. Data Storage Wouldn’t it be great if it were something like this? SELECT FirstName, MiddleName, LastName, SalesOrderID, OrderDate, UnitPrice, OrderQty, LineTotal, ProductName, Color, ProductNumber, ProductModel, ProductCategory, ProductParentCategory FROM CustomerSales WHERE FirstName = 'David' AND OrderDate > (GETDATE()-30)
  • 14. Data Storage Maybe a document database can be of use Number out there ◦ MongoDB ◦ RavenDB ◦ Couchbase Flattened structures without relational ties to other collections Essentially object databases
  • 15. Looking at MongoDB Server can have databases Databases contain collections (like a table) Collections contain documents (like rows) Documents can be structured, have constraints, have primary key
  • 16. Working with Mongo’s C# Client public class MongoContext<T> : IContext<T> where T : class, new() { private IDictionary<String, String> _config; private readonly MongoCollection<T> _coll; public MongoContext(IDictionary<String, String> config) { _config = config; var client = new MongoClient(config["mongo.serverUrl"]); var server = client.GetServer(); var database = server.GetDatabase(config["mongo.database"]); _coll = database.GetCollection<T>(config["mongo.collection"]); } public IQueryable<T> Items { get { return _coll.FindAll().AsQueryable(); } } }
  • 17. Working with Mongo’s C# Client Encapsulate my queries and commands public class FindPageById : ICriteria<Page> { private readonly String _id; public FindPageById(String pageId) { _id = pageId; } public IEnumerable<Page> Execute(IContext<Page> ctx) { return ctx.Items.Where(p => p.Id == _id).AsEnumerable(); } }
  • 18. Working with Mongo’s C# Client Invoke my query/command public class TemplateController : IportalBaseController { private readonly IContext<Page> _pages; public TemplateController(IContext<Page> ctx) : base() { _pages = ctx; } [HttpGet] public async Task<IportalPageMetadata> Section(String cat, String page) { var id = String.Format("{0}/{1}", cat, page); var thePage = new FindPageById(id) .Execute(_pages) .FirstOrDefault(); ... } }
  • 19. Working with Mongo’s C# Client Writing to Mongo is just as simple... [HttpPost] public async Task<Boolean> Post(Page page) { var userId = await GetUserId(); new CreatePage(page, userId) .Execute(_pages); _searchPage.Insert(page); return true; }
  • 20. Evolving Architecture Client Web Server App Server Data Repository Some data (?) Session Cache (?) Search Service Query Write Document Repository Write Query
  • 21. Session and Cache Data Generally short-lived for users Fairly static for cached data Key/value stores can serve us well here ◦ Redis Redis has two good .NET client libraries ◦ StackExchange.Redis ◦ ServiceStack.Redis
  • 22. Using Redis public class RedisSessionManager : ISessionManager { private static ConnectionMultiplexer _redis = null; private readonly IDictionary<String, String> _config; public RedisSessionManager(IDictionary<String, String> config) { if (_redis == null) { _redis = ConnectionMultiplexer.Connect(config["session.serverUrl"].ToString()); } _config = config; } public async Task<Boolean> CreateSession(String portalId, String userId, String fullName) { var time = DateTime.UtcNow.ToString(); var timeout = _config.ContainsKey("session.timeout"); var vals = new HashEntry[] { new HashEntry("userid", userId), new HashEntry("login", time), new HashEntry("lastAction", time), new HashEntry("fullName", fullName) }; await RedisDatabase.HashSetAsync(portalId, vals); return await RedisDatabase.KeyExpireAsync(portalId, TimeSpan.FromMinutes(timeout)); } }
  • 23. Using Redis public async Task<Boolean> ExtendSession(String portalId) { var timeout = _config.ContainsKey("session.timeout"); await RedisDatabase.HashSetAsync(portalId, "lastAction", DateTime.UtcNow.ToString()); return await RedisDatabase.KeyExpireAsync(portalId, TimeSpan.FromMinutes(timeout)); } public async Task<Boolean> ExpireSession(String portalId) { return await RedisDatabase.KeyDeleteAsync(portalId); }
  • 24. Using Redis At login: await Session.CreateSession(portalid, UserHandle, fullName); Upon log out: await Session.ExpireSession(portalCookie.Value);
  • 25. Evolving Architecture Client Web Server App Server Data Repository Some data (?) Search Service Query Write Document Repository Write Query Session/ Cache Service
  • 26. Why Data Store We’re left with a database with not much use ◦ Transactional data in document store ◦ Search documents in Solr ◦ Session, caching, etc. in key/value or caching service like Redis What it probably ends up acting as is…
  • 27. Evolving Architecture Client Web Server App Server Event Store 2-3 flat tables Event data Search Service Query Write Document Repository Write Query Session/ Cache Service Queue?
  • 28. (D)Evolved Architecture Client Web Server App Server Event Store Search Service Query Write Doc Repo Write Query Session/ Cache Service Queue?
  • 29. (D)Evolved Architecture Pick and choose what components work best Don’t use them just to use them Proof-of-Concept / Prototype
  • 30. Why look to be RDBMS free Searching ◦ More than just full-text needs Data ◦ Choose a system that you can model the business ◦ Not the other way around Caching / Session Values / PubSub ◦ Offload necessary? ◦ Ensure performance Maintenance and support big factors to consider