Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. DAT336 Connected vs Disconnected Data Access in ADO.NET Pablo Castro Program Manager – ADO.NET Team Microsoft Corporation
  2. 2. Agenda <ul><li>Disconnected is good </li></ul><ul><ul><li>Some connected bits can help </li></ul></ul><ul><li>Balancing connected/disconnected </li></ul><ul><ul><li>Scrolling and streaming </li></ul></ul><ul><ul><li>Custom aggregation </li></ul></ul><ul><ul><li>Incremental load </li></ul></ul><ul><li>Summary </li></ul>
  3. 3. Disconnected? <ul><li>In the context of this talk: </li></ul><ul><ul><li>“Disconnected” apps are online applications that don’t keep database connections open for long periods of time such as the lifetime of a session </li></ul></ul><ul><ul><ul><li>Usually 3-tier applications </li></ul></ul></ul><ul><ul><li>We’re not going to discuss “off-line” applications where a user can operate an application even if the server is not available </li></ul></ul>
  4. 4. Disconnected is Good <ul><li>Simpler to design </li></ul><ul><ul><li>3-tier applications follow the disconnected model naturally </li></ul></ul><ul><li>Simpler to code once the pieces are set </li></ul><ul><ul><li>i.e. data binding, marshalling in 3-tier apps, no connection management for long-lived business objects </li></ul></ul><ul><li>Simpler to make apps scale </li></ul><ul><ul><li>Middle-tier layer easy to scale-out </li></ul></ul><ul><ul><ul><li>It’s harder to scale-up/out database servers </li></ul></ul></ul>
  5. 5. Disconnected ADO.NET <ul><li>ADO.NET has first-class disconnected apps support </li></ul><ul><ul><li>DataSet as a relational data cache </li></ul></ul><ul><ul><ul><li>Plays well with remoting & web services </li></ul></ul></ul><ul><ul><ul><li>Single implementation, no database-specific behavior </li></ul></ul></ul><ul><ul><li>DataAdapter/DataSet provide services for getting, updating and merging data </li></ul></ul><ul><li>Custom types for non-relational </li></ul>
  6. 6. …but too disconnected… <ul><li>Handling large volumes of data </li></ul><ul><ul><li>Batch processing </li></ul></ul><ul><ul><li>Reporting </li></ul></ul><ul><ul><li>Custom aggregation </li></ul></ul><ul><li>Performance-sensitive code-paths </li></ul><ul><ul><li>ASP.NET pages in apps without heavy business logic </li></ul></ul><ul><ul><li>Gateway applications </li></ul></ul>Avoid buffering large portions of data Avoid data-shipping costs
  7. 7. Discrete Objects Scenarios <ul><li>This is the “easy part” </li></ul><ul><ul><li>Stand-alone business objects </li></ul></ul><ul><li>Business object state representation </li></ul><ul><ul><li>DataSets – ADO.NET helps with retrieval, update and change tracking </li></ul></ul><ul><ul><li>Custom objects – model your business entity state using regular classes </li></ul></ul><ul><li>Remoting or webservices enable this scenario easily </li></ul>
  8. 8. Sessions in the Middle Tier Scenarios <ul><li>Per-session state can’t be avoided sometimes </li></ul><ul><ul><li>If needed, keep only in-memory state </li></ul></ul><ul><ul><li>No database connections if at all possible </li></ul></ul><ul><li>Quite common in ASP/ASP.NET apps </li></ul><ul><li>“Sticky sessions” take care of scale-out </li></ul><ul><ul><li>So it’s not that bad if you don’t keep connections or other external resources </li></ul></ul>
  9. 9. Scrolling and Streaming Scenarios <ul><li>In general, handling large results in pieces </li></ul><ul><ul><li>UI: scrolling, paging </li></ul></ul><ul><ul><li>Batch processing: chunking, scanning large results </li></ul></ul><ul><ul><li>Custom aggregation </li></ul></ul><ul><li>Options change depending on each case </li></ul>
  10. 10. Scrolling, Paging <ul><li>Goal is to fetch rows from a large result, a few at a time </li></ul><ul><ul><li>Cursors </li></ul></ul><ul><ul><li>DataAdapter.Fill method </li></ul></ul><ul><ul><li>SQL-based solutions </li></ul></ul>
  11. 11. Scrolling, Paging: Cursors <ul><li>Not available in all databases </li></ul><ul><li>Provide scrolling support </li></ul><ul><ul><li>Sometimes even bi-directional </li></ul></ul><ul><ul><li>No need for extra logic in the application </li></ul></ul><ul><li>Scalability issues </li></ul><ul><ul><li>Require to maintain state </li></ul></ul><ul><ul><ul><li>Database connections, keyset or temporary tables in the database </li></ul></ul></ul><ul><ul><li>Cursor escalation </li></ul></ul><ul><ul><ul><li>Server may need to materialize some/all data </li></ul></ul></ul>
  12. 12. Scrolling, Paging: Cursors <ul><li>Design issues </li></ul><ul><ul><li>Hard to include in 3-tier applications </li></ul></ul><ul><ul><li>Need to keep connection/cursor objects alive in middle tier </li></ul></ul><ul><ul><li>How is data propagated to the presentation layer? </li></ul></ul><ul><ul><ul><li>Tends to be a chatty interface </li></ul></ul></ul><ul><li>Result stability </li></ul><ul><ul><li>Need to use transactions or static cursors if stability is required </li></ul></ul>
  13. 13. Scrolling, Paging: Fill() <ul><li>ADO.NET DataAdapter.Fill() method </li></ul><ul><ul><li>There’s an overload that takes first row and number of rows </li></ul></ul><ul><ul><li>Under the covers, this method: </li></ul></ul><ul><ul><ul><li>Skips rows as needed </li></ul></ul></ul><ul><ul><ul><li>Copies as many rows as requested to the target DataTable </li></ul></ul></ul><ul><ul><ul><li>Scans and discards the rest of the rows </li></ul></ul></ul><ul><ul><li> Don’t use it for paging in large result-sets </li></ul></ul>
  14. 14. Scrolling, Paging: SQL <ul><li>If at all possible, use SQL constructs for paging </li></ul><ul><ul><li>Stored-procedures </li></ul></ul><ul><ul><ul><li>if you know the table schema and sort order </li></ul></ul></ul><ul><ul><li>SQL can help in other cases </li></ul></ul><ul><ul><ul><li>where the query is not known but constrained </li></ul></ul></ul><ul><li>Scalability/performance issues </li></ul><ul><ul><li>Time taken to execute query is not amortized across requests </li></ul></ul>
  15. 15. Scrolling, Paging: SQL <ul><li>Design issues </li></ul><ul><ul><li>Fits nicely for 3-tier apps </li></ul></ul><ul><ul><ul><li>Each page is an independent database operation </li></ul></ul></ul><ul><ul><ul><li>No state held between hits </li></ul></ul></ul><ul><ul><li>Ad-hoc queries are hard to handle </li></ul></ul><ul><li>Result stability </li></ul><ul><ul><li>Need to use transactions if stability is required </li></ul></ul>
  16. 16. Streaming <ul><li>Goal is to handle very large result-sets </li></ul><ul><ul><li>No buffering proportional to size of data </li></ul></ul><ul><ul><li>Concurrency issues </li></ul></ul><ul><ul><li>Interleaving </li></ul></ul><ul><li>Common scenarios for streaming </li></ul><ul><ul><li>Batch processing </li></ul></ul><ul><ul><li>Reporting </li></ul></ul><ul><ul><li>Custom aggregation </li></ul></ul>
  17. 17. Streaming: DataReaders <ul><li>ADO.NET providers have an streaming interface </li></ul><ul><ul><li>DataReader class exposes a row at a time </li></ul></ul><ul><ul><ul><li>Some minor buffering might happen internally </li></ul></ul></ul><ul><ul><li>Can scan millions of rows without taking much resources </li></ul></ul><ul><li>Scalability issues </li></ul><ul><ul><li>Usually large scans happen in batch processes </li></ul></ul><ul><ul><li>Not many at the same time </li></ul></ul>
  18. 18. Streaming: DataReaders <ul><li>Design issues </li></ul><ul><ul><li>DataReaders cannot be marshaled across tiers </li></ul></ul><ul><ul><ul><li>Move the batch process code to the middle tier </li></ul></ul></ul><ul><ul><ul><li>Send the data in chunks to the next tier (too much overhead in most cases) </li></ul></ul></ul><ul><ul><li>Contention can be high </li></ul></ul><ul><li>Result stability </li></ul><ul><ul><li>Depends on the isolation level </li></ul></ul>
  19. 19. Custom Aggregate Logic <ul><li>Similar case: scan lots of rows </li></ul><ul><ul><li>But end-result is a small result-set </li></ul></ul><ul><li>No need to ship the data out of the server </li></ul><ul><ul><li>Cursors can help here </li></ul></ul><ul><ul><li>Scan and aggregate inside the server </li></ul></ul><ul><ul><li>Ship only the aggregated information to the client </li></ul></ul><ul><li>Avoids moving lots of data across tiers </li></ul>
  20. 20. Incremental Load Scenarios <ul><li>Present first bit of data in UI quick </li></ul><ul><li>Incrementally load the rest in the background </li></ul><ul><ul><li>DataSet merge support is great here </li></ul></ul><ul><ul><li>Be aware of multi-threading issues </li></ul></ul>
  21. 21. Incremental Load & Merge <ul><li>ADO.NET DataSet can merge results </li></ul><ul><li>Incremental load UI </li></ul><ul><ul><li>Chunking API in the middle tier </li></ul></ul><ul><ul><li>Bring down the first DataSet and display it </li></ul></ul><ul><ul><li>As more data comes, merge the DataSets and update UI </li></ul></ul><ul><ul><ul><li>This even preserves changes in existing data </li></ul></ul></ul><ul><li>Multi-threading issues </li></ul><ul><ul><li>DataSet is not thread-safe </li></ul></ul><ul><ul><li>Same for WinForms UI controls </li></ul></ul>
  22. 22. Summary <ul><li>Disconnected is good </li></ul><ul><ul><li>ADO.NET has great support for it </li></ul></ul><ul><ul><li>Good to start disconnected by default </li></ul></ul><ul><li>You will need some connected pieces </li></ul><ul><ul><li>ADO.NET also helps there </li></ul></ul><ul><ul><li>You can add connected parts as needed </li></ul></ul><ul><li>Extremes can hurt your app performance or scalability </li></ul>
  23. 23. Attend a free chat or web cast http://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asp List of newsgroups http://communities2.microsoft.com/ communities/newsgroups/en-us/default.aspx MS Community Sites http://www.microsoft.com/communities/default.mspx Locate Local User Groups http://www.microsoft.com/communities/usergroups/default.mspx Community sites http://www.microsoft.com/communities/related/default.mspx
  24. 24. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.