Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Samedi SQL Québec - La plateforme data de Azure

901 views

Published on

6 juin 2015
Samedi SQL à Québec

Session 3 - Data (SQL Azure, Table et Blob Storage) (Eric Moreau)

SQL Azure est une base de données relationnelle en tant que service, Azure Storage permet de stocker et d'extraire de gros volumes de données non structurées (par exemple, des documents et fichiers multimédias) avec les objets blob Azure ; de données NoSql structurées avec les tables Azure ; de messages fiables avec les files d'attente Azure.

Published in: Technology
  • Login to see the comments

Samedi SQL Québec - La plateforme data de Azure

  1. 1. Azure Data Overview Eric Moreau, MVP www.emoreau.com – eric@moer.ca Microsoft Azure
  2. 2. SQL Database SQL Server VM DocumentDB Search HDInsight Additional databases Agenda – Part 1
  3. 3. Agenda – Part 2 Blobs Files Queues Tables
  4. 4. Agenda – Part 3 – Announced in May 2015 Data Lake Azure SQL Data Warehouse
  5. 5. SQL Database SQL Server VM DocumentDB Search HDInsight Additional databases Microsoft Azure Data Platform
  6. 6. Run SQL on VM Run any SQL product on cloud VM Support for SQL Server, Oracle, MySql Ready to go VM images available in Gallery Persistent storage using attached disk in blob storage (Azure Files) Has all the benefits and powers of VMs combined with the full features of a SQL Engine
  7. 7. SQL Database SQL Server VM DocumentDB Search HDInsight Additional databases Microsoft Azure Data Platform
  8. 8. A Server is not a machine TDS = Tabular Data Stream
  9. 9. SQL Server database technology “as a Service” Fully Managed (PaaS) Enterprise-ready with automatic support for HA, DR, Backups, replication and more SQL Database – The Basics
  10. 10. Scale out with ElasticScale Built-in regional database replicas for additional protection Uptime SLA of 99.99% (4.38 minutes / month) SQL Database – The Basics
  11. 11. ElasticScale
  12. 12. • New portal • Some features in preview • Full Text Search • Column Store Index • Users in contained database (+row-level security) • Parallel queries (Premium only) • CLR Integration • T-SQL window functions (over clause) • Change tracking SQL Database – v12
  13. 13. Service Tier Performance Level Common App Pattern Performance Business Continuity Max DB Size Transaction Perf. Objective DTU PITR DR / GEO-Rep Basic Basic Small DB, SQL opp 2 GB Reliability / Hr. 5 7 Days DB Copy + Manual Export Standard S0 S1 S2 Wrkgp/cloud app, multiple concurrent operations 250 GB Reliability / Min. 10 20 50 14 Days DB Copy + Manual Export Premium P1 P2 P3 Mission Critical, High volume, Many concurrent Users 500 GB Reliability / sec. 100 200 800 35 Days Active Geo- replication Selecting the right SQL Database edition This information is subject to change over time.
  14. 14. Blended Measure Relative
  15. 15. SQL Database SQL IaaS Why (at least) two offerings of SQL in Azure?
  16. 16. SQL Database (PaaS) SQL in a VM (IaaS)
  17. 17. SQL Database SQL Server VM DocumentDB Search HDInsight Additional databases Microsoft Azure Data Platform
  18. 18. Fully managed, scalable JSON document database service
  19. 19. Fully Managed NoSQL Document-based Database Service Automatic and consistent indexing of all properties Tunable High Performance Backed by write-optimized SSD Easy to scale Native Support for JSON and JavaScript Designed cloud first
  20. 20. A document store Collections Request Document 1 { "name": "John", "country": "Canada", "age": 43, "lastUse": "March 4, 2014" } { "name": "Eva", "country": "Germany", "age": 25 } Document 2 { "name": "Lou", "country": "Australia", "age": 51, "firstUse": "May 8, 2013" } Document 3 { "docCount": 3, "last": "May 1, 2014" } Document 4 {…} DocumentDB Application
  21. 21. DocumentDB Overview A NoSQL document database-as-a-service, fully managed by Microsoft Azure. For cloud-designed apps when query over schema-free data; reliable and predictable performance; and rapid development are key. First of its kind database service to offer native support for JavaScript, SQL query and transactions over schema-free JSON documents. Perfect for cloud architects and developers who need an enterprise-ready NoSQL document database. Release Timing Public Preview: August 21, 2014 GA: H2 FY15
  22. 22. Proven @ Scale Data store powering production Microsoft services.  Scale tested with databases over 10TBs of JSON document data from Office OneNote API service  Rapidly evolving data models and query patterns based on 3rd party applications  Serving over 1 million active users  Operating consistently with 99.95% availability First Party Use Cases  Store, roam and query user generated data across applications  Extensible data models for 3rd party integration  Social activity and awareness feeds with views and filtering
  23. 23. Programmability Broad language and platform support  REST/HTTP APIs  C#, Node.js, JavaScript and Python SDKs for Public Preview  SDKs open sourced through GitHub C# Client SDK  Supports gateway and direct connectivity  Async APIs for all operations  HTTP and TCP transports available  POCOs, inherited document types and dynamics  LINQ provider for query execution client = new DocumentClient(new Uri(endpointUrl), authKey); Database database = await client.CreateDatabaseAsync(new Database{Id = “myNewDatabase”}); DocumentCollection collection = await client.CreateDocumentCollectionAsync(database.SelfLink, new DocumentCollection{Id = “Pages"}); Page page = new Page { Id = "Sample", Title = "About Paris", Language = new Language { Name = "English" }, Author = new Author { Name = "Don” }, Content = "Don's document in DocDB is a valid JSON document", PageViews = 10000, Topics = new Topic[] { new Topic { Title = "History" }, new Topic { Title = "Places to see" }} }; ResourceResponse<Document> resp = await client.CreateDocumentAsync(collection.DocumentsLink, page);
  24. 24. SQL Database SQL Server VM DocumentDB Search HDInsight Additional databases Microsoft Azure Data Platform
  25. 25. Data size Access Updates Structure Integrity Scaling Relational DB vs. Hadoop
  26. 26. SQL Database SQL on IaaS DocumentDB Search HDInsight MySQL, Oracle Microsoft Azure Data Platform
  27. 27. Additional Database options in Azure Oracle VMs are supported in Azure. MySQL is offered from the partner ClearDB.
  28. 28. Agenda – Part 2 - Storage Blobs Files Queues Tables
  29. 29. Microsoft Azure Storage Blob
  30. 30. Two Types of Blobs Under the Hood
  31. 31. Two Types of Blobs Under the Hood Block Blob:
  32. 32. Two Types of Blobs Under the Hood Page Blob:
  33. 33. Blob Storage Concepts
  34. 34. Microsoft Azure Storage Files
  35. 35. Azure Files – Usage Share data across VMs and applications Share settings throughout services Dev/Test/Debug
  36. 36. Azure Files Shared Network File Storage for Azure Availability, durability, scalability are managed automatically Supports two interfaces: SMB and REST
  37. 37. “I wish I could go to storage and provision a cloud drive, giving it a namespace, and that drive would then be UNC-addressable by the OSes.” “I need two VM's running with a shared drive. One will write to the drive, the other will read [it].” “Hi, I have two VM's in Microsoft Azure. All I want to do is set up a file share between them. Is this possible?” “Is it possible to share a secondary disk between different VM instances?” Azure Files – Customer Quotes
  38. 38. Microsoft Azure Storage Queue
  39. 39. Producers Consumers Queue C1 C2 P1 P2 4 3 12 Queue-based Load Levelling Pattern
  40. 40. Queue length reflects how well the backend processing nodes are doing. Decouples the application. Flexibility of efficient resource usage within an application. Absorb traffic bursts and reduce the impact of individual component failures. Why use a Queue?
  41. 41. Messages are ordered but not guaranteed FIFO. Message will be processed at least once. Message may be processed more than once. .DequeueCount increases every time. -> Processing must be idempotent. Queue Considerations
  42. 42. Messages are stored up to 7 days Event buses are built on top of queues Queue Considerations
  43. 43. Microsoft Azure Storage Table
  44. 44. Table Storage Details
  45. 45. Table Storage Concepts
  46. 46. No Fixed Schema
  47. 47. Querying
  48. 48. Table Storage Details
  49. 49. Table Storage Details – Entity Properties
  50. 50. A key/value store Tables Partition B Partition A Property Entity 2B Property Type Data Property Name String intString Date Name LastUseCountry Age String intString Name Country Age String intString Date Name FirstUseCountry Age Row key 1 2 1 2Partition key A A B B int Date LastCount 2B Azure Storage Tables Application
  51. 51. To recap Azure Storage Tables Scalable service for non-relational data Blob Storage Storage of files, VHDs, mp4s, pngs, etc… Files SMB File Sharing Queues Low latency message processing
  52. 52. Azure Data Lake
  53. 53. Azure Data Lake
  54. 54. Azure Data Lake
  55. 55. Microsoft Azure SQL Data Warehouse
  56. 56. Azure SQL Data Warehouse
  57. 57. Azure SQL Data Warehouse
  58. 58. SQL Database (PaaS) SQL Server VM (IaaS) DocumentDB Search HDInsight MySQL, Oracle Microsoft Azure Data Platform - Recap
  59. 59. Relational database in the cloud (PaaS) SQL Server VM (IaaS) DocumentDB Search HDInsight MySQL, Oracle Microsoft Azure Data Platform - Recap
  60. 60. Relational database in the cloud (PaaS) Full relational database in the cloud (IaaS) DocumentDB Search HDInsight MySQL, Oracle Microsoft Azure Data Platform - Recap
  61. 61. Relational database in the cloud (PaaS) Full relational database in the cloud (IaaS) NoSQL – Json documents Search HDInsight MySQL, Oracle Microsoft Azure Data Platform - Recap
  62. 62. Relational database in the cloud (PaaS) Full relational database in the cloud (IaaS) NoSQL – Json documents Search Big Data MySQL, Oracle Microsoft Azure Data Platform - Recap
  63. 63. Relational database in the cloud (PaaS) Full relational database in the cloud (IaaS) NoSQL – Json documents Search Big Data Any other database in the cloud (IaaS/PaaS) Microsoft Azure Data Platform - Recap
  64. 64. Blobs Files Queues Tables Microsoft Azure Data Platform - Recap
  65. 65. Huge storage Block = Streaming Page = Random access Files Queues Tables Microsoft Azure Data Platform - Recap
  66. 66. Huge storage Block = Streaming Page = Random access Shared access between services Queues Tables Microsoft Azure Data Platform - Recap
  67. 67. Huge storage Block = Streaming Page = Random access Shared access between services Asynchronous processes Tables Microsoft Azure Data Platform - Recap
  68. 68. Huge storage Block = Streaming Page = Random access Shared access between services Asynchronous processes NoSQL – Key/Value pair Microsoft Azure Data Platform - Recap
  69. 69. Data Lake Azure SQL Data Warehouse Microsoft Azure Data Platform - Recap
  70. 70. fully featured RDBMS transactional processing rich query managed as a service elastic scale internet accessible http/rest schema-free data model arbitrary data formats Microsoft Azure Data Platform - Recap
  71. 71. Azure Data Overview Eric Moreau, MVP www.emoreau.com – eric@moer.ca Microsoft Azure

×