Windows Azure Storage & Sql Azure


Published on

Windows Azure Storage & Sql Azure

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • – Provide a simple interface for storing named files along with metadata for the fileTables – Provide structured storage. A Table is a set of entities, which contain a set of propertiesQueues – Provide reliable storage and delivery of messages for an applicationTab
  • Harvesting!
  • Use queues as a way of communicating w/ the backend worker rolesWRs call getmessage and pass timeoutTimeout value is importantExpiration time is important; message is marked in the queue as invisible; for duration of timeout it’s invisibleWhen we’re done processing, we call a message to remove the message through a deleteTh reason we do this is imagine we have a second worker role; if something goes wrong, once the timeout expires, the message becomes visible, and the next person to do a get message will get the message
  • The PartitionKey combined with the RowKey uniquely identifies an entity in a table.
  • 11:53Getting the all of dunnry’s post it fast because we’re selecting the entities by a partition keyGetting all of the posts after a certain is slow because we may have to traverse across multiple servers because we’re selecting entities that span partition keysA query without the partition key is really a scan
  • We have included this feature comparison table in anticipation of your likely questions about differences between using a relational database table as you may be currently doing with your SQL Server databases and the new Windows Azure Tables included in Windows Azure.
  • As I stated earlier, SQL Azure is based on SQL Server 2008. At this time it is only a subset of the features of the server product.My intention here is to convey the high level features that are supported and the ones that are not.SQL Azure will support most of the things we need… Tables, Index, Views, Stored Procedures, Triggers, and Constraints… in my book… that’s all the functionality that I need for most of my applications.There are some other adjunct technologies that ship as part of SQL Server 2008 such as SQL Reporting Services and Analysis Services which are not supported. The Service Broker is also not supported.
  • So let’s assume that we have designed our relational database with local developer and data modeling tools.We can begin our story then by assuming that we want to get our database deployed to the cloud.There are some tools that will expedite this process which I will show you later, but for now lets assume that we have scripted our database schema. We apply this script to SQL Azure which speaks native TDS.If you created your database through the SQL Azure Portal, then SQL Azure will have created one master database and three replicas of that database. If you create your database with the script the same will be true.These replicas are stored in different database centers from the master to provide redundancy and protection against geographical catastrophe.
  • Configuring our application to use SQL Azure storage instead of SQL Server is simply a matter of modifying the connection string in our application’s configuration file.When our application requests data, ADO.NET speaks to the TDS which directs our queries to the master database server. The master database server performs our query and returns the results to our application.
  • From our application’s point of view, there is only one SQL Azure database.As we make updates to our database, those updates are replicated to other copies stored in other data centers so that in the event that our database fails for any reason, the other databases will be standing by ready to take its place.
  • But what if that master database server fails for some reason?TDS is receives notification of the database failure and automatically redirects the call to the replica!The Azure Cloud Fabric is self-healing… and the details are outside the scope of this presentation; however, the fabric will get busy repairing itself like drones on a Borg mother ship… essentially with the objective of keeping three replicas online at a time.
  • I will demonstrate creating a SQL Azure account in session 3 where I will walk you through the entire process.For now I simply want to give you some background information to prepare you for our first demonstration.When we create our SQL Azure database server, we’ll be prompted for an Administrator’s name and a password.This username and password will be the granted a system administrator role that is similar to the “sa” account on a local SQL Server 2008 box. The account has permission to create and drop databases and database ownership authority in any databases that you create with this account.
  • After creating your SQL Azure database server, you will want to grant appropriate access through the SQL Azure firewall.SQL Azure provides a very simple and easy to maintain firewall. The firewall is so easy to use that it’s only going to get one slide in my deck!The firewall allows us to expose our database to Windows Azure services via a checkbox and to add ranges of IP addresses such as your home office and your business… or possibly the address of a 3rd party server hosting some application that needs data access.I’ll do a thorough demo of this feature in session 3…
  • When you created your SQL Azure database server, you supplied an administrator’s user name and password. I have named my user accordingly… to remind me of its power.The SQL Portal will offer you the ability to copy these credentials in connection string format to your clip board… tempting you into believing that you should just paste this into your configuration file.This is terrific for demos like mine… BUT you should NEVER, EVER do this…A database server system administrator password placed in a configuration file in clear text format… there has got to be something naive in the extreme going on here… and worse… no way to create non-sa-like users through the UI… you must script your database users and then apply the script to the database. And to anticipate your question… no… you can’t use SQL Server Management Studio to do this either.I will demo this as well in session 3… so hang tight…
  • Windows Azure Storage & Sql Azure

    1. 1. Windows Azure StorageSQL Azure<br />Developing with Windows Azure storage & SQL Azure<br />Maarten Balliauw<br /><br /><br /><br />
    2. 2. Whoam I?<br />Maarten Balliauw<br />Antwerp, Belgium<br /><br />Focus on web<br />ASP.NET, ASP.NET MVC, PHP, Azure, VSTS, …<br />Special interest in interop<br />PHPExcel, PHPLinq, PHPMEF, Windows Azure SDK for PHP, ...<br /><br /><br />
    3. 3. Agenda<br /><ul><li>Storage concepts
    4. 4. Windows Azure SDK for PHP
    5. 5. SQL Azure</li></li></ul><li>Storage concepts<br />Where can I store my stuff?<br />
    6. 6. Windows Azure Data Storage<br />Queue<br />Blob<br />Account<br />Tables<br />Drives<br />
    7. 7. Azure Platform Data Storage Options<br />Windows Azure Data Storage<br />Blobs<br />Unstructured data storage<br />Tables<br />Semi-structured or tabular data storage<br />Queues<br />Buffered delivery data storage<br />Drives<br />Durable NTFS volumes that Windows Azure applications can use. See:<br />SQL Azure<br />Relational data storage<br />
    8. 8. PHP + Cloud Storage<br />Windows Azure Storage<br />PHP<br />Instance (web/worker)<br />On-Premise<br />VIP<br />Load Balancer<br />PHP App<br />SQL Azure<br />Windows Azure Platform<br />
    9. 9. Windows Azure Data Storage<br />
    10. 10. Windows Azure Data Storage<br />
    11. 11. Windows Azure SDK for PHP<br />A PHP porgramming model for Windows Azure Storage<br />
    12. 12. PHP with Windows Azure Storage<br />Windows Azure SDK for PHP at<br />PHP programming model for Windows Azure Storage<br />Features <br />PHP classes for Blobs, Tables & Queues<br />Store PHP sessions in Table Storage<br />File system wrapper for Blob Storage<br />
    13. 13. Blob<br /> Container<br />Entities<br />Account<br />Table<br />http://<account><container><br />Messages<br />Windows Azure Storage Concepts<br />http://<account><table><br />Queue<br />http://<account><queue><br />
    14. 14. Windows Azure Blobs<br />Unstructured data<br />Scale massively<br />At least 3 instances, 5 in optimal situation<br />Can be used as CDN<br />Can be mapped using a custom domain<br />
    15. 15. Tools for connecting to blob storage<br />CloudBerryLab Explorer<br />Azure Storage Explorer<br />Online /<br />Windows Azure tooling for Eclipse<br />
    16. 16. Blobs Sample<br />$blobStorage= new Microsoft_WindowsAzure_Storage_Blob();<br />// Create<br />if (!$blobStorage->containerExists($containerName))<br />{<br />$blobStorage->createContainer($containerName);<br />$blobStorage->setContainerAcl($containerName, <br />Microsoft_WindowsAzure_Storage_Blob::ACL_PUBLIC);<br />}<br />// Store<br />$blob = $blobStorage->putBlob($containerName, $blobName, <br />$localFilename, $metadata);<br />/* @var $blob Microsoft_WindowsAzure_Storage_BlobInstance */<br />
    17. 17. Blobs Sample Cont…<br />// Copy<br />$result = $blobStorage->copyBlob(<br /> $containerName, $blobName, $containerName, $blob2Name);<br />// Retrieve<br />$tempStore= azure_getlocalresourcepath('tempstore');<br />$tempPath= $tempStore. 'apos; . $imageId;<br />$blobStorage->getBlob($containerName, $blobName, $tempPath);<br />// Delete<br />$result = $blobStorage->deleteBlob($containerName, $blobName);<br />$result = $blobStorage->deleteContainer($containerName);<br /><br />
    18. 18. Blob Stream Wrapper<br />$blobStorage= new Microsoft_WindowsAzure_Storage_Blob();<br />// Register:<br />$blobStorage->registerStreamWrapper(); // registers azure://<br />// or<br />$blobStorage->registerStreamWrapper('blob://'); // use blob://<br />// Use<br />$fp= fopen('azure://mycontainer/myfile.txt', 'r');<br />// ...<br />fclose($fp);<br />
    19. 19. Windows Azure Drive (a.k.a. Xdrive)<br />Virtual NTFS volume that can be mounted<br />.vhd format<br />Use existing NTFS API’s<br />Easier migration<br />Stored on blob storage provides quick mount/unmount in other VM<br />
    20. 20. Queue Workflow Concepts<br />Windows Azure Queue Provides<br />Guarantee delivery (two-step consumption)<br />Worker Dequeues Message and mark it as Invisible<br />Worker Deletes Message when finished processing it<br />If Worker role crashes, message becomes visible for another Worker to process<br />Doesn’t guarantee “only once” delivery<br />Doesn’t guarantee ordering<br />Best effort FIFO<br />Worker Role<br />Web Role<br />Input Queue (Work Items)<br />Worker Role<br />Azure Queue<br />Web Role<br />Worker Role<br />Web Role<br />Worker Role<br />
    21. 21. Azure Queues<br />RemoveMessage<br />GetMessage (Timeout)<br />Worker Role<br />PutMessage<br />Queue<br />Msg 1<br />Msg 2<br />Msg 2<br />Msg 1<br />Web Role<br />Worker Role<br />Worker Role<br />Msg 3<br />Msg 4<br />Msg 2<br />
    22. 22. Loosely Coupled Work with Queues<br />Worker-Queue Model<br />Load work in a queue<br />Many workers consume the queue<br />Input Queue (Work Items)<br />Azure Queue<br />Worker Role<br />Web Role<br />Worker Role<br />Web Role<br />Worker Role<br />Web Role<br />Worker Role<br />
    23. 23. Queues<br />$queueClient= new Microsoft_WindowsAzure_Storage_Queue();<br />// Create<br />$result = $queueClient->createQueue('imageQueue');<br />// Delete<br />$queueClient->deleteQueue('imageQueue');<br />// Add message<br />$queueClient->putMessage('imageQueue', $message, $ttl);<br />// Retrieve Messages<br />$messages = $queueClient->getMessages('imageQueue', 10);<br />foreach($messages as $message) {<br />// Do work here...<br />$queueClient->deleteMessage('imageQueue', $message);<br />}<br />
    24. 24. Windows Azure Data Storage – Tables (Terms Part 1)<br />Table <br />Contains a set of entities. <br />Entity (Row) <br />Basic data items stored in a table. <br />Property (Column) <br />Single value in an entity. <br />RowKey<br />Unique ID of the entity within a partition<br />Timestamp<br />Time it was created<br />
    25. 25. Windows Azure Data Storage – Tables (Terms Part 2)<br />Partition <br />Entities in a table with the same partition key<br />PartitionKey <br />Segments entities in to partitions to automatically distribute the table’s entities over many storage nodes.<br />Sort Order<br />There is a single index provided, where all entities in a table are sorted by PartitionKey and then RowKey<br />
    26. 26. Key Example – Blog Posts<br />Partition 1<br />Partition 2<br />Getting all of dunnry’s blog posts is fast<br />Single partition<br />Getting all posts after 2008-03-27 is slow<br />Traverse all partitions<br />
    27. 27. Table Sample<br />$tableStorage= new Microsoft_WindowsAzure_Storage_Table(<br />'', 'myaccount', 'myauthkey');<br />// Create<br />$result = $tableStorage->createTable($tableName);<br />// List <br />$result = $tableStorage->listTables();<br />foreach($result as $table) {<br />echo 'Table name is: ' . $table->Name . " ";<br />}<br />// Delete<br />$tableStorage->deleteTable($tableName);<br />
    28. 28. Tables with Entities<br />// Structured entity<br />class ImageEntityextends <br />Microsoft_WindowsAzure_Storage_TableEntity<br />{<br />/**<br /> * @azure filename<br /> */<br />public $filename;<br />/**<br /> * @azure size Edm.Int64<br /> */<br />public $size;<br />}<br />// Unstructured entity<br />// Microsoft_WindowsAzure_Storage_DynamicTableEntity<br />
    29. 29. Tables with Entities Cont…<br />// Insert<br />$image = new ImageEntity($partitionKey, $rowKey);<br />$image->filename = $_FILES['imageUpload']['name'];<br />$image->size = $_FILES['imageUpload']['size'];<br />$image = $tableStorageClient->insertEntity($tableName, $image);<br />// Retrieve<br />$image = $tableStorage->retrieveEntityById($tableName, <br />$partitionKey, $rowKey, 'ImageEntity');<br />// Update<br />$image->filename = 'newname.jpg';<br />$result = $tableStorage->updateEntity($tableName, $image);<br />// Delete<br />$result = $tableStorage->deleteEntity($tableName, $image);<br />
    30. 30. Table Queries<br />// Filter condition<br />$select = "filesizegt 1024 and PartitionKeyeq '$partitionKey'";<br />// Or fluent interface<br />$select = $tableStorage->select()->from($tableName)<br /> ->where('filesizegt 1024')<br /> ->andWhere('PartitionKeyeq ?', $partitionKey);<br />// Run query<br />$images = $tableStorage->storageClient->retrieveEntities(<br />'testtable', $select, 'ImageEntity'<br />);<br />foreach($images as $image) {<br />echo 'Name: ' . $image->filename . " ";<br />}<br />
    31. 31. Batching Operations<br />// Start batch<br />$batch = $tableStorage->startBatch();<br />// Insert multiple<br />$images = generateEntities();<br />foreach($images as $image) {<br />$tableStorage->insertEntity($tableName, $image);<br />}<br />// Commit<br />$batch->commit();<br />
    32. 32. Table session handler<br />$sessionHandler<br /> = new Microsoft_WindowsAzure_SessionHandler($tableStorage);<br />$sessionHandler->register();<br />session_start();<br />if (!isset($_SESSION['start'])) {<br />$_SESSION['start'] = time();<br />}<br />
    33. 33. SQL Azure<br />A relational database for the cloud<br />
    34. 34. SQL Azure and Windows Azure Table Comparison<br />SQL Azure Tables<br />Windows Azure Tables<br />Fully structured<br />Strongly typed<br />Relational(RDBMS)<br />Highly scalable<br />Semi-structured<br />Loosely typed<br />Non-Relational(Not RDBMS)<br />Massively scalable<br />
    35. 35. MySQL: Simple Configuration<br />VIP<br />Load Balancer<br />Web Role<br />MySQL<br />Worker Role<br />
    36. 36. MySQL in a Windows Azure Application<br />Running MySQL in a worker role<br />Copy MySQL to the worker role sub-directory<br />Copy to read-write local storage<br />Configure MySQL to listen on the right port<br />Monitor MySQL health<br />Consuming MySQL<br />Discover IP address and port<br />Normal access from then on<br />Handle topology changes<br />
    37. 37. Simple Configuration<br />VIP<br />Load Balancer<br />MySQL<br />
    38. 38. Replication<br />VIP<br />Load Balancer<br />M<br />S<br />S<br />MySQL<br />MySQL<br />MySQL<br />
    39. 39. Windows Azure Drive with Hot Spare<br />VIP<br />Load Balancer<br />MySQL<br />MySQL<br />
    40. 40. Windows Azure Drive with Hot Spare<br />VIP<br />Load Balancer<br />MySQL<br />MySQL<br />
    41. 41. SQL Azure Features<br />Supported<br />Not supported<br />Tables, Indexes, Views<br />Stored Procedures<br />Triggers<br />Constraints<br />Table Variables<br />Temp Tables (#Name)<br />Physical Server Access <br />Catalog DDL<br />Common Language Runtime<br />Service Broker<br />Reporting Services<br />Analysis Services<br />Distributed Transactions and Queries<br />
    42. 42. SQL AzureDeployment<br />Web Portal<br />(API)<br />DB Script<br />SQL Azure<br />TDS<br />
    43. 43. SQL AzureAccessing databases<br />Web Portal<br />(API)<br />Your App<br />SQL Azure<br />TDS<br />Change Connection String<br />
    44. 44. Database Replicas<br />Single Database<br />Multiple Replicas<br />Replica 1<br />Single Primary<br />Replica 2<br />DB<br />Replica 3<br />
    45. 45. SQL AzureDatabase Monitoring & Recovery<br />Web Portal<br />(API)<br />!<br />Your App<br />SQL Azure<br />TDS<br />
    46. 46. SQL Azure Server Creation<br />
    47. 47. SQL Azure Firewall Maintenance<br /><ul><li>Simple rules
    48. 48. Easy one-screen portal maintenance</li></li></ul><li>SQL Azure Database Connection String<br />An administrative user is created with the server<br />User has system administrator permissions like “sa”<br />; Database=FabrikamAzureDB;User ID=SaPaulM;Password=myPassword;Trusted_Connection=False;Encrypt=True;<br />
    49. 49. Migrating schema and data<br />SQL Azure Migration Wizard<br />SQL Azure Data Sync Tool for SQL Server<br />SQL Server Management Studio 2008 R2 November Community Technology Preview<br />
    50. 50. PHP with SQL Azure<br />SQL Server Driver for PHP @<br />Supports PHP access to SQL Azure<br />Features<br />Choose between SQL Server and SQL Azure by changing connection string<br />Use from on-premises or in Windows Azure<br />
    51. 51. Resources<br />Learn more!<br />
    52. 52. Resources<br />Microsoft Windows Azure Interop<br /><br />Interop Bridges<br /><br />
    53. 53. More resources<br />PHP<br /><br /><br /><br />MySQL<br />Windows Azure MySQL PHP Solution Accelerator<br /><br />
    54. 54. Questions?<br />More information?<br />Contact me<br /><br /><br /><br /><br />Thankyouforwatching!<br />