Your SlideShare is downloading. ×
Data Storage In Windows Azure Ntk 2012
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Data Storage In Windows Azure Ntk 2012

1,210
views

Published on

Na predavanju bomo spoznali tehnologije Windows Azure Blob in Table Storage, ki omogočata shranjevanje in obdelavo velikih količin podatkov v oblaku. Predstavljeni bodo primeri in najboljše prakse …

Na predavanju bomo spoznali tehnologije Windows Azure Blob in Table Storage, ki omogočata shranjevanje in obdelavo velikih količin podatkov v oblaku. Predstavljeni bodo primeri in najboljše prakse uporabe omenjenih tehnologij, ter njihove prednosti in omejtive. Pogledali bomo, kako se SQL Azure razlikuje od Table Storage.


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,210
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Data Storage inWindows AzureBlaž Lipušček,Diventic d.o.o. Nivo 300
  • 2. Cloud ComputingComputing capability delivered as a utility throughInternet standards and protocols.• Software as a Service• Platform as a Service• Infrastructure as a Service Nivo 300
  • 3. Windows Azure Fabric• Fabric Controller is the brain of the entire ecosystem• Highly redundant hardware and software environment• It can‘t ever fail! Nivo 300
  • 4. Windows Azure Storage Options • Instance local storage is not persistent: • Usefull for cache or temp files! • Blob service: • Store large text or binary data • Files can be served via CDN • Table service • Azure Drive: • NTFS, can be used with I/O • Can connect only one instance at once Nivo 300
  • 5. Windows Azure Blob Service • REST API Nivo 300
  • 6. Store Files to Blob service • WindowsAzure.StorageClient (REST API) Nivo 300
  • 7. Grant Access to Files • Shared Access Signatures • Read • Write • Delete • List https://mycontainer.blob.core.windows.net/files/image.jpg?se=2011-05- 18T12%3A50%3A38Z&sr=b&sp=r&sig=EUtAbKQ%2F%2FYY7k1fJtzE6hWt%2FjadF AqxDYh5tAPK%2FeIw%3D Nivo 300
  • 8. Grant Access to Files:Shared Access Signatures • Remember to use UTC on DateTime • Azure hides resources with invalid Shared Access Signatures • DeleteIfExists will never fail • Make sure URL is properly encoded Nivo 300
  • 9. Windows Azure Table Storage • http://myaccount.table.core.windows.net Nivo 300
  • 10. Windows Azure Table Storage • Requires major changes to application data logic • No Schema, NO-SQL • More scalable than SQL Azure • Can handle large datasets • Relatively cheap Nivo 300
  • 11. Windows Azure Tables • Table names must conform to these rules: ^[A-Za-z][A-Za-z0-9]{2,62}$ • An entity can have up to 255 properties • The combined size of all data in an entitys cannot exceed 1 MB • 64KB limitation per property • An entity always has the following system properties: • PartitionKey, RowKey,Timestamp property Nivo 300
  • 12. Windows Azure Tables • Not allowed in PartitionKey and RowKey values: /, , #, ? • The partition and row key is a string value up to 1 KB in size. • You must include the PartitionKey and RowKey property in every insert, update, and delete operation. Nivo 300
  • 13. Windows Azure Tables • REST service via HTTP(S): • CREATE: http://myaccount.table.core.windows.net/Tables • LIST: http://myaccount.table.core.windows.net/Tables • GET, UPDATE, DELETE: http://myaccount.table.core.windows.net/MyTable() • SINGLE TABLE: http://myaccount.table.core.windows.net/Tables(MyTable) Nivo 300
  • 14. Query Timeout and Pagination • Query can return up to 1,000 rows at a time: • x-ms-continuation-NextTableName • x-ms-continuation-NextPartitionKey • x-ms-continuation-NextRowKey • Construct new query using continuation tokens: • NextTableName, NextPartitionKey , NextRowKey • 30 seconds for scheduling and processing Nivo 300
  • 15. Query Timeout and PaginationDate: Mon, 13 Feb 2012 20:11:08 GMTContent-Type: application/atom+xml;charset=utf-8Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0Cache-Control: no-cachex-ms-request-id: f9b2cd09-4dec-4570-b06d-4fa30179a58ex-ms-version: 2011-08-18x-ms-continuation-NextPartitionKey: 1!8!U21pdGg-x-ms-continuation-NextRowKey: 1!12!QmVuMTg5OA--Content-Length: 880298 http://myaccount.table.core.windows.net/Customers? NextPartitionKey=1!8!U21pdGg-& NextRowKey=1!12!QmVuMTg5OA-- Nivo 300
  • 16. Query OptionsOperator URI expressionEqual eqGreaterThan gtGreaterThanOrEqual geLessThan ltLessThanOrEqual le Operator URI expressionNotEqual ne And and Not not Or or Nivo 300
  • 17. Windows Azure Table Storage PartitionKey RowKey Timestamp Property Collection Name Type Value Jeff C_Jeff DateTime Kind String “Customer” Name String “Jeffrey” City String “Seattle” Jeff O_Cereal DateTime Kind String “Order” Item String “Cereal” Quantity Int32 1 Jeff O_Milk DateTime Kind String “Order” Item String “Milk” Quantity Int32 2 Paul C_Paul DateTime Kind String “Customer” Name String “Paul” City String “New York” Nivo 300
  • 18. Windows Azure Table StorageDate Type DescriptionString A UTF-16-encoded string of characters up to 64 KB in size.Byte array An array of bytes up to 64 KB in size.Guid A 128-bit globally unique identifier (can be null).DateTime A 64-bit value expressed as Coordinated Universal Time (UTC) ranging from 12:00 midnight, January 1, 1601 A.D. (C.E.) to December 31, 9999 (can be null).Int32 A 32-bit signed integer (can be null)Int64 A 64-bit signed integer (can be null).Double A 64-bit floating point value (can be null)Boolean A Boolean value (can be null) Nivo 300
  • 19. DataServiceContext • It is not stateless (like WCF Data Services) • Any instance members are not guaranteed to be thread safe Nivo 300
  • 20. Table Storage: Good to know! • DataServiceContext is not stateless! • Create it when you need to perform multiple transactions • Set timeout by using the Timeout property • Query using PartitionKey and RowKey • Only this two parameters are indexed. • Order records by latest: • DateTime.MaxValue.Ticks - date.ToUniversalTime().Ticks Nivo 300
  • 21. Retrieve a Single Entity • Fastest query • Handle 404 Not Found Nivo 300
  • 22. Retrieve Using Prefix Nivo 300
  • 23. Entity Group Transactions • Limited to a single partition in a single table • context.SaveChanges( SaveChangesOptions.Batch);PartitionKey RowKey ID Kind Amount Title ... Timestamp7635 7635 7635 ExpenseItem MyExpense 15.5...30:007635 7635_8215 8215 ExpenseItemRow 100 15.5...30:057635 7635_8216 8216 ExpenseItemRow 50 15.5...30:10 Nivo 300
  • 24. Parallel Querying for Full Table Scans • Fetch smaller list of partition keys (eg. Categories) • Execute multiple queries in parallel: • serviceContext.FoodTable.AsTableServiceQuery() .Where(c => c.PartitionKey == _category && c.Color == "Green"); • Parallel LINQ Nivo 300
  • 25. Use Command Pattern TableCommand (abstract) Insert Update Delete Nivo 300
  • 26. TableCommand abstract class Nivo 300
  • 27. Select Command Nivo 300
  • 28. Update Command Nivo 300
  • 29. Update Command Nivo 300
  • 30. Fat Entities • Instead of going for 64KB limitation per property > single 1MB limitation for the whole entity. • Table provider also abstracts away all subtle retry policies: • Handles OperationTimedOut error • 4MB restriction of the Table Storage for transactions • http://code.google.com/p/lokad-cloud/wiki/FatEntities Nivo 300
  • 31. Fat Entities Nivo 300
  • 32. Table Storage PerformanceOptimizationUse when table and entity name aredifferenct. 8-15ms for deserializingeach entity received in a query. Payload delay. Overhead with POST and PUT requests. Nivo 300
  • 33. Windows Azure Storage Analytics • Track, Analyze, and Debug usage of Windows Azure Storage • Logs: Provide trace of executed requests for Blobs, Tables and Queues • Metrics: Provide summary of key capacity and request statistics for Blobs, Tables and Queues • Data stored inside your account • Turn it on with Get Storage Service Properties API call Nivo 300
  • 34. Windows Azure Storage Analytics • Transaction data and capacity data is stored in the following tables: • $MetricsCapacityBlob • $MetricsTransactionsBlob • $MetricsTransactionsTable • $MetricsTransactionsQueue Nivo 300
  • 35. Logs • Loggin Authenticated Requests: • Successful requests • Failed requests, including timeout, throttling, network, authorization ... • Requests using a SAS, including failed and successful requests • Requests to analytics data • Loggin Anonymous Requests: • Successful requests • Server errors • Timeout errors for both client and server • Failed GET requests with error code 304 (Not Modified) Nivo 300
  • 36. Metrics • Request information: Provides hourly aggregates of number of requests, average server side latency, average E2E latency, average bandwidth, total successful requests and total number of failures and more. • Capacity information: Provides daily statistics for the space consumed by the service, number of containers and number of objects that are stored in the service. Note, this is currently only provided for the Windows Azure Blob service. Nivo 300
  • 37. Get/Set Storage Service PropertiesElement Name DescriptionVersion Required. The version of Storage Analytics to configure.Delete Required. Applies only to logging configuration. Indicates whether all delete requests should be logged.Read Required. Applies only to logging configuration. Indicates whether all read requests should be logged.Write Required. Applies only to logging configuration. Indicates whether all write requests should be logged.Enabled Required. Indicates whether metrics is enabled for the storage service.IncludeAPIs Required only if metrics is enabled. Applies only to metrics configuration. Indicates whether metrics should generate summary statistics for called API operations.RetentionPolicy/Enabled Required. Indicates whether a retention policy is enabled for the storage service.RetentionPolicy/Days Required only if a retention policy is enabled. Indicates the number of days that metrics or logging data should be retained. All data older than this value will be deleted. The minimum value you can specify is 1; the largest value is 365 (one year). Nivo 300
  • 38. Windows Azure StorageAnalytics Tools• Cerebrata WAS Configuration Utility https://www.cerebrata.com/Downloads/SACU/SACU.application• Sample Tool http://storageanalytics.cloudapp.net• Cerebrata Cloud Storage Studio• Cerebrata Diagnostics Manager Nivo 300
  • 39. Connecting to SQL Azure • Connecting to SQL Azure only requires change in configuration • No support for Windows Authentication • Encrypt credentials Nivo 300
  • 40. Connecting to SQL AzureOpen connections late and close early. • A logged-in session that has been idle for 30 minutes will be terminated automatically. Nivo 300
  • 41. SQL Azure Feature Limitations • Common Language Runtime (CLR) and CLR User-Defined Types • Integrated Full-Text Search • SQL Server 2008 R2 Features (SQL Server Utility, PowerShell Provider, Master Data Services) • SQL Server Replications, Backup and Restore CREATE DATABASE destination_database_name AS COPY OF [source_server_name.]source_database_name Nivo 300
  • 42. Troubleshooting SQL Azure • Service Unavailable or Does Not Exist • Check if you are unable to resolve the name • Firewall blocking port 1433 used by SQL Azure • Using proxy server not configured properly • General Network Errors • Connection is idle for an extended period of time. • Connection consumes an excessive amount of resources • Connection holds onto a transaction for an extended period of time • If the server is too busy https://www.windowsazure.com/en-us/support/service-dashboard/ Nivo 300
  • 43. SQL Azure vs Table StorageFeature SQL Azure Common Benefit(s) Azure Table Storage Cross-table queries Primary key-based lookup Single key lookup (bySelect semantics partition) High performance via Automatic mass scale by multiple indices, partition and consistentPerformance and scale normalized data structures. performance even at large scale Well-known management Familiar high-level Direct serialization;User experience tools and traditional developer experience simplified design model by database design removing relational model Traditional relational design Data storage for all types of Type variability in a singleStorage style model data table No transaction cost, pay by Network traffic cost outside No space overhead cost,Cost factors database size of same datacenter pay for what is used Synchronizing between local and cloud-basedData loading and sync stores; data easily moved in and out. Nivo 300
  • 44. Windows Azure Cost structureIn the context of the cost-oriented architecture, variousdecisions will impact monthly operational expenses.• Compute charges Compute• Storage charges Storage• Charges for Data Transfer Transfer Services• Add-on service charges Nivo 300
  • 45. Windows Azure PricingWindows Azure Capability ChargeServer Usage Small: $0.12 /service-hour Medium: $0.24/service-hour Large: $0.48/service-hour XLarge: $0.96/service-hourWindows Azure Blobs and $0.14/GBTablesTransactions $0.01/10K transactionsSQL Azure: Web Edition $9.99/month (1GB RDBMS)SQL Azure: Business Edition $99.99/month (10GB RDBMS)Windows Azure $0.15/100K message operations Nivo 300
  • 46. VPRAŠANJA?Po zaključku predavanja, prosimo, izpolnite vprašalnik.Vprašalniki bodo poslani na vaš e-naslov, dostopni pabodo tudi preko profila na spletnem portalu konferencewww.ntk.si.Najlepša hvala!Blaž Lipušček, Diventic d.o.o.blaz.lipuscek@diventic.si Nivo 300