azure overview, impressions from build 2015, cloud storage, compute, data lake, analytics, service fabric, containerization, etc.

1. Build 2015 – BB
Azure

2. Why cloud ?
PRO CON
Scaling (”infinite”) Bandwidth (bottleneck)
Metered (Pay-as-you-go) Storage (comparatively expensive)
Redundancy (resilient) Hands-off (lack of control)
Less on-prem admin (TCO) SLA (compromises)
DevOps made possible (eliminate the middleman) Perception (is it safe ?)
SLA (guarantees) Not right for everything
I/P/SaaS

3. Why Azure ?
Azure (Microsoft) AWS (Amazon) GCP (Google)
Cost Per minute – rounded
up
Per hour – rounded
up
Per minute – rounded
up (minimum 10
minutes)
SLA
Technologies .Net, Hyper-V, Hybrid
clouds, WinServer,
NoSql and Sql storage,
HDInsight (Hadoop),
ML, etc
Xen hyper-V, NoSql
and Sql storage,
analytics (Hadoop),
ML, etc
KVM hyper-V, Linux,
Hadoop, Java/Python
(primarily), NoSql and
Sql (mysql), etc
”Barrier-to-entry” (for
a Microsoft stack
company)
Low Higher Higher

4. Regional presence (why is this relevant ?)

6. Analogy

8. Continuous integration / delivery  DevOps
• “A working product after every check-in”
• “The adoption of DevOps is being driven by factors such as:
• Use of agile and other development processes and methodologies
• Demand for an increased rate of production releases from application and business unit
stakeholders
• Wide availability of virtualized and cloud infrastructure from internal and external providers
• Increased usage of data center automation and configuration management tools”
• “You’re doing continuous delivery when
• Your software is deployable throughout its lifecycle
• Your team prioritizes keeping the software deployable over working on new features
• Anybody can get fast, automated feedback on the production readiness of their systems any
time somebody makes a change to them
• You can perform push-button deployments of any version of the software to any
environment on demand”

9. MS DevOps stages

10. The DevOps virtuous cycle

11. (...AKA Deming Cycle)

12. Moving on to...

13. Build 2015 Azure themes (rough bundling)
• Azure management and VS integration, ”new” Azure portal, resource groups
• Dockers and containerization
• App service + web and mobile on Azure
• Big data & analytics
• Azure Media Services
• Azure Elastic Search
• Azure Compute platform
• Storage updates (DocumentDB, Azure Sql, Blob updates)
• Azure Service Fabric
• IoT with Azure
• Office 365 and Azure
Want everything ?
http://channel9.msdn.com/Events/Build/2015?sort=status&direction=asc&term=azure#tab_sortBy_status

14. Cloud computing trends
• Hybrid cloud and piecemeal migration
• Private clouds less so
• Containerization and DevOps
• Agile => CI
• Cloud APIs (REST, JSON)
• Ease and scalability / automation of resource management
• Portal, API, scripting
• Security
• Encryption, automation, orchestration
• Industry clouds
• Case in point, Media Services, see also reference architectures
• Continuing commodification
• Metering, billing, modes of tweaking resource usage, monitoring
Sources: IBM, Gartner, IDC, Forrester

15. Storage – old and new

16. Storage
• SQL Database
• SQL Server VM
• DocumentDB
• Search
• HDInsight
• Other DBs
• Blobs
• Files
• Queues
• Tables
• DataLake (Hbase)

17. Azure storage offerings

18. Storage architecture

19. Core storage category API overview

20. Azure management and VS integration, ”new”
Azure portal, resource groups

21. Mgmt portal and Visual Studio integration
• Download the Azure SDK to integrate
• Start your VS or open your portal
• Portal : logon via Azure AD
• VS : use the Azure server explorer
• Or use PowerShell (get the Azure cmdlets)
• Or use cross-platform xplat-cli
• Or write the automation code in any .net
language
Dockers and containerizationDockers and containerization

22. ALTER DATABASE Milestone
MODIFY (edition='Standard')
Set-AzureSqlDatabase
-ResourceGroupName MS
-ServerName MS-Server
-DatabaseName Milestone
-RequestedServiceObjectiveName Standard;
PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/
{resourceGroupName}/providers/microsoft.sql/servers/{serverName}/databases/
{databaseName}?api-version=2014-04-01
https://msdn.microsoft.com/en-us/library/azure/mt163571.aspx
{
"location": “<location>”,
"properties": { "requestedServiceObjectiveId":“<serviceObjectiveID>“ }
}
Request body
T-SQL:
PowerShell:
REST API:

23. Dockers and containerization

24. The problem

25. The problem

26. The containerization solution – e.g. Dockers

27. ...the analogy to shipping containers...

28. VM
Virtualization spectrum – where VMs and
Containers fit

29. Azure Media Services

30. Azure Media Services
• Upload and storage
• Encoding, packaging, indexing
• Content protection (encryption, signing)
• Live and VoD streaming
• Azure Media Player (cross platform)
• Azure Media Indexer (NLP, transcripts, search, CC)
• Ingest => encode => encrypt => publish => deliver

31. Live streaming data flow

33. App service / Service Fabric

34. Azure Private Clouds
Applications composed of microservices
High Availability
Hyper-Scale
Hybrid Operations
High Density Rolling Upgrades
Stateful services
Low Latency
Fast startup &
shutdown
Container Orchestration
& lifecycle management
Replication & Failover
Simple
programming
models
Load balancing
Self-healingData Partitioning
Automated Rollback
Health
Monitoring
Placement
Constraints
Service Fabric

36. Abstraction, encapsulation, decoupling
• Reliable actors API (“Orleans”)
• Build reliable stateless and stateful objects with a virtual Actor Programming Model
• Suitable for applications with multiple independent units of state and compute
• Automatic state management and turn based concurrency (single threaded execution)
• Reliable services API
• Build stateless services using existing technologies such as ASP.NET
• Build stateful services using reliable collections
• Manage the concurrency and granularity of state changes using transactions
• Communicate with services using the technology of your choice (e.g WebAPI, WCF)
“Self-contained, independent, substitutable containers vs interdependent tiers”

37. Reliable Collections
• Atomically update one or more collections using transactions
• Changes are replicated and durably stored on multiple replicas
• Reads are repeatable within the transaction
• Enumerations are snapshot based
IReliableDictionary<K,V> IReliableQueue<T>

38. Reliable services sample

39. Reliable actors sample

40. Azure App Service
• Abstraction layer over Web Apps, Moble Apps, API Apps, Logic Apps
• “Point-and-click” / CASE-like functionality
• Seamless authentication
• API definition and generation, auto-discovery and update using
Swagger / Swashbuckle
• Galleries (Marketplace)

41. App Service – four categories of app

42. Point-n-click in the Azure portal

43. Big data & analytics

44. Elastic database resource provisioning

45. Elastic database management

46. Data lake concept

47. Event data and analytics

48. Code example, Azure Stream Analytics
{“XO”, 4, “Win10”} {“Jo”, 0, “Surface”} {“Foo”,4, “Bing”}{“Dip”, 2, “XBox”}
{“XO”, 0, “Win10”} {“Dip”, 0, “Xbox”}{“Jo”, 4, “Surface”} {“Foo”, 0, “Bing”}Twitter Stream:
(same stream,
further down the timeline)
SELECT TS1.UserName, TS1.Topic
FROM TwitterStream TS1 TIMESTAMP BY CreatedAt
JOIN TwitterStream TS2 TIMESTAMP BY CreatedAt
ON TS1.UserName = TS2.UserName AND TS1.Topic = TS2.Topic
AND DateDiff(second, TS1, TS2) BETWEEN 1 AND 60
WHERE TS1.SentimentScore != TS2.SentimentScore
“List all users and the topics on which they switched their sentiment within a minute“

49. Parallelized stream analytics of event hubs
WITH Step1 AS (
SELECT Count(*) AS CountTweets, Topic
FROM TwitterStream PARTITION BY PartitionId
GROUP BY TumblingWindow(second, 3), Topic, PartitionId
),
Step2 AS (
SELECT Avg(CountTweets)
FROM Step1
GROUP BY TumblingWindow(minute, 3)
)
SELECT * INTO Output1 FROM Step1
SELECT * INTO Output2 FROM Step2
SELECT * INTO Output3 FROM Step2
• A querycan have multiplesteps
to enable pipeline execution
• A step is a sub-query defined
using WITH (“common table
expression”)
• Can be used todevelop complex
queries more elegantly by
creating a intermediary named
result
• Creates unit ofexecution for
scaling outwhen PARTITIONBY
is used
• Each step’s outputcan besent to
multiple outputtargets using
INTO

50. Azure Machine Learning Studio

51. ML Studio - detail

52. Resources
• Everything Azure @ Build2015
• https://channel9.msdn.com/Events/Build/2015?sort=status&direction=asc&t
erm=azure#tab_sortBy_status
• MS Azure Blog & Announcements
• https://azure.microsoft.com/blog/
• All links referenced for this slide pack =>
• Try-an-app :
• https://tryappservice.azure.com/
• Azure storage explorers
• http://blogs.msdn.com/b/windowsazurestorage/archive/2014/03/11/window
s-azure-storage-explorers-2014.aspx