Community day 2013 applied architectures


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Community day 2013 applied architectures

  1. 1. #comdaybeApplied ArchitecturesPanagiotis “Panos” KefalidisWindows Azure MVP@pkefal
  2. 2. Who is Panos?• Panagiotis “Panos” Kefalidis• Founder of VoxVoi• Windows Azure MVP• AZUG Board Member, Azure Insider• Microsoft vTS•• @pkefal –• Add your favorite title here
  3. 3. Setting the expectations• You will not–See code–Find an answer to everything• You will–Get a better understanding of what applies on WindowsAzure–You will see real architectures we propose to customers–You will see applied techniques of specific products
  4. 4. Now that everybody is excited..
  5. 5. Finding the right serviceSmall checklist
  6. 6. It can be like that..
  7. 7. Poor man’s checklist - Messaging• Do you need transactions?• Do you need bigger messages?• Do you have specific scalability targets?• Does it have to be cost effective vs performance?• Do you need interoperability out-of-the-box?
  8. 8. Poor man’s checklist – Front-end apps• Do you need fast deployments?• Do you need easy tests of prototypes?• Do you need CI out-of-the-box?• Do you need special software to make your webapp run?• Do you need session state on the web app?
  9. 9. Poor man’s checklist – PaaS vs IaaS• Do you need installs longer than 30 mins?• Do you need persistence on the disk itself?• Fulfill your requirement by automating aninstall/customization during startup?• Persistent disks mounted on OS level, not applevel?• Guaranteed IOPS per disk?
  10. 10. Poor man’s checklist – Security/Connectivity• Do you need identity federation?• Two factor authentication on your portal?• Graph API to discover objects in your directory?• Private IPs and ACLs on public endpoints?• Secure connection on-premise to cloud?• Choose your own internal VIPs
  11. 11. Many services, different roles• Windows Azure Service Bus• Windows Azure Storage Queues• Windows Azure IaaS• Windows Azure Virtual Network (VNET)• Windows Azure VPN• Windows Azure _we_can_go_on_forever
  12. 12. Some alternatives• MongoDb – Offered on Windows Azure through theStore• Redis – Offered with a ready to use script fordeployment
  13. 13. MongoDb• Document based database• Still has queries, secondary indexes etc.• Alternative to Windows Azure Tables• Less hassle to do stuff• Automatic capabilities (scaling, sharding andothers)• Much more expensive though
  14. 14. Redis• In memory database (key value pairs)• Insanely fast• It can be queried• Highly available if you set it up correctly• It has Windows Azure Support from Microsoft OSS
  15. 15. Redis• You can do Pub/Sub• You can have transactions• You can have distributed locks out of the box.–Increment counters from multiple threads etc.
  16. 16. Redis vs Windows Azure Role Cache• Redis is faster• Redis can be queried (real queries, not tags)• Redis has smaller footprint• Redis can be deployed and be connected cross-DC• Redis can be access from the outside world ifnecessary
  17. 17. Redis vs Windows Azure Role Cache• Redis can persist data as well–It’s recommended to do AOF and only on Slaves• Windows Azure Role Cache is cheaper–You need dedicated roles on Redis to haveperformance• It’s that tiny easier to enable Windows Azure RoleCache• You don’t have to maintain anything yourself onWindows Azure Role Cache
  18. 18. Things to remember as wellVisiting a customer
  19. 19. During your first meeting..
  20. 20. Source of problems• Legacy• Interoperability• 3rd party vendor software• Custom software• Company policies and human factor
  21. 21. Legacy• Legacy technologies– .NET 1.1, Windows 2000, Windows 2003, Exchange 2003• Legacy mindset– 80s approach on problems e.g. synchronous operations– Adding hardware to solve performance issues• Legacy deployment models/tools– Out-dated tools, like SourceSafe– Continuous integration vs Manual deploy– Paper work
  22. 22. Interoperability• Existing integration–Tightly coupled integration between systems–Bad implementations/practices on Enterprise ServiceBus• Out-dated technology–Old tools and implementations–Different implementations for each platform(.NET/JAVA)–No cloud support (PaaS model)
  23. 23. 3rd party vendor software• No cloud readiness– Unattended installs for PaaS support– Sticky sessions– In-memory state/cache– File system persistence• Unwillingness– To fix the issues• Willingness– To sell new versions instead
  24. 24. Custom software• Not cloud ready– Same problems as the vendors• Developers not trained– No experience with HA systems– No experience with de-coupled architectures– No experience with asynchronous operations– Learn how to embrace failure• Developers get offended– A significant mindset change causes denial
  25. 25. Finding who is responsible…
  26. 26. Examples of architecturesTo the real stuff
  27. 27. Example 1
  28. 28. Read Only ADReplication (VPN)Load BalancerCustomer HQVPN
  29. 29. Distributed cacheRead Only ADOffice 365WA StorageWA SQL DB
  30. 30. Service BusMeter 1 Meter 2Meter 1 Meter 2Service BusPolling PollingCountry 1Country 2FilterFilterPollingPollingMessage processing
  31. 31. Example 2
  32. 32. LegendEnd-UserWeb SiteModeratorWeb SiteEnd-UserBlob Storage(Uploaded Content, Generated Mezzanine File, Thumbnails, Streaming Content, Message Queues,Encoding profiles)Content structure:/{CAMPAIGN}/{USER}/{DATE}/ORIGINAL_FILE/{FILE_NAME}/{CAMPAIGN}/{USER}/{DATE}/METADATA/{FILE_NAME}/{CAMPAIGN}/{USER}/{DATE}/MEZZANINE_MASTER/{FILE_NAME}/{CAMPAIGN}/{USER}/{DATE}/THUMBNAILS/{FILE_NAMES}/{CAMPAIGN}/{USER}/{DATE}/STREAMING/{SEGMENTS}/{CAMPAIGN}/EncodeProfiles/{NAME}.xmlUPLOADED QUEUE7. Read Messagefrom QueueMedia ServicesDONE/ARCHIVE QUEUEOrchestratorSQL Server(Metadata & Status)13. Update Publish State (Success)13. Update Publish State (Error)Customer modERROR QUEUE11. Put Message on Queue12. Read Message from Queue9. Put message on Queue10. Read message from QueueAzureAPPROVE QUEUE1. View All Content3. View Details (Item)5. Approve (Item)2. Fetch Metadata4. Fetch Tumbnails6. Put Messageon QueueCustomer s publishingsystem8. Execute Job10. Return Success10. Return ErrorPUBLISHED QUEUE 11. Put Message on Queue12. Read Message from QueueCustomer smedia platformCustomer9. File Drop2. Upload video1. Upload video3. Put messageon Queue4. Read Messagefrom Queue7. Create Job9. Read Success8. Read Error5. Create video record with metadata10. Update video metadata11. Update video metadata (Error).Needs admin attention7. Drop final file11. Put message into Queue6. Read job metadata(encode profile .xml)Admin flowUser Flow
  33. 33. Example 3
  34. 34. Startup scripts automation• We connect to a service and get a SAS• We use that to securely access storage• We download JDK, JBoss and EAR/WAR• Unzip/install as necessary• Continue with other tasks
  35. 35. Worker Role IaaSWeb Roleabstraction controlWR1ProxyWR2JBoss AS 7 AP+AAJBossAS binariesJBossconfigurationEAR(s) andWAR(s)
  36. 36. Moving JBoss to Windows Azure• jGroups discovery uses PING• mod_cluster discovery by advertising• Both are multicast protocols• Multicast is not supported on Windows Azure
  37. 37. PING solutionWe implemented “AZURE_PING” as ajGroups protocol
  38. 38. mod_cluster solutionAzure advertising AAProxyJBoss AS 7 AAProxyJBoss AS 7 AAProxyJBoss AS 7 AAMetadata-------------------------------------------------
  39. 39. ProxyJBoss AS 7AAWR1WR2ProxyJBoss AS 7AAProxyJBoss AS 7AAJBoss AS 7AAJBoss AS 7AAJBoss AS 7AAJBoss AS 7AAScaling
  40. 40. Results• Highly scalable JBoss clustered solution• We have state if we need it• They can add/remove instances–Helps them reach their cost targets• Easy upgrades using VIP SWAP• Easy deployment of new packages–Just restart the instance, the script will do the rest
  41. 41. Anything to ask?