Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experiences using CouchDB inside Microsoft's Azure team


Published on

Co-presented with Will Perry (@willpe). Real-world experiences using CouchDB inside Microsoft, and also how to get started with CouchDB on Microsoft Azure.

Published in: Internet, Technology
  • Be the first to comment

  • Be the first to like this

Experiences using CouchDB inside Microsoft's Azure team

  1. 1. CouchDB at Microsoft We’re running it in Azure, and so can you
  2. 2. Will Perry @willpe Windows Azure Group Brian Benz @bbenz Microsoft Open Tech
  3. 3.  An app we’ve built  How we built it. Some tips and tricks  Share some experience on helping folks get used to non-relational databases.  Open Source at Microsoft  Running CouchDB in Windows Azure
  4. 4. the app.  Daylight – a test automation system for the cloud  Stores metadata about every revision of every test case in our product  About 10k tests updated about 6 times a day  Stores details of test runs and results  About 5k results stored 6 times a day  100% in the cloud
  5. 5. the database. Must haves:  Scale to millions/billions of documents  Fairly loose schema  Not a single point of failure  Multiple indexes – really fast access to docs Nice to haves:  Web friendly  Aggregate oriented  Http Interface  Easily partitionable SQL TABLE COUCH
  6. 6. CouchDB was the most natural fit for the data we had and the app we were building …and the Apache 2.0 license helped keep the lawyers happy
  7. 7. the architecture. Multi-node deployment with application server and couch db running on each node and replicating across the deployment Node 0 Load Balancer Node 1 Node 2 VHD VHD VHD
  8. 8. time-partitioned data. Mar-2013 Feb-2013Apr-2013 Jan-2013 Test data becomes exponentially less useful over time. Part of our scale strategy is deleting old data in a ‘sliding-window’ fashion.
  9. 9. semi-static typing. [JsonProperty("attributes")] public JObject Attributes { get; set; } [JsonProperty("full_name", Required = Required.Always)] public string FullName { get; set; } [JsonProperty("priority")] public int Priority { get; set; } Strong typing when we want it, loose typing when we don’t
  10. 10. http. using (var httpClient = new HttpClient(couchBaseUri)) { return await httpClient.GetAsync(resultId); } We didn’t feel the need to use a client library, and just extended the built-in http stack to encapsulate some of our conventions
  11. 11. design documents.  In the source tree  Used a directory tree to map to the json tree  Compile into a json document at start-up  Compare to existing design doc and overwrite if different
  12. 12. culture shock. Map/Reduce != T-SQL Help out SQL junkies by talking in terms of the index.
  13. 13. experience. We really like CouchDB. It’s been really reliable, simple to manage and reduced our implementation costs. Sometimes indexing can slow us down and there’s a reasonable ramp-up required for new devs on the project. Overall, we’re confident we made the right choice.
  14. 14. Wholly owned subsidiary of Microsoft To advance Microsoft’s investment in openness Interoperability Open source Open standards
  15. 15. Windows Azure: What It Is Flexible
  16. 16. Windows Azure Virtual Machines Windows Server 2008 R2 Windows Server 2012 OpenSUSE 12.1 CentOS 6.2 Ubuntu 12.04 SUSE Linux Enterprise Server SP2
  17. 17. VM Depot  Call to action: Developers can use VM Depot to reach new audiences!
  18. 18. Windows Azure: What It Is
  19. 19. Deployment Complexity Operational Responsibility Flexibility OS Security IaaS Some High High Linux or Windows Custom PaaS Some Low High Windows only Built-in SaaS Low/none None Limited N/A Built-in What’s the Difference?
  20. 20. Virtual Machine Sizes VM Size CPU Cores Memory Bandwidth (Mbps) # Data Disks Extra Small Shared 768 MB 5 1 Small 1 1.75 GB 100 2 Medium 2 3.5 GB 200 4 Large 4 7 GB 400 8 Extra Large 8 14 GB 800 16
  21. 21. Supported Web Frameworks
  22. 22. Supported Publishing Methods
  23. 23. Windows Azure Developer Center  Open-source SDKs for popular programming languages, to get you up and running quickly  Choice of popular IDEs such as Visual Studio and Eclipse  Consistent REST protocols and APIs across Windows Azure services .NET driver Node.js driver Java driver PHP driver Python driver Driver download page
  24. 24. azure topic verb options Command Line Syntax Overview prompt> account account location account affinity-group vm vm disk vm endpoint vm image service service cert site config download import list show delete start restart shutdown capture create attach detach browse set username password dns-prefix vm-name lb-port target-image-name source-path disk-image-name size-in-gb thumbprint value -v -vv
  25. 25. Example: Provisioning a Cluster azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531- en-us-30GB.vhd" username password -l "West US" -e azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531- en-us-30GB.vhd" username password -l "West US" -e 23 -c azure vm create couchsd "OpenLogic__OpenLogic-CentOS-62-20120531- en-us-30GB.vhd" username password -l "West US" -e 24 -c azure vm endpoint create couchsd 27017 27017 azure vm endpoint create couchsd-2 27018 27018 azure vm endpoint create couchsd-3 27019 27019
  26. 26. Getting Started Sign up for a free trial Learn about the latest features
  27. 27. Will Perry @willpe Windows Azure Group Brian Benz @bbenz Microsoft Open Tech