MongoDB on Windows Azure


Published on

Mark Eisenberg's presentation at Mongo Boston on September 20, 2010

Published in: Technology
  • Be the first to comment

MongoDB on Windows Azure

  1. 1. MongoDB on Windows Azure<br />WIP Report<br />MongoBoston September 2010<br />
  2. 2. Brief Survey<br />Business Decision Makers?<br />IaaS (ex. AWS or RackSpace) users<br />
  3. 3. Management of Resources: What you manage<br />Platform as a Service (PaaS)<br />Software as a Service (SaaS)<br />PRIVATE<br />(On-Premise)<br />Infrastructure<br />as a Service (IaaS)<br />Applications<br />Applications<br />Applications<br />Applications<br />You Manage<br />You Configure<br />Development & Runtime Kernels<br />Development & Runtime Kernels<br />Development & Runtime Kernels<br />Development & Runtime Kernels<br />You Manage<br />Databases<br />Databases<br />Databases<br />Databases<br />Security, Management, Load Balancing & Integration<br />Security, Management, Load Balancing & Integration<br />Security, Management, Load Balancing & Integration<br />Security, Management, Load Balancing & Integration<br />Logical Servers, Storage<br />Logical Servers, Storage<br />Logical Servers, Storage<br />Logical Servers, Storage<br />You Manage<br />Managed by Vendor<br />Managed by Vendor<br />Virtualization<br />Virtualization<br />Virtualization<br />Virtualization<br />OS<br />OS<br />OS<br />OS<br />Managed by Vendor<br />Server Hardware<br />Server Hardware<br />Server Hardware<br />Server Hardware<br />Networking, Utilities, Physical<br />Networking, Utilities, Physical<br />Networking, Utilities, Physical<br />Networking, Utilities, Physical<br />
  4. 4. The Windows Azure Platform<br />Windows Azure is an internet-scale cloud services platform hosted in Microsoft data centers around the world, proving a simple, reliable and powerful platform for the creation of web applications and services. <br />
  5. 5. Defining the Windows Azure Architecture<br />
  6. 6. Compute Services in Windows Azure<br />GOAL: Massive Scalability<br />Two role types: Web Role & Worker Role<br />Windows Azure applications are built with <br />web roles, worker roles, or a combination of both deployed to a number of instances.<br />Scale out – not up – by replicating worker instances as needed.<br />Allow applications to scale <br />user and compute processing independently.<br />Each instance runs on its own VM (virtual machine), replicated as needed<br />
  7. 7. BLOBS: <br />Provide a simple interface for storing named files along with metadata for the file.<br />QUEUES: <br />Provide reliable storage and delivery of messages for <br />an application.<br />TABLES: <br />Provide structured storage. A table is a set of entities which contain a set of properties.<br />Blobs, Tables, Queues and Drives<br />DRIVES: <br />A durable NTFS file system volume, sharable across instances.<br />
  8. 8. Running even one Mongo instance<br />Azure runs Windows 2008 server VMs<br />.NET 4 included. All else must be bundled.<br />XCOPY = GOOD. Registry and MSI = BAD<br />MongoDB = XCOPY = GOOD!!!<br />Mongo needs disk storage.<br />Choose: non-durable or durable<br />Local disk: non-durable. Blob: durable!<br />Mongo needs connection port<br />Multiple instances can’t talk to each other<br />Client can’t choose server instance<br />Can’t scale – multiple instances are independent!<br />Azure uses random ports – Mongo needs to use this port<br />Mongod won’t allow mapping of http server port!<br />
  9. 9. Single-Instance Solution<br />Single worker role instance<br />Local (non-durable) or blob (durable) storage<br />Single port mapped to mongod.exe<br />Server only; no web server access<br />Mongod instance as spawned process<br />Not as service<br />Must specify mapped port, data path, no http<br />If server exits, recycle instance<br />
  10. 10. Here’s How It’s Done<br />var storage = RoleEnvironment.GetLocalResource("MongoDBStorage");<br />varstoragePath = storage.RootPath;<br />string mongoRoot = Path.Combine(Environment.GetEnvironmentVariable("RoleRoot") + <br /> @"", @"approotMongoDB");<br />intport = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["MongoIn"].IPEndpoint.Port;<br />varcmdline = String.Format( "--nohttpinterface --port {0} --dbpath {1} ", port, storagePath);<br />MongoProcess= new Process()<br />{<br />StartInfo= new ProcessStartInfo(Path.Combine(mongoRoot, @"mongod.exe"), cmdline)<br /> {<br />UseShellExecute= false,<br />WorkingDirectory= mongoRoot<br />}<br />};<br />MongoProcess.Start();<br />
  11. 11. Managing Instances<br />MongoD.exe provides status<br />http (28017), mongod shell<br />Azure Load Balancer hides instances!<br />Single IP/Port inbound from client apps<br />Multiple port round-robin internally<br />Azure-hosted apps can access instances!<br />Must treat all instances equally<br />Assuming specific instance access = bad<br />
  12. 12. Client works against single IP/Port<br />No way for client app to access individual server<br />No shared storage<br />
  13. 13. Multiple servers accessible via Azure server<br />Mainly for management purposes<br />
  14. 14. Replica Sets Challenges<br />No shell access<br />how to configure?<br />Single IP <br />How to access?<br />How to monitor?<br />DB access<br />How to access Master in set?<br />What about storage?<br />
  15. 15. Replica Set Solution<br />ReplicaSet worker role<br />Runs mongod with –replSet<br />Management role (either worker or part of web)<br />Enumerates all ReplicaSet role instances<br />Builds configuration document<br />Connects to one node; uploads configwhich initializes replica set<br />Client application<br />Connects to replica set via compatible driver<br />Storage<br />Either local storage or blob storage.<br />If blob storage, each replica set node has its own blob<br />
  16. 16. Areas to be Explored<br />Replica set per deployment<br />Self configuring replica sets and shards<br />Configuration data held in WA Storage<br />Instance and Replica Set information pushed<br />Mongo HTTP port configuration<br />
  17. 17. Coming Soon to a Cloud Near You<br />RDP access to instances<br />Mongo Sharding will be simpler to implement than replication due to its cloud friendly architecture<br />