Successfully reported this slideshow.
Building a Private CloudtoEmpower the Business at Goldman Sachs
What are we building with MongoDB?SecureDocs What is it? GS employees secureebriefcase Access from mobile andtraditiona...
What are we building with MongoDB?Social PipeLine What is it? Internal social platform for quickinformation sharing Rea...
Why MongoDB? Scale out For Performance and Size Global Availability and Resiliency Statement-Level Transaction and Con...
Why MongoDB? “Sweet-spot” Between Filesystems and Relational Database Security Model Primary Keys and Secondary Indexes...
ChallengesWe’re a bank …June 21, 2013 MongoNYC 6
DaaS in a Private Cloud: Motivations Facilitate Scale out For Performance and Size Global Availability and Resiliency ...
DaaS in a Private Cloud: Challenges Building for unknown use cases Defining “shapes” Database platform specific hardwar...
DaaS in a Private Cloud: Challenges #1 Challenge CPU :: Memory :: Storage :: Price Moving to “cloud” means limiting cho...
Onboarding MongoDB @ Goldman SachsBefore: MongoDB Cluster Topologies Not StandardizedDevOps Model, Informal User GroupsInf...
Engineering MongoDB for Private CloudSupply FlowProvisionVirtual MachineRegister Node asAvailableNodes are NOTconfigured f...
MongoDB for Private CloudTopologyRequired global topology for out of region resiliency (min = 3 nodes)Each cluster is cons...
MongoDB for Private CloudBackupPeriodic backups to object storageWorking toward Point in Time Recovery (PITR)SecurityKerbe...
Private Cloud ChallengesUnlike PublicCloud• We don’t profitfrom underutilization• Our incentivesare different• This dramat...
Looking Forward: Cloud-Oriented Feature Requests• Multi-tenancy on shared data repositoriesBetterSecurity Models• Addresse...
Questions?June 21, 2013 MongoNYC 16
Upcoming SlideShare
Loading in …5
×

Business Track: Building a Private Cloud to Empower the Business at Goldman Sachs

7,679 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Business Track: Building a Private Cloud to Empower the Business at Goldman Sachs

  1. 1. Building a Private CloudtoEmpower the Business at Goldman Sachs
  2. 2. What are we building with MongoDB?SecureDocs What is it? GS employees secureebriefcase Access from mobile andtraditional clients What tech backs it? MongoDB 2.2 and ApacheTomcat 7 Hardware load balancing Why Mongo? Completely user driventagging structure Out of the box HA2June 21, 2013 MongoNYC
  3. 3. What are we building with MongoDB?Social PipeLine What is it? Internal social platform for quickinformation sharing Real time analytics platform forexternal social trends What tech backs it? MongoDB 2.2, Apache Kafka, Solrand Apache Tomcat 7 Commodity hardware on alllayers Why Mongo? Highly unstructured data acrossall possible social sources Sharding and performance3June 21, 2013 MongoNYC
  4. 4. Why MongoDB? Scale out For Performance and Size Global Availability and Resiliency Statement-Level Transaction and Consistency Semantics Strong Consistency Where Needed Relaxed Consistency Where Possible Easy to Use Powerful APIs No ORM required 10genJune 21, 2013 MongoNYC 4
  5. 5. Why MongoDB? “Sweet-spot” Between Filesystems and Relational Database Security Model Primary Keys and Secondary Indexes Replication and Sharding Highly Structured – but Not EnforcedJune 21, 2013 MongoNYC 5RDBMS
  6. 6. ChallengesWe’re a bank …June 21, 2013 MongoNYC 6
  7. 7. DaaS in a Private Cloud: Motivations Facilitate Scale out For Performance and Size Global Availability and Resiliency Rapid Deployment + Development Efficiencies and Economies of Scale “Late Affinity” of purpose Platform Version Infrastructure Agility Spare hardware On-boarding pipeline Supply-side Inventory Management Keep the platform “easy to use”June 21, 2013 MongoNYC 7
  8. 8. DaaS in a Private Cloud: Challenges Building for unknown use cases Defining “shapes” Database platform specific hardware pools Virtualization + Shared tenancy Performance and scale considerations SSD Storage Security and Controls Integrated into on-boarding pipeline Audit Backups and Archive Off-host / large footprint Sensitive Data and Masking Inventory Management Location aware for geographic resiliencyJune 21, 2013 MongoNYC 8
  9. 9. DaaS in a Private Cloud: Challenges #1 Challenge CPU :: Memory :: Storage :: Price Moving to “cloud” means limiting choice on these ratios Scale out for storage May over-allocate compute Scale out for compute May over-allocate storageJune 21, 2013 MongoNYC 9
  10. 10. Onboarding MongoDB @ Goldman SachsBefore: MongoDB Cluster Topologies Not StandardizedDevOps Model, Informal User GroupsInformal 10gen EngagementVarious Versions of MongoDBAfter: Private Cloud Service w/ Standardized TopologiesFully Onboarded and Supported Database PlatformFormalized 10gen Relationship (via Database Group)Standardize on MongoDB Enterprise EditionJune 21, 2013 MongoNYC 10
  11. 11. Engineering MongoDB for Private CloudSupply FlowProvisionVirtual MachineRegister Node asAvailableNodes are NOTconfigured forspecific clusterDemand FlowUser Orders ClusterBased on Primary& Resiliency RegionReserve Nodesfrom AvailableInventoryPerform operationsto give node“Personality”ConfigurationSeed First Node,Expand WithOthersBuild Based onInventoryDelivery of Clusterto RequestorJune 21, 2013 MongoNYC 11
  12. 12. MongoDB for Private CloudTopologyRequired global topology for out of region resiliency (min = 3 nodes)Each cluster is considered a “building block” for larger sharded clustersMongoC and MongoS co-located with MongoDShardingTeams encouraged to consider Shard Key even if no sharding plansSharding is the only supported way to grow (fixed internal storage)Provided with a single Shard by defaultMonitoring&Self ServiceCustom Monitoring StackOrdering Automation and Developer Self-ServiceJune 21, 2013 MongoNYC 12
  13. 13. MongoDB for Private CloudBackupPeriodic backups to object storageWorking toward Point in Time Recovery (PITR)SecurityKerberos ticket based authentication requiredAuthorization policies will continue to matureSupportDatabase team supports service offering, onlyUse cases that utilize “Sensitive Data” are not yet supportedJune 21, 2013 MongoNYC 13
  14. 14. Private Cloud ChallengesUnlike PublicCloud• We don’t profitfrom underutilization• Our incentivesare different• This dramaticallyaffects our scaleout approachOne SizeFits All• Sharding isprimary strategyfor growing• Both small andlarge apps wasteresources• CPU/Memory vs.StorageMongoDBCloud Goals• Utilize availableresourcesefficiently• Maintaincustomizationexpected byusers• Maintainease of useIdeal ShapeDiffers by App• SmallImpedancemismatchpreferred if itenables scale• Evaluate moreshapes ifmismatch isegregiousJune 21, 2013 MongoNYC 14Take a < $10,000 Machine, Split it 1,2,4,8 ways and Build MongoDB Service...
  15. 15. Looking Forward: Cloud-Oriented Feature Requests• Multi-tenancy on shared data repositoriesBetterSecurity Models• Addresses a broader array of use casesEnhancedMulti Master• Increase utilization of fixed storageCompression• Object StorageOff-host Backups• Address Shape Mismatches?Better Shard Sizing• Introduce more “Named Resource” conceptsNamed Clusters15June 21, 2013 MongoNYC
  16. 16. Questions?June 21, 2013 MongoNYC 16

×