Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Business Track: Building a Private Cloud to Empower the Business at Goldman Sachs

5,635
views

Published on

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
5,635
On Slideshare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
62
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Building a Private CloudtoEmpower the Business at Goldman Sachs
  • 2. What are we building with MongoDB?SecureDocs What is it? GS employees secureebriefcase Access from mobile andtraditional clients What tech backs it? MongoDB 2.2 and ApacheTomcat 7 Hardware load balancing Why Mongo? Completely user driventagging structure Out of the box HA2June 21, 2013 MongoNYC
  • 3. What are we building with MongoDB?Social PipeLine What is it? Internal social platform for quickinformation sharing Real time analytics platform forexternal social trends What tech backs it? MongoDB 2.2, Apache Kafka, Solrand Apache Tomcat 7 Commodity hardware on alllayers Why Mongo? Highly unstructured data acrossall possible social sources Sharding and performance3June 21, 2013 MongoNYC
  • 4. Why MongoDB? Scale out For Performance and Size Global Availability and Resiliency Statement-Level Transaction and Consistency Semantics Strong Consistency Where Needed Relaxed Consistency Where Possible Easy to Use Powerful APIs No ORM required 10genJune 21, 2013 MongoNYC 4
  • 5. Why MongoDB? “Sweet-spot” Between Filesystems and Relational Database Security Model Primary Keys and Secondary Indexes Replication and Sharding Highly Structured – but Not EnforcedJune 21, 2013 MongoNYC 5RDBMS
  • 6. ChallengesWe’re a bank …June 21, 2013 MongoNYC 6
  • 7. DaaS in a Private Cloud: Motivations Facilitate Scale out For Performance and Size Global Availability and Resiliency Rapid Deployment + Development Efficiencies and Economies of Scale “Late Affinity” of purpose Platform Version Infrastructure Agility Spare hardware On-boarding pipeline Supply-side Inventory Management Keep the platform “easy to use”June 21, 2013 MongoNYC 7
  • 8. DaaS in a Private Cloud: Challenges Building for unknown use cases Defining “shapes” Database platform specific hardware pools Virtualization + Shared tenancy Performance and scale considerations SSD Storage Security and Controls Integrated into on-boarding pipeline Audit Backups and Archive Off-host / large footprint Sensitive Data and Masking Inventory Management Location aware for geographic resiliencyJune 21, 2013 MongoNYC 8
  • 9. DaaS in a Private Cloud: Challenges #1 Challenge CPU :: Memory :: Storage :: Price Moving to “cloud” means limiting choice on these ratios Scale out for storage May over-allocate compute Scale out for compute May over-allocate storageJune 21, 2013 MongoNYC 9
  • 10. Onboarding MongoDB @ Goldman SachsBefore: MongoDB Cluster Topologies Not StandardizedDevOps Model, Informal User GroupsInformal 10gen EngagementVarious Versions of MongoDBAfter: Private Cloud Service w/ Standardized TopologiesFully Onboarded and Supported Database PlatformFormalized 10gen Relationship (via Database Group)Standardize on MongoDB Enterprise EditionJune 21, 2013 MongoNYC 10
  • 11. Engineering MongoDB for Private CloudSupply FlowProvisionVirtual MachineRegister Node asAvailableNodes are NOTconfigured forspecific clusterDemand FlowUser Orders ClusterBased on Primary& Resiliency RegionReserve Nodesfrom AvailableInventoryPerform operationsto give node“Personality”ConfigurationSeed First Node,Expand WithOthersBuild Based onInventoryDelivery of Clusterto RequestorJune 21, 2013 MongoNYC 11
  • 12. MongoDB for Private CloudTopologyRequired global topology for out of region resiliency (min = 3 nodes)Each cluster is considered a “building block” for larger sharded clustersMongoC and MongoS co-located with MongoDShardingTeams encouraged to consider Shard Key even if no sharding plansSharding is the only supported way to grow (fixed internal storage)Provided with a single Shard by defaultMonitoring&Self ServiceCustom Monitoring StackOrdering Automation and Developer Self-ServiceJune 21, 2013 MongoNYC 12
  • 13. MongoDB for Private CloudBackupPeriodic backups to object storageWorking toward Point in Time Recovery (PITR)SecurityKerberos ticket based authentication requiredAuthorization policies will continue to matureSupportDatabase team supports service offering, onlyUse cases that utilize “Sensitive Data” are not yet supportedJune 21, 2013 MongoNYC 13
  • 14. Private Cloud ChallengesUnlike PublicCloud• We don’t profitfrom underutilization• Our incentivesare different• This dramaticallyaffects our scaleout approachOne SizeFits All• Sharding isprimary strategyfor growing• Both small andlarge apps wasteresources• CPU/Memory vs.StorageMongoDBCloud Goals• Utilize availableresourcesefficiently• Maintaincustomizationexpected byusers• Maintainease of useIdeal ShapeDiffers by App• SmallImpedancemismatchpreferred if itenables scale• Evaluate moreshapes ifmismatch isegregiousJune 21, 2013 MongoNYC 14Take a < $10,000 Machine, Split it 1,2,4,8 ways and Build MongoDB Service...
  • 15. Looking Forward: Cloud-Oriented Feature Requests• Multi-tenancy on shared data repositoriesBetterSecurity Models• Addresses a broader array of use casesEnhancedMulti Master• Increase utilization of fixed storageCompression• Object StorageOff-host Backups• Address Shape Mismatches?Better Shard Sizing• Introduce more “Named Resource” conceptsNamed Clusters15June 21, 2013 MongoNYC
  • 16. Questions?June 21, 2013 MongoNYC 16