Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NoSQL on AWS - Pop-up Loft Tel Aviv

Using various NoSQL technologies on AWS, best practices and common pitfalls: What is NoSQL? NoSQL databases overview, sizing considerations, availability and disaster recovery, backup & retention.

  • Login to see the comments

  • Be the first to like this

NoSQL on AWS - Pop-up Loft Tel Aviv

  1. 1. NoSQL at Scale: Architecture and Common Pitfalls Arthur Gimpel DataZone
  2. 2. About Me • Working with databases for the past 8 years • 5 years as SQL Server DBA & .NET • 3 years with NoSQL, Python, Node.js • 2015 - DataZone
  3. 3. DataZone Data is our business • High end consultancy & projects • Private and public training • Multi vendor MSP (Managed Service Provider) • Part of Matrix, CloudZone Family • Proud partners of:
  4. 4. NoSQL Why not?
  5. 5. All the wrong reasons.. • Web scale! web scale! • Cost reduction • Free school for developers
  6. 6. Common Pitfalls & Best Practices
  7. 7. Normalizing in the low cost storage era
  8. 8. Expectations gone wild • Stored procedures • Jobs • Triggers • Mailing Servers • Shell Commands
  9. 9. ODM != ORM ORM • Close the modelling gaps • Simple Standard CRUD ODM • Typing the untyped • Retries • Transactions • Events
  10. 10. Best Practices • DynamoDB • Streams: Change Data Capture • Cross region replication & synchronisation • Event driven development: Send your new user a welcome email • Polyglot Persistence: Initiate an ETL, update your Elasticsearch index
  11. 11. Best Practices • Elasticsearch • AWS Plugin: • Allow service discovery based on security groups • Run snapshots & restore directly to/from s3 • Storage: • In small clusters, use EBS. • In large clusters, use instance store, if you have replication enabled. • Leverage Watcher & AWS Lambda for anything!
  12. 12. Best Practices • Couchbase • XDCR: • Make sure you use cross datacenter replication for multi region deployments • Use encrypted communication, especially on XDCR • Sizing: • All members must be identical • Prefer Memory & IO Optimized for most deplyments
  13. 13. When Family Can Be Chosen • Every engine has its flavours - know your tradeoffs • Splitting drives is good for on prem. EBS is different • Scaling up & down is always X2 • Prefer linux AMI by AWS- always check for official support
  14. 14. Eventually… • Don’t work for technology. Make technology for you. Chose your engines wisely • Beware of your cluster’s distribution model - don’t over-estimate elasticity • Experimenting is easy: Change, Benchmark, Analyse. Repeat. • Run engine backups - not EBS snapshots
  15. 15. Thank You! Arthur Gimpel arthurgi@datazone.io

×