NoSQL and Cloud Services - Philip Balinow, Comfo

372 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
372
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

NoSQL and Cloud Services - Philip Balinow, Comfo

  1. 1. Introduction into Cloud Computing philip.balinov@komfo.com DevOps Engineer
  2. 2. SOME DEFINITIONS def. CLOUD COMPUTING def. BIG DATA
  3. 3. SOME DEFINITIONS def. CLOUD COMPUTING - distributed computing over a network - the ability to run a program or application on many connected computers at the same time. def. BIG DATA - data sets so large and complex that it becomes difficult to process using traditional data processing applications
  4. 4. SOME DEFINITIONS, CONTD. Q: No, seriously. What is the Cloud? A: Well, there are three types  Infrastructure-as-a-Service  Platform-as-a-Service  Software-as-a-Service
  5. 5. THE CLOUD – PROS AND CONS WHY CLOUD COMPUTING IS SO GREAT  Better hardware utilization  Economy of scale  Usage-based pricing  In-built resilience (here be monsters)  No front-up costs  No long-term contracts
  6. 6. Retail Finance E-Commerce Telecommunicatio n B2B Publishing/Media Government & NGO Automotive Travel KOMFO'S CLIENTS
  7. 7. KOMFO PLATFORM WORKFLOW OVERVIEW
  8. 8. EXTERNAL PROVIDERS
  9. 9. OUR TECHNOLOGY  PHP, Python, C  MySQL  MongoDB, Elasticsearch  Javascript (node.js)  Ruby  Freedom to use any tool fit for the job
  10. 10. KOMFO MAIN CHALLENGES Continuously changing workload Fast feature changes BigData (Predictive) Analytics Security Availability
  11. 11. HOW TO USE THE CLOUD  Automation  Horizontal scaling  Break tasks into many small sub-tasks  Synchronize all the workers  Write for eventual consistency
  12. 12. HOW TO USE THE CLOUD  Beware! There are traps* *Minimally shared with fair weighting. Allows burst when idle resources are available. In rare cases, resources may be throttled back under heavy host contention. – **Disk I/O is shared across the host. – ***A vCPU corresponds to a physical CPU thread. –
  13. 13. HOW TO USE THE CLOUD  There are more traps*  Coordinate tasks – good messaging system (AMQP, DB, MemCached)  Asynchronous task execution (see above, also API Callback hooks)  Implement transactions in software
  14. 14. CLOUD ARCHITECTURE  Messaging  Ensure communication between dynamic number of nodes  Message-oriented middleware  Exactly-once delivery  At-least-once delivery  Transaction-based delivery  Timeout-based delivery
  15. 15. MIX & MATCH  Crunch numbers in the cloud  Application servers  Slow running tasks  Temporary services  Test servers  Automation – automatic deployment of multi- tiered environments
  16. 16. MIX & MATCH, CONTD.  Traditional servers for:  Incompatible apps (single-threaded, memory, disk intensive, specialized hardware) do not work well in cloud environments  Database servers are best kept on dedicated machines
  17. 17. OK, so we have an (endlessly) scalable cloud app now. What are we forgetting?
  18. 18. DATABASES, NOSQL  def. NoSQL  a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases.
  19. 19. DATABASES  Postgres, Hadoop, MongoDB, Cassandra, Riak  In-memory dataset for faster operation  No predefined structure  Integrated sharding, load-balancing and failover  Versatility - can be used for anything from data storage to real-time messaging to search indexes
  20. 20. DATABASES  Use the best tool for the job depending on the task  NoSQL Advantages  Some sources generate a lot of data  Complex interconnections, cyclical dependencies  Aggregations must be performed on both new and old data  Structure of foreign sources may change on short notice
  21. 21. DATABASES  Use the best tool for the job depending on the task  NoSQL Disadvantages (Classic SQL advantages)  Not ACID compliant  No transactions  No relations between data  Lack of structure means aggregations are slow
  22. 22. DATABASES, LONG TERM STRATEGY  Data quickly becomes irrelevant  Archive it, but keep it accessible  Online Data Warehouse solutions  Amazon Redshift  Keep Everything  Terabytes for pennies
  23. 23. Summary The cloud rocks, mmkay?
  24. 24. Questions?

×