MagnetoDB
Key-Value Storage
service for
OpenStack
Mirantis, 2014
Serge.Kovaleff
gmail.com
linkedin.com
facebook.com
@
sviridov.ilya@gmail.com
Odessa, GeeksLab 4/12/2014@
isviridov at Free...
WII FM
WII F
WII FM
What the heck is OpenStack?
What the heck is OpenStack?
to be continued ...
Is it Big?
Community
1227 active contributors in IceHouse
more than 166 organizations
Communication
44 active official IRC...
Is it Big?
Is it about data?
CERN
- 3 PB per custer
- 50000 + 35000 core
PayPal 80000 VMs (according to Forbes)
OpenStack Big Data
OpenStack Sahara - Elastic Hadoop clusters
provisioning and management on OpenStack and elastic data
pr...
What is MagnetoDB?
MagnetoDB - OpenSource Amazon DynamoDB API
implementation for OpenStack, the key-value database service...
MagnetoDB key features
● Easy-to-integrate REST-like API (AWS SDK, boto
clients)
● OpenStack standards following REST API
...
MagnetoDB use cases
● High request traffic level applications
● Time series data
● Queries are distributed across the data...
Architecture highlights
● Layered architecture
● Horizontally scalable
● Pluggable database backend
● Uses native OpenStac...
Overall architecture
Integration with OpenStack services
Current status
○ 2.0.2 version released
○ Open source, Apache 2 license
○ Launchpad homepage: https://launchpad.net/magnet...
MagnetoDB Pilot Features
● Table CRUD API
○ ListTables
○ DescribeTable
○ CreateTable
○ DeleteTable
● Item CRUD API
○ PutIt...
MagentoDB use-case: dynamodb-geo
“Geo Library for Amazon DynamoDB
allows to easily create and query
geospatial data”
Next steps
● General
○ Move the development to OpenStack Infra - done
○ Integration with OpenStack Keystone - done
● Magne...
Summary
● The 2.0.2 implementation is available on GitHub in public
repository
● MagnetoDB is compatible with AWS SDK and ...
Links and Resources
○ Wiki: https://wiki.openstack.org/wiki/MagnetoDB
○ Launchpad: https://launchpad.net/magnetodb/
○ Sour...
Q&A
Got Questions?
ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов
ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов
ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов
ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов
Upcoming SlideShare
Loading in...5
×

ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов

9,528

Published on

Конференция "AI&BigData Lab", 12 апреля 2014

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,528
On Slideshare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

ManetoDB: Key/Value storage, BigData in Open Stack_Сергей Ковалев, Илья Свиридов

  1. 1. MagnetoDB Key-Value Storage service for OpenStack Mirantis, 2014
  2. 2. Serge.Kovaleff gmail.com linkedin.com facebook.com @ sviridov.ilya@gmail.com Odessa, GeeksLab 4/12/2014@ isviridov at FreeNode
  3. 3. WII FM
  4. 4. WII F
  5. 5. WII FM
  6. 6. What the heck is OpenStack?
  7. 7. What the heck is OpenStack?
  8. 8. to be continued ...
  9. 9. Is it Big? Community 1227 active contributors in IceHouse more than 166 organizations Communication 44 active official IRC channels 2887 e-mails in March
  10. 10. Is it Big?
  11. 11. Is it about data? CERN - 3 PB per custer - 50000 + 35000 core PayPal 80000 VMs (according to Forbes)
  12. 12. OpenStack Big Data OpenStack Sahara - Elastic Hadoop clusters provisioning and management on OpenStack and elastic data processing (on-demand Hadoop job workflow) OpenStack MagentoDB - key-value storage service
  13. 13. What is MagnetoDB? MagnetoDB - OpenSource Amazon DynamoDB API implementation for OpenStack, the key-value database service for storing any amount of data with seamless scalability and predictable performance.
  14. 14. MagnetoDB key features ● Easy-to-integrate REST-like API (AWS SDK, boto clients) ● OpenStack standards following REST API ● Schemaless, non-relational table-based model ● Put/get/query/scan item operations ● Eventual and strong consistency reads ● Local Secondary indexes ● Batch read/write operations ● Designed to handle any amount of data and any level of request traffic ● Seamless throughput and storage scaling ● Fault tolerance
  15. 15. MagnetoDB use cases ● High request traffic level applications ● Time series data ● Queries are distributed across the data ● Any other applications ● Migration of workloads between AWS and OpenStack
  16. 16. Architecture highlights ● Layered architecture ● Horizontally scalable ● Pluggable database backend ● Uses native OpenStack infrastructure ○ OpenStack HEAT for provisioning ○ OpenStack Neutron LoadBalancer ○ Integration with OpenStack Keystone planned
  17. 17. Overall architecture
  18. 18. Integration with OpenStack services
  19. 19. Current status ○ 2.0.2 version released ○ Open source, Apache 2 license ○ Launchpad homepage: https://launchpad.net/magnetodb ○ Code published on github: https://github. com/stackforge/magnetodb ○ Deployable to Vanilla OpenStack ○ Compatible with existing DynamoDB clients
  20. 20. MagnetoDB Pilot Features ● Table CRUD API ○ ListTables ○ DescribeTable ○ CreateTable ○ DeleteTable ● Item CRUD API ○ PutItem ○ UpdateItem ○ DeleteItem ○ GetItem ● Data querying API ○ Query ○ Scan ● Error Handling API
  21. 21. MagentoDB use-case: dynamodb-geo “Geo Library for Amazon DynamoDB allows to easily create and query geospatial data”
  22. 22. Next steps ● General ○ Move the development to OpenStack Infra - done ○ Integration with OpenStack Keystone - done ● MagnetoDB API, following OpenStack tenets - in progress ● DynamoDB API ○ Atomic counters ○ Asynchronous table creation ○ Atomic conditional write operations ○ Batch operations - in progress ● Other features ○ Management API ○ Quota management
  23. 23. Summary ● The 2.0.2 implementation is available on GitHub in public repository ● MagnetoDB is compatible with AWS SDK and boto library ● It is now possible to run applications which use DynamoDB as a storage on top of OpenStack using MagnetoDB
  24. 24. Links and Resources ○ Wiki: https://wiki.openstack.org/wiki/MagnetoDB ○ Launchpad: https://launchpad.net/magnetodb/ ○ Source code: https://github.com/stackforge/magnetodb ○ IRC: #magnetodb @ FreeNode ○ AWS DynamoDB http://aws.amazon.com/dynamodb/ ○ dynamodb-geo https://github.com/awslabs/dynamodb-geo
  25. 25. Q&A Got Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×