Uploaded on

NoSQL databases implemented using GT.M

NoSQL databases implemented using GT.M

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,469
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
37
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. M/DB & M/DB:X Open Source NoSQL Projects based on GT.M Rob Tweed: M/Gateway Developments Ltd
  • 2. M/DB & M/DB:X
    • What are they?
    • Why might you use them?
    • What is GT.M?
    • Why are they based on GT.M?
    • What should you be interested in GT.M?
  • 3. M/DB
      • Free Open Source alternative to SimpleDB
      • API-compatible with SimpleDB
        • Behaves identically
        • Accessed identically
        • Just a different endpoint URL
      • Designed for:
        • Alternative Public Cloud to Amazon
        • Private Cloud
        • Local use
  • 4. Why emulate Amazon SimpleDB?
    • Great concept for data that fits its model
      • “ spreadsheet-like” data structure
      • Simple to use
      • Simple to manage
    • Potential for freedom from SimpleDB’s limits
    • Make SimpleDB model no longer exclusive to Amazon cloud
    • Potential for hybrid environment
  • 5. Why emulate Amazon SimpleDB?
    • Great concept for data that fits its model
      • “ spreadsheet-like” data structure
      • Simple to use
      • Simple to manage
    • Potential for freedom from SimpleDB’s limits
    • Make SimpleDB model no longer exclusive to Amazon cloud
    • Potential for hybrid environment
    • Because we can…!
  • 6. SimpleDB Model
    • Domains: individual tables or databases
    • Items: rows in a table
    • Attributes: columns in a table
    • Values: cells (note: an attribute can have multiple values)
  • 7. Main SimpleDB Operations
    • CreateDomain
    • PutAttributes
    • List Domains
    • GetAttributes
    • Select
    • DeleteDomain
    • DeleteAttributes
  • 8. SimpleDB v M/DB Limits Unlimited 5 seconds Max query execution time XML or JSON over HTTP XML over HTTP Responses Immediately Consistent Eventually Consistent Consistency Free Open Source Pay as you go, can get expensive! Cost Dynamic data type, 4k – 32k max (depends on GT.M block-size) Text only, 1k max size Attributes 512Gb – 8Tb (depends on GT.M block-size) 10Gb Max domain size Effectively unlimited 100 per account No of domains M/DB SimpleDB
  • 9. SimpleDB Client m_apache gateway + xinetd GT.M M/DB Security M/DB Methods Ubuntu (or other) Linux Physical or Virtual Machine HTTP Request XML Response creation HTTP Response M/DB Database m_python gateway Python M/DB APIs Web Server (Apache)
  • 10. M/DB: Availability
    • Free Debian Installer at www.mgateway.com
      • Installs entire architecture
        • GT.M
        • Apache & m_apache
        • M/DB
    • Ubuntu Enterpr
      • Private cloud
        • Eucalyptus used to emulate Amazon APIs
      • M/DB now available in the UEC Image Store
        • Adds SimpleDB API to UEC’s emulation of Amazon cloud
  • 11. M/DB Client Support
    • Many SimpleDB Clients already support M/DB:
      • Bolso
      • Mindscape's SimpleDB Management Tools and LightSpeed
      • Boto (Python client)
      • Others?
        • Just requires ability to change the endpoint URL
    • M/DB emulates SimpleDB API precisely
  • 12. M/DB:X
  • 13. M/DB:X
    • Simple Native XML Database
      • Persistent XML DOM model
      • Multiple XML documents
    • API based on SimpleDB
      • Uses SimpleDB security
      • W3C XML DOM API, expressed as:
        • HTTP request, XML response
        • JSON over HTTP
      • XPath, also expressed as:
        • HTTP request, XML response
        • JSON over HTTP
  • 14. M/DB:X
    • Creating DOMs:
      • Parse an XML document
        • Non-validating parser
      • JSON input
        • Converted to corresponding XML
      • Build programmatically using DOM API methods, eg
        • createDocument
        • createElement
        • createTextNode
        • appendChild
        • insertBefore
  • 15. M/DB:X
    • Modifying DOMs:
      • programmatically using DOM API methods, eg
        • createElement
        • createTextNode
        • appendChild
        • insertBefore
        • setAttribute
        • removeChild
        • removeAttribute
        • copyNode
  • 16. M/DB:X
    • Searching DOMs:
      • programmatically using DOM API methods, eg
        • getElementById
        • getElementsByTagName
        • getAttribute
        • getFirstChild
        • getNextSibling
      • XPath
        • Individual DOM
        • Across multiple DOMs
  • 17. M/DB:X
    • Deleting DOMs:
      • removeDocument
      • Otherwise a DOM will persist indefinitely
  • 18. M/DB:X
    • Identifying DOMs:
      • Each DOM has a unique:
        • DocumentName
        • DocumentOID
    • Identifying Nodes:
      • Each node in a DOM has:
        • nodeType
        • nodeOID (unique across all DOMs)
  • 19. M/DB:X
    • Typical API call (XML)
      • Request:
        • db=mdbx &
        • Action=SetAttribute &
        • AttributeName=width &
        • AttributeValue=100px &
        • ElementOID=1-3 &
        • MDBAccessKeyId=[your access key id] &
        • Signature=[computed signature] &
        • SignatureMethod=HmacSHA256 &
        • SignatureVersion=2 &
        • Timestamp=2009-06-14T13%3A19%3A41 &
        • Version=2009-04-15
      • Response:
        • <?xml version='1.0'?>
        • <SetAttributeResponse xmlns=&quot;http://mdbx.mgateway.com/doc/2009-06-05/&quot; />
  • 20. M/DB:X
    • Typical API call (JSON)
      • Request:
        • db=mdbx &
        • Action=SetAttribute &
        • JSON={AttributeName:”width”,AttributeValue:”100px”, ElementOID:”1-3”} &
        • MDBAccessKeyId=[your access key id] &
        • Signature=[computed signature] &
        • SignatureMethod=HmacSHA256 &
        • SignatureVersion=2 &
        • Timestamp=2009-06-14T13%3A19%3A41 &
        • Version=2009-04-15
      • Response:
        • {ok:true}
  • 21. M/DB:X Hybrid JSON/XML Database
    • JSON input:
      • {animals:{dog:[{name:&quot;Rufus&quot;,breed:&quot;labrador&quot;},{name:&quot;Marty&quot;,breed:&quot;whippet&quot;}], cat:{name:&quot;Matilda&quot;}}}
    • XML representation in DOM
      • <animals>
        • <dog>
          • <name>Rufus</name>
          • <breed>labrador</breed>
        • </dog>
        • <dog>
          • <name>Marty</name>
          • <breed>whippet</breed>
        • </dog>
        • <cat>
          • <name>Matilda</name>
        • </cat>
      • </animals>
    • Output as:
      • XML
      • JSON
  • 22. M/DB:X: Use cases
    • XML meets SimpleDB and CouchDB!
    • Lightweight XML “scratchpad”
    • Simple, easy to use XML DOM
    • Perfect for storage/manipulation of hierarchical data
    • If you get XML documents in and send out XML documents, why convert to/from some database format?
      • Store as XML
  • 23. M/DB:X Client Support
    • None available to my knowledge
    • XML API based on SimpleDB model + SimpleDB security
      • Should be simple to adapt existing SimpleDB clients
  • 24. M/DB:X availability
    • Bundled with M/DB
      • Install M/DB and you also have M/DB:X
  • 25. M/DB and M/DB:X – Underlying technology
    • GT.M
  • 26. What is GT.M?
    • Open source database
    • Runs on Linux
    • Schema-free, key-pair database
    • Schema implicit in application
    • Persistent dynamic sparse arrays
      • Alphanumeric subscripts
      • ^employee(“london”,”mgt”,1234)=“Rob Tweed”
  • 27. M/DB and M/DB:X – Why GT.M?
    • Why GT.M was the ideal technology:
      • Flexibility of its database structure:
        • Very simple to represent SimpleDB model and its indices
        • Very simple to represent XML DOM
        • No pre-declared schema
        • Build/modify “on the fly”
          • No pre-declared array size
          • Completely dynamic
          • Automatic garbage collection
  • 28. M/DB and M/DB:X – Why GT.M?
    • Why GT.M was the ideal technology:
      • Free Open Source
      • Very high performance database
      • Highly scalable:
        • from netbook to replicated network of high-end servers
      • Rock-solid reliability
        • Its pedigree is in banking & healthcare systems since mid-1980s
        • It runs the world’s largest core banking system
          • >17 million accounts
  • 29. M/DB and M/DB:X – Why GT.M?
    • So GT.M is no toy!
      • It’s a serious, well tried and tested enterprise-grade database
    • M/DB and M/DB:X can make use of GT.M’s system management facilities:
      • Backup
      • Journalling
      • Replication (multi-tier)
  • 30. Why you should check out GT.M?
    • “ Swiss Army Knife of Databases”
    • Underlying persistent sparse array model:
      • Can be used to represent many other logical database structures:
        • Tables
        • Hierarchicial
        • XML
        • Relational
        • Object
    • Own built-in language, directly linked to database
    • Or can use most other languages, eg:
      • Python
      • Java
      • Ruby, etc
  • 31. Why you should check out GT.M?
    • Speed of development:
      • M/DB took 1 week to write and release
      • M/DB:X about the same
        • Documentation was the most time-consuming task!
    • Open source
      • Can redistribute freely on Linux
      • No limitations imposed
      • Full version, full functionality
  • 32. Find out more
    • M/DB & M/DB:X:
      • http://www.mgateway.com
    • GT.M:
      • http://www.fis-gtm.com