0
Service Entities Using
MongoDB
NoSQL Now 2013, San Jose
Arjen Schoneveld
Friday, August 23, 13
Friday, August 23, 13
Arjen Schoneveld
• co-Founder of Dexels,Amsterdam based
software company
• Architect of Sportlink Sports ERP system
• PhD ...
Our Story
• How it started: Sportlink Sport Management System
• Navajo: a Service Oriented Application Framework
• Limitat...
Sportlink Architecture (2003)
Sport Club
Application
Federation
Application
Federation
Website
Database (Oracle) Crystal
R...
Data Model
Object Model
Service Facade
Web Services
JSP/HTML/
Javascript/CSS
ORM
JAX-WS
SOAP/WSDL
Service Proxy
A Service ...
How to simplify?
Friday, August 23, 13
Services First
• Changes are “the rule” in many environments
• Most Services are CRUD and instances have
very short live-s...
Usable Services
• Loose Coupling: no client stub generation!
• Metadata is key ingredient: self describing
• Type, descrip...
What we came up with
Friday, August 23, 13
Data Model
The Navajo Application Framework
UIs
[Platform Agnostic
Declarative UI Language]
Canonical
Service
Language
Ser...
How we implemented it
Friday, August 23, 13
fd
[Resources]
Database
Mail Server
FTP Server
SOAP Service
[Adapters]
[Tipi]
UIW
idgets
UIEvents
Service
Listeners
CallSe...
How Sportlink Evolved...
Friday, August 23, 13
Sportlink Ecosystem (2013)
Club
(Tipi/Swing)
Association
(Java/Swing)
Website
(Third Party)
Oracle
Crystal
FTP
~4000 servi...
Growing Pains
• More end user applications
• More functionality
• More sport associations with different
requirements
• Mo...
Oracle Limitations
• License Costs!
• Unpredictable query behavior
• Only partly/expensive High Availability
• Poor afford...
Oracle
Mongo
Neo4j
MySQL
Dynamo
Cassandra
PostgreS
QL
MatchReport
Web Plugin
Mobile App
Club App
Association
App
Associati...
Our MongoDB Use Case
• Digital Match Report (Mobile,Web and
Desktop)
• Player Roster for home- and away teams
(role, capta...
Why MongoDB
• MatchReport application is introduced
gradually → Scalability → NoSQL
• MatchReport can be fetched and updat...
Home Away Official Mongo Oracle
Fetch Match Report
MatchReportReport Fetch Match Report
Report
Populate Home Players Report...
How do we integrate Navajo and
MongoDB?
Friday, August 23, 13
Navajo Services
• A simple XML/HTTP/POST-based Web Service
• Uses a Canonical Service Language: Navajo Document
• Consumes...
Navajo Document
• Navajo Services use Navajo Documents for all
message processing (Canonical Service Language)
• All data ...
Properties
• A property has metadata:
• name. Name must be unique within a message
• type. Natively supported types are: s...
Serializations
• The default serialization of Navajo Document is XML
• Different other serialization are supported, e.g. J...
A Navajo Document<tml>
<message name=”Person”>
<property name=”Lastname” length=”100” description=”Lastname of the person”...
Collections:Array Messages
<tml>
<message name=”Persons” type=”array”>
<message name=” Persons” type=”definition”>
<propert...
Club
Application
[Tipi]
QueryMatch
Oracle
2. Request
<tml rpc_name=”QueryMatch”>
<message name=”Match”>
<property name=”Id...
<navascript>
<validations>
<check code=”Empty Id”>
[/Match/Id] != ‘’
</check>
</validations>
<map.sqladapter>
<sqladapter....
Friday, August 23, 13
MongoDB
• MongoDB is a document-based NoSQL database
• MongoDB is schema-less, unlike SQL databases that
rely on a predefin...
Navajo and MongoDB
• Store Navajo Document directly in
MongoDB
• Navajo Document can be easily
transformed to JSON
• No ne...
JSONify Navajo Document
<message name=”MatchReport”>
<property name=”MatchId” value=”1” type=”integer”/>
<property name=”H...
Query MongoDB
• Entity in MongoDB can be queried using MongoDB query syntax
• Entity in MongoDB can be queried using a Nav...
<navascript>
<message name=”MatchReport”>
<property name=”MatchId” value=”1” type=”integer”/>
</message>
<map.navajomap us...
Result of Query
<message name=”MatchReport”>Unique Id
<property name=”_id” value=”aabf3fdde323g”/>
<property name=”MatchId...
How can we resolve this?
Friday, August 23, 13
Query using Entity
Service
• Define a Navajo Service for producing the Entity
Navajo Document: Entity Service as a Schema
•...
Example Entity Message
<message name=”MatchReport”>
<property name=”MatchId” type=”integer”/>
<property name=”HomeTeamId” ...
Mongo Query Service
(revisited)
<navascript>
<message name=”__Mongo__”>
<property name=”EntityService” value=”entity/Match...
Result of Query (revisited)
<message name=”MatchReport”>
<property name=”_id” value=”aabf3fdde323g”/>
<property name=”Matc...
Flexibility Harnessed
• Entity adds missing metadata
• Entity can be used for type validation
• Adding properties is easy,...
Insert Example
<navascript>
<message name=”__Mongo__”>
<property name=”EntityService” value=”entity/MatchReport”/>
</messa...
Update Example<navascript>
<message name=”__Mongo__”>
<property name=”EntityService” value=”entity/MatchReport”/>
</messag...
Delete Example
Assume that we accidentally inserted the following document:
<message name=”MatchReport”>
<property name=”_...
What about more fine grained mutations?
Friday, August 23, 13
Simple Modify Example
Let’s just change the date
<navascript>
<message name=”__Mongo__”>
<property name=”EntityService” va...
Inserting new Array
Element
Insert a new player for the home team:
<message name=”MatchReport”>
<property name=”_id” value...
<message name=”MatchReport”>
<property name=”_id” value=”aabf3fdde323g”/>
<message name=”HomePlayers” type=”array”>
<messa...
Deleting an Array
Element
<message name=”MatchReport”>
<property name=”_id” value=”aabf3fdde323g”/>
<message name=”HomePla...
What have we reached?
• A Flexible Schema to manage MongoDB
document data
• Schema is managed outside Mongo
• Basically a ...
Normal Services
Entity Services
... Symmetry at last!
Friday, August 23, 13
What is missing?
• Notion of entity instances (to enable client
push)
• Relations between entities
• Definition of key/requ...
Entity First
• Thus far: Operation-Oriented Services:
DoUpdateMatchReport, QueryMatchReport,
FinalizeMatchReport, etc.
• R...
REST in 1 Slide
• A REST architecture relies fully on HTTP as an
Application Protocol
• REST revolves around URI resources...
RESTFul Navajo
• Navajo Entity == Resource
• Extend to support inheritance and referencing
other Navajo Entities
• Add key...
Entity Service<validations>
<check code=”A MatchReport should have a date unless it is a DRAFT
MatchReport”>
[/Match/Statu...
Entity Operations
<navascript>
<operations>
<operation service=”mongo/ProcessQueryMongo” method=”GET”/>
<operation service...
Pure HTTP GET
http://dexels.example.org/MatchReport?MatchId=33
Friday, August 23, 13
Backwards Compatibility
<navascript>
	

 <map.entity entity="'MatchReport'">
	

 	

 <entity.createproperty name="'MatchId...
Entity Insert
<navascript>
	

 <message name="MatchReport">
	

 	

 <property name="MatchId">
	

 	

 	

 <expression valu...
Caching and Consistency
• ETag can help us with
• Preventing that multiple GET operation
return same data (send 304 instea...
Polyglot Persistence
• Entity Operations can implement different
persistence methods
• Some Entities can “live” inside Mon...
Entity drives Service drives Entity drives Service drives Entity
drives Service drives Entity drives Service drives Entity...
Normal Services
Entity Services
... from process- to resource oriented
Friday, August 23, 13
Upcoming SlideShare
Loading in...5
×

NoSQL Now 2013 Presentation

563

Published on

Presentation on Schema Service for harnessing MongoDB schemaless feature. Implemented using Navajo Service Oriented Application Framework. Digital Match Report of the Sportlink ERP system as use case.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
563
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "NoSQL Now 2013 Presentation"

  1. 1. Service Entities Using MongoDB NoSQL Now 2013, San Jose Arjen Schoneveld Friday, August 23, 13
  2. 2. Friday, August 23, 13
  3. 3. Arjen Schoneveld • co-Founder of Dexels,Amsterdam based software company • Architect of Sportlink Sports ERP system • PhD in Computer Science • arjen@dexels.com, http://nl.linkedin.com/in/ aschoneveld/ Friday, August 23, 13
  4. 4. Our Story • How it started: Sportlink Sport Management System • Navajo: a Service Oriented Application Framework • Limitations of monolithic persistence • MongoDB use case: Sportlink Digital Match Report • Using Navajo Entity Services as a Schema • RESTfull Entities Friday, August 23, 13
  5. 5. Sportlink Architecture (2003) Sport Club Application Federation Application Federation Website Database (Oracle) Crystal Reports Comp. Planning Friday, August 23, 13
  6. 6. Data Model Object Model Service Facade Web Services JSP/HTML/ Javascript/CSS ORM JAX-WS SOAP/WSDL Service Proxy A Service Oriented Application Stack Object Model Friday, August 23, 13
  7. 7. How to simplify? Friday, August 23, 13
  8. 8. Services First • Changes are “the rule” in many environments • Most Services are CRUD and instances have very short live-span • Many services do not need “OOP models” • Use OOP for complex algorithms (e.g. match scheduling) only • Build, not generate, Services Friday, August 23, 13
  9. 9. Usable Services • Loose Coupling: no client stub generation! • Metadata is key ingredient: self describing • Type, description, input validation, subsequent services (protocol) • Self describing Abstract User Interface → enables automatic creation of usable first order UIs • Use a Canonical Service Language as the “lingua franca” between the services and the client Friday, August 23, 13
  10. 10. What we came up with Friday, August 23, 13
  11. 11. Data Model The Navajo Application Framework UIs [Platform Agnostic Declarative UI Language] Canonical Service Language Services [Service Script Language] Friday, August 23, 13
  12. 12. How we implemented it Friday, August 23, 13
  13. 13. fd [Resources] Database Mail Server FTP Server SOAP Service [Adapters] [Tipi] UIW idgets UIEvents Service Listeners CallServices UIActions Layout + CSS [Services] Client State N avajo Expressions [Logging] [Work Flows] Use Adapters [Authentication] Validation Service Links JDBC MongoDB [Entities] SOAP CallService Joinmessage Email Crystal [Service Queue] Friday, August 23, 13
  14. 14. How Sportlink Evolved... Friday, August 23, 13
  15. 15. Sportlink Ecosystem (2013) Club (Tipi/Swing) Association (Java/Swing) Website (Third Party) Oracle Crystal FTP ~4000 services Comp. Planning E-Mail Digital Match Report (iOS/Android/Tipi) Internet Payment (PHP) Facility Mngtmt (GWT+G. Earth) Course Registration (GWT+G. Maps) Official Portal (Tipi/VaadIn) MongoDB MySQL Website Plugins (PHP) Financial Admin ServiceWorkflow Enterprise DB Friday, August 23, 13
  16. 16. Growing Pains • More end user applications • More functionality • More sport associations with different requirements • More end users • Single database has become the bottleneck Friday, August 23, 13
  17. 17. Oracle Limitations • License Costs! • Unpredictable query behavior • Only partly/expensive High Availability • Poor affordable(!) horizontal scalability in combination with good single node performance • Service/Persistence impedance mismatch! Friday, August 23, 13
  18. 18. Oracle Mongo Neo4j MySQL Dynamo Cassandra PostgreS QL MatchReport Web Plugin Mobile App Club App Association App Association App Courses App Facility Mngmnt Polyglot Persistence Friday, August 23, 13
  19. 19. Our MongoDB Use Case • Digital Match Report (Mobile,Web and Desktop) • Player Roster for home- and away teams (role, captain, base player) • Match Result, Match Events, Player Validation, Digital Players Pass • 40.000 matches per weekend (fixed times: 11.00, 12:30, 14:00, 15:30, ...) • typical: 3 users per match (official, two captains) Friday, August 23, 13
  20. 20. Why MongoDB • MatchReport application is introduced gradually → Scalability → NoSQL • MatchReport can be fetched and updated like a Data Aggregate → Document Based NoSQL Friday, August 23, 13
  21. 21. Home Away Official Mongo Oracle Fetch Match Report MatchReportReport Fetch Match Report Report Populate Home Players Report Populate Away Players Report Fetch Match Report Report Report Report Fill In Report Report Finalize Report Report Match Players Events Players Events Friday, August 23, 13
  22. 22. How do we integrate Navajo and MongoDB? Friday, August 23, 13
  23. 23. Navajo Services • A simple XML/HTTP/POST-based Web Service • Uses a Canonical Service Language: Navajo Document • Consumes and produces a Navajo Document (Everything is a Document) • May contain additional (Service) links to next-step services (implementing an application protocol, like hyperlinks) • Navascript consumes and produces a Navajo Document, using expressions and access (Java) adapters for: database access, calling other Navajo Services, calling SOAP, manipulating messages (e.g. joins), etc. • JavaScript and JRuby are also supported Navajo Service implementation languages • Navajo Services are (hot) deployed in a Navajo Server/OSGi runtime using realtime generation of OSGi bundles Friday, August 23, 13
  24. 24. Navajo Document • Navajo Services use Navajo Documents for all message processing (Canonical Service Language) • All data required by and delivered from a Navajo Service is contained in a Navajo Document • A Navajo document consists of the following elements: • messages, for grouping: • properties and other messages • methods (links to relevant subsequent services) Friday, August 23, 13
  25. 25. Properties • A property has metadata: • name. Name must be unique within a message • type. Natively supported types are: string, integer, float, boolean, date, binary, selections(!). • value.The value of the property. • description. Used to indicate the semantical meaning of the property • subtype. May contain additional formatting hints for clients, like matching regular expressions for strings or the mime-type of a binary property • direction. Indication for a client to render a property read-only or not. Friday, August 23, 13
  26. 26. Serializations • The default serialization of Navajo Document is XML • Different other serialization are supported, e.g. JSON • Navajo Document contains a lot of metadata automatic User Interface Generation is supported (an Eclipse Plugin is provided) • Serialization of binary type properties is Base64. Navajo XML parser supports transparent streaming of binary properties Friday, August 23, 13
  27. 27. A Navajo Document<tml> <message name=”Person”> <property name=”Lastname” length=”100” description=”Lastname of the person” type=”string” subtype=”regexp=[A-z]*” direction=”out” value=”Kant”/> <property name=”FirstName” type=”string” subtype=”regexp=[A-z]*” length=”50” direction=”out” value=”Immanuel”/> <property name=”DateOfBirth” type=”date” direction=”out” value=”1724-04-22”/> <property name=”Photo” type=”binary” direction=”in”> </property> </message> </tml> Friday, August 23, 13
  28. 28. Collections:Array Messages <tml> <message name=”Persons” type=”array”> <message name=” Persons” type=”definition”> <property name=”Lastname” type=”string” subtype=”regexp=[A-z]*” length=”100” direction=”out” description=”Last name of this Person”/> <property name=”Firstname” type=”string” subtype=”regexp=[A-z]*” length=”50” direction=”out” description=”First name of this Person”/> </message> <message name=” Persons” index=”0”> <property name=”Lastname” value=”Kant”/> <property name=”Firstname” value=”Immanuel”/> </message> <message name=” Persons” index=”1”> <property name=”Lastname” value=”Godel”/> <property name=”Firstname” value=”Kurt”/> </message> <message name=” Persons” index=”2”> <property name=”Lastname” value=”Turing”/> <property name=”Firstname” value=”Alan”/> </message> </message> </tml> Friday, August 23, 13
  29. 29. Club Application [Tipi] QueryMatch Oracle 2. Request <tml rpc_name=”QueryMatch”> <message name=”Match”> <property name=”Id” value=”1”/> </message> </tml>4. Response <tml rpc_name=”QueryMatch”> <message name=”Match”> <property name=”Id”/> <property name=”Date”/> <property name=”HomeTeam”/> <property name=”AwayTeam”/> </message> <methods> <method name=”UpdateMatch”/> <method name=”DeleteMatch”/> </methods> </tml> 3. SELECT * FROM match WHERE matchid = ? <navascript> [see next slide] </navascript> <tipi> <button> <onPush> (1) <callservice name=”QueryMatch”> </onPush> </button> (5) <panel listener=”QueryMatch”> <label name=”Calendar date” value=”{navajo:/QueryMatch/ MatchDate}”/> </panel> </tipi> Friday, August 23, 13
  30. 30. <navascript> <validations> <check code=”Empty Id”> [/Match/Id] != ‘’ </check> </validations> <map.sqladapter> <sqladapter.query> SELECT * FROM MATCH WHERE matchid =? </sqladapter.query> <sqladapter.parameter value=”[/Match/Id]”/> <message name=”Match”> <property name=”FormattedMatchDate”> <expression value=”FormatDate($columnValue(‘Date’),’dd-MMM-yyy'”/> </property> </message> </map.sqladapter> <methods> <method name=”ProcessUpdateMatch”/> <method name=”ProcessDeleteMatch”/> </methods> </navascript> Friday, August 23, 13
  31. 31. Friday, August 23, 13
  32. 32. MongoDB • MongoDB is a document-based NoSQL database • MongoDB is schema-less, unlike SQL databases that rely on a predefined schema (or metadata) to define e.g. tables and table structures • MongoDB serializes data as JSON objects • MongoDB supports a sharded setup → scalability • Advantage: ease of use due to lack of structure • Disadvantage: unmaintainable due to lack of structure Friday, August 23, 13
  33. 33. Navajo and MongoDB • Store Navajo Document directly in MongoDB • Navajo Document can be easily transformed to JSON • No need for writing “mapping code” (no impedance mismatch) Friday, August 23, 13
  34. 34. JSONify Navajo Document <message name=”MatchReport”> <property name=”MatchId” value=”1” type=”integer”/> <property name=”HomeTeamId” value=”1234” type=”integer” description=”Unique id of the home team”/> <property name=”AwayTeamId” value=”4321” type=” integer” description=”Unique id of the away team”/> <property name=”Date” value=”2013-08-24” type=”date” description=”Match date”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers” type=”arrayelement”> <property name=”PersonId” type=”string” value=”CHGP12Y”/> <property name=”Role” type=”selection” cardinality=”1” type=”selection”> <option name=”Defender” value=”DEF” selected=”1”/> <option name=”Keeper” value=”KPR” selected=”0”/> </property> </message> </message> [... same for AwayPlayers ... left for brevity ] </message> The Navajo Document XML above can be converted to JSON as follows: { “MatchId” : 1,“HomeTeamId” :“1234”,“AwayTeamId” :“4321”, “Date” : Date ( 23242873 ), “HomePlayers” : [ {“PersonId” :“CHGP12Y”,“Role” :“DEF”}] } Friday, August 23, 13
  35. 35. Query MongoDB • Entity in MongoDB can be queried using MongoDB query syntax • Entity in MongoDB can be queried using a Navajo Document -> will be converted to a JSON query: <message name=”MatchReport”> <property name=”MatchId” value=”1”/> </message> can be converted to Mongo query (message name maps to Collection name): MatchReport.find({“MatchId” : 1}) Friday, August 23, 13
  36. 36. <navascript> <message name=”MatchReport”> <property name=”MatchId” value=”1” type=”integer”/> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessQueryMongo” append=”’/’”/> </map.navajomap> </navascript> Mongo Query Service Construct Query Call Query Service and append result Friday, August 23, 13
  37. 37. Result of Query <message name=”MatchReport”>Unique Id <property name=”_id” value=”aabf3fdde323g”/> <property name=”MatchId” value=”1” type=”integer”/> <property name=”HomeTeamId” value=”1234” type=”integer”/> <property name=”AwayTeamId” value=”4321” type=”integer”/> <property name=”Date” value=”2013-08-24” type=”date”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”__id” value=”abb344gf67”/> <property name=”PersonId” type=”string” value=”CHGP12Y”/> <property name=”Role” type=”string” value=”DEF”/> </message> <message name=”HomePlayers”> <property name=”__id” value=”ffhggab3232b”/> <property name=”PersonId” type=”string” value=”MJGP845”/> <property name=”Role” type=”string” value=”KEP”/> </message> </message> </message> What is Lost? What is Added? Unique Id Unique Id Friday, August 23, 13
  38. 38. How can we resolve this? Friday, August 23, 13
  39. 39. Query using Entity Service • Define a Navajo Service for producing the Entity Navajo Document: Entity Service as a Schema • Expects no input • Simply returns Navajo Document describing the Entity • Pass Entity Service as a “parameter” to the Navajo Service that queries or mutates Mongo • Entity service is used to decorate the “raw” Mongo document Friday, August 23, 13
  40. 40. Example Entity Message <message name=”MatchReport”> <property name=”MatchId” type=”integer”/> <property name=”HomeTeamId” type=”integer” description=”Unique id of the hometeam”/> <property name=”AwayTeamId” type=”integer” description=”Unique id of the person”/> <property name=”Date” type=”date” description=”Match date”/> <message name=”HomePlayers” type=”array”> <message name=” HomePlayers” type=”definition”> <property name=”PersonId” type=”string” value=”Unique person id”/> <property name=”Role” type=”selection” cardinality=”1” type=”selection”> <option name=”Defender” value=”DEF”/> <option name=”Keeper” value=”KPR”/> </property> </message> </message> </message> Navajo Document above describes the Entity type “MatchReport” Friday, August 23, 13
  41. 41. Mongo Query Service (revisited) <navascript> <message name=”__Mongo__”> <property name=”EntityService” value=”entity/MatchReport”/> </message> <message name=”MatchReport”> <property name=”MatchId” value=”1” type=”integer”/> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessQueryMongo” append=”’/’”/> </map.navajomap> </navascript> Friday, August 23, 13
  42. 42. Result of Query (revisited) <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <property name=”MatchId” value=”1” type=”integer”/> <property name=”HomeTeamId” value=”1234” type=”string” description=”Unique id of the home team”/> <property name=”AwayTeamId” value=”4321” type=”string” description=” Unique id of the away team”/> <property name=”Date” value=”2013-08-24” type=”date” description=”Match date”/> <message name=”HomePlayers” type=”array”> <message name=” HomePlayers” type=”definition”> <property name=”PersonId” type=”string” value=”Unique person id”/> <property name=”Role” type=”string” type=”selection”> <option name=”Defender” value=”DEF”/> <option name=”Keeper” value=”KPR”/> </property> </message> <message name=”HomePlayers”> <property name=”__id” value=”abb344gf67”/> <property name=”PersonId” type=”string” value=”CHGP12Y”/> <property name=”Role” type=”string” value=”DEF”/> </message> <message name=”HomePlayers”> <property name=”__id” value=”ffhggab3232b”/> <property name=”PersonId” type=”string” value=”MJGP845”/> <property name=”Role” type=”string” value=”KEP”/> </message> </message> </message> Friday, August 23, 13
  43. 43. Flexibility Harnessed • Entity adds missing metadata • Entity can be used for type validation • Adding properties is easy, simply add it to the Entity, next time it will appear in the query • Deleting properties is easy, simply delete it from the Entity, next time it won’t appear in the query. Friday, August 23, 13
  44. 44. Insert Example <navascript> <message name=”__Mongo__”> <property name=”EntityService” value=”entity/MatchReport”/> </message> <message name=”MatchReport”> <property name=”MatchId” value=”2” type=”integer”/> <property name=”HomeTeamId” value=”9787” type=”string” description=”Unique id of the home team”/> <property name=”AwayTeamId” value=”6543” type=”string” description=” Unique id of the away team”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”PersonId” type=”string” value=”BBFW63X”/> <property name=”Role” type=”string” value=”DEF”/> </message> </message> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessInsertMongo”/> </map.navajomap> </navascript> This will fail! Friday, August 23, 13
  45. 45. Update Example<navascript> <message name=”__Mongo__”> <property name=”EntityService” value=”entity/MatchReport”/> </message> <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <property name=”MatchId” value=”1” type=”integer”/> <property name=”HomeTeamId” value=”1234” type=”integer”/> <property name=”AwayTeamId” value=”4321” type=”integer”/> <property name=”BogusAttribute” value=”FooBar” type=”string”/> <property name=”Date” value=”2013-09-22” type=”date”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”__id” value=”abb344gf67”/> <property name=”PersonId” type=”string” value=”CHGP12Y”/> <property name=”Role” type=”string” value=”DEF”/> </message> <message name=”HomePlayers”> <property name=”__id” value=”ffhggab3232b”/> <property name=”PersonId” type=”string” value=”MJGP845”/> <property name=”Role” type=”string” value=”KEP”/> </message> </message> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessUpdateMongo”/> </map.navajomap> </navascript> This will be removed Friday, August 23, 13
  46. 46. Delete Example Assume that we accidentally inserted the following document: <message name=”MatchReport”> <property name=”_id” value=”aaa666”/> <property name=”MatchId” value=”3” type=”integer”/> <property name=”HomeTeamId” value=”809832” type=”integer” /> <property name=”AwayTeamId” value=”321311” type=”integer” /> <message name=”HomePlayers” type=”array”/> </message> This can be deleted using following script: <navascript> <message name=”MatchReport”> <property name=”_id” value=”aaa666”/> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessDeleteMongo”/> </map.navajomap> </navascript> Friday, August 23, 13
  47. 47. What about more fine grained mutations? Friday, August 23, 13
  48. 48. Simple Modify Example Let’s just change the date <navascript> <message name=”__Mongo__”> <property name=”EntityService” value=”entity/MatchReport”/> </message> <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <property name=”Date” value=”2013-09-22” type=”date”/> </message> <map.navajomap useCurrentOutDoc=”true”> <navajomap.callwebservice name=”mongo/ProcessModifyMongo”/> </map.navajomap> </navascript> Friday, August 23, 13
  49. 49. Inserting new Array Element Insert a new player for the home team: <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”Insert” type=”boolean” value=”true”/> <property name=”PersonId” type=”string” value=”BBFW06E”/> <property name=”Role” type=”string” value=”DEF” /> </message> </message> </message> Note that the properties _id and Insert (set to true) are required Id is not required, it will be assigned Friday, August 23, 13
  50. 50. <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”Modify” type=”boolean” value=”true”/> <property name=”__id” value=”ffhggab3232b”/> <property name=”Role” type=”string” value=”KEP”/> </message> </message> </message> Id is required Set Modify flag Modifying an Array Element Change the role of a player: Friday, August 23, 13
  51. 51. Deleting an Array Element <message name=”MatchReport”> <property name=”_id” value=”aabf3fdde323g”/> <message name=”HomePlayers” type=”array”> <message name=”HomePlayers”> <property name=”Delete” type=”boolean” value=”true”/> <property name=”__id” value=”agb3f4g4468”/> </message> </message> </message> Id is required Set Delete flag Friday, August 23, 13
  52. 52. What have we reached? • A Flexible Schema to manage MongoDB document data • Schema is managed outside Mongo • Basically a metadata template to filter/ validate both incoming and outgoing data • No need to write any “Mapping Code” Friday, August 23, 13
  53. 53. Normal Services Entity Services ... Symmetry at last! Friday, August 23, 13
  54. 54. What is missing? • Notion of entity instances (to enable client push) • Relations between entities • Definition of key/required properties • Data consistency • Entity defined persistence Friday, August 23, 13
  55. 55. Entity First • Thus far: Operation-Oriented Services: DoUpdateMatchReport, QueryMatchReport, FinalizeMatchReport, etc. • Resource-Oriented, the Way of the Web • RESTful resources can be viewed as First Class Citizens (everything is a resource) • REST method semantics enable use of Cache Friday, August 23, 13
  56. 56. REST in 1 Slide • A REST architecture relies fully on HTTP as an Application Protocol • REST revolves around URI resources (nouns) and HTTP methods (actions) and ... • Rely on hypermedia to drive the client(!) state machine defined by the application Friday, August 23, 13
  57. 57. RESTFul Navajo • Navajo Entity == Resource • Extend to support inheritance and referencing other Navajo Entities • Add key properties • Navajo Entity support one or more HTTP methods (GET, PUT, POST and DELETE) • Entity Operations (HTTP methods) are implemented by Navajo Services Friday, August 23, 13
  58. 58. Entity Service<validations> <check code=”A MatchReport should have a date unless it is a DRAFT MatchReport”> [/Match/Status] != ‘DRAFT’ OR CheckDate([/MatchReport/Date)</check> </validations> <message name=”MatchReport” extends="navajo://Report"> <property name="_id" type="string" key="true,auto"/> <property name="MatchId" type="string" key="true" reference=”navajo:// Match/MatchId”/> <property name="Status" type="string"/> <property name="HomeTeamId" type="integer" reference=”navajo://Team/ TeamId”/> <property name="AwayTeamId" type="integer" reference=”navajo://Team/ TeamId”/> <message name=”HomePlayers” type=”array”> <message name=” HomePlayers” type=”definition”> <property name=”PersonId” type=”string” reference=”navajo:// Person”> <property name=”Role” type=”string”/> </message> </message> </message> Friday, August 23, 13
  59. 59. Entity Operations <navascript> <operations> <operation service=”mongo/ProcessQueryMongo” method=”GET”/> <operation service=”mongo/ProcessModifyMongo” method=”POST”/> </operations> <message name=”MatchReport” extends="navajo://Report"> [ .... ] </message> </navascript> [NOTE: operation == method] Friday, August 23, 13
  60. 60. Pure HTTP GET http://dexels.example.org/MatchReport?MatchId=33 Friday, August 23, 13
  61. 61. Backwards Compatibility <navascript> <map.entity entity="'MatchReport'"> <entity.createproperty name="'MatchId'" value="3" /> <entity.get append="'/'" /> </map.entity> </navascript> Friday, August 23, 13
  62. 62. Entity Insert <navascript> <message name="MatchReport"> <property name="MatchId"> <expression value="‘31232’"/> </property> <property name="HomeTeamId"> <expression value="‘1231232’"/> </property> <property name="AwayTeamId"> <expression value="‘345324’"/> </property> </message> <map.entity useCurrentOutDoc="true" entity="'MatchReport'"> <entity.insert/> </map.entity> </navascript> Friday, August 23, 13
  63. 63. Caching and Consistency • ETag can help us with • Preventing that multiple GET operation return same data (send 304 instead) • Preventing that Stale data is used for updating an Entity (check ETag on PUT,DELETE,POST) (send 409) Friday, August 23, 13
  64. 64. Polyglot Persistence • Entity Operations can implement different persistence methods • Some Entities can “live” inside MongoDB while other reside in e.g. Oracle • Non-functional data requirements (e.g. consistency, availability) can drive persistence decision • “Transparent” migration of Data to other datasources Friday, August 23, 13
  65. 65. Entity drives Service drives Entity drives Service drives Entity drives Service drives Entity drives Service drives Entity drives Service • Role of Normal Service and Entity Service are reversed • Instead of QueryMatchForm, DoInsertMatchForm, DoDeleteMatchForm, we have MatchForm (GET, DELETE, INSERT) Friday, August 23, 13
  66. 66. Normal Services Entity Services ... from process- to resource oriented Friday, August 23, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×