Long story short, a true tale about an hopeless DevTeamCMF: a pain in the F David Funaro & Alessandro Nadalin             ...
let us introduceThe Problem™
There comes the boss            < theres a new project
There comes the boss              < seems easy,                lets use Joomla!
Dont open that door!
CMSThe 80/20 dilemma
80% is cool at all
but asthe requirements grows
(you always knew that)
the project becomes a mess
There comes the boss            < can I haz fotonic              menthal scanner              in the website
There comes the boss              < cant do boss             Joomla! is not meant...
There comes the boss           < STFU NOOB DO THAT             OR BLA BLA BLA
what if
There comes the boss               < seems complex,              no Joomla this time,             lets use a framework!
There comes the boss           < STFU NOOB WE             SHOULD GO LIVE             NEXT WEEK BLA BLA
FrameworkThe 20/80 dilemma
20% is cool at all
the 80%is really funny
but has a cost   Time To Market
again, what if
There comes the boss               < seems complex,              no Joomla this time,             when should it go live?
There comes the boss           < salesguy just told the             client tomorrow
There comes the boss             < ok, lets use a CMF
unfortunately, there is no stable  CMF in the PHP ecosystem
so it has a cost   initial development
investment                 =       know how (team) +      quality (framework) +velocity (that customers pretend)          ...
"Our" CMF
Target: End User/Site adminshould be able to manage a site editing pages with                different elements.          ...
Target: Developer    should be able to plug the CMF with NO PAIN,using the kick-ass framework at the bottom of the CMF.
2things
Pages
Pages main structure of a web site organized as a tree a way to express our contents usually composed by a series of conta...
Boxes
BoxesThe way to separatesingle elements of a pageLoosely coupled HMVCstructures
Boxes                 {Bridge}Entities                              Pages Boxes let entities be shown inside webpages
BoxesWe dispose of different types ofboxes:  Common  DSL
Common BoxSimple   Input field   Textarea   Video   Image   File   RichtextComposite   Input field + image   input field +...
what about the DSL?
Domain Specific LanguageDSL = ! (Website 1   Website 2 )
Example: Identify DSL    Event    Page        Event           PageBooks                    Cars            Users          ...
Web Elements = CMF Entities            Page        CarsBooks      NewsReview     Widget                      Contract     ...
back to the CMF
CMF StackFramework: symfony, for exampleCMF entities: Site, Pages, Menus, Users ...DSL entities: cars, contracts (for Hert...
Common Boxes + DSL Boxes                 =The main tool for the developer to build a page                with the CMF
Example : page & box composition
The Page-Box Binding StorageData can change its structure fastly, according to the domain model.        We need a technolo...
so,  How will wemanage the data?
RDBMS   , of course
because differentdata always have the same need
one size fits all
We tend to think at the way we should store data          not how were gonna use it
and we think that theshit-it-always-happens migration          will be easy
also if that column is useful for one and one only row
among milions
so we know were gonna facea few well-known problems...
Data Mapping Data Structure                              RDBMS                                    id               10     ...
Data MappingData Structure                         RDBMS      is better than before, but isnt the best way
Data MappingData Structure              RDBMS                            ?         ... and NOW??????
The Alienspend more time to normalizeyour models          BUT    you always find  something that as a  different Behaviour
Unpredictable    TomorrowUser Story evolve continous   Worse than the Alien
EAV                 formerly known as:              normalization WTF?!?!?!                          or:how to do 70k join...
Overnormalizeduser_id | mail | phone | address                    Large schema is broken                       into smalle...
Overnormalizeduser_id | mail_id LEFT JOINuser_id | phone_id LEFT JOIN...............user_id | address_id   LEFT JOIN
Overnormalized              Users ListLEFT JOIN                  LEFT JOIN                  LEFT JOIN                     ...
OvernormalizedLEFT JOIN                 LEFT JOIN                     LEFT JOIN         LEFT JOINList             Users   ...
back to the CMF
Bind Page with BoxesForeing Key -> TextareaBox
Bind Page with BoxesForeing Key -> ?       AND NOW        AND NOW ???
#1   Mantain the RI
#2   Not Mantain the RI
Maintain or not Maintain   the RI; that is the       question
we have to choose
have you ever thought"there should be another way!"            ?
Can someone help us?
lets tryNoSQL world
Written in: C/C++                                Main point: Blazing fast                                Protocol: Telnet-...
Best used:                                                     rapidly changing data                                      ...
very fast
very fastObject != key -> value
very fast              Object != key -> value              Not reliableCan I haz otha nozql?
Written in: C++                                                Main point: Retains some friendly properties               ...
Best used:                                               you need dynamic queries                                         ...
Document Oriented
Document OrientedSchema Free
Document Oriented            Schema Free            Navigate ObjectCan I haz otha nozql?
Written in: Erlang                                         Main point: DB consistency, ease of use                        ...
Best used:                                              accumulating occasionally-                                        ...
Reliable
ReliableDocument Oriented
ReliableDocument OrientedSchema Free
Reliable           Document Oriented           Schema Free           Navigate ObjectCan I haz otha nozql?
Why do we so hardly need a GraphDB  like OrientDB?
"GraphDBs dont avoid relations but they embrace them ina way that they are not a computational problemanymore, by making t...
Easy joinsselect from users where city.country.continent = Oceania
Same link semantics,  different objects   select links from menu >> 28:1, 17:5
Same link semantics,    different objects                 select from [28:1, 17:5]          28:1                          ...
28:1                  17:5        ExternalLink         ProductLinkurl: http://www.google.com      id: 4          $link->re...
Written in: Java                                    Main point: Graph                                    Protocol: Binary ...
Reliable
ReliableGraph - Documental
ReliableGraph - DocumentalSchema-*
ReliableGraph - DocumentalSchema-*Easy traversal
ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendly
ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendlyUberFast ( binary protocol )
ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendlyUberFast ( binary protocol )Universal ( HTTP protocol )
is so cool
but has a problem...
ver 1.0
rc1
but has a couple problems...
JAVA
but somebody
started writing the binary-protocol binding     https://github.com/AntonTerekhov/OrientDB-PHP                    ( beta, 2...
and others
wrote the HTTP onehttps://github.com/odino/Orient
$driver = new OrientHttpClientCurl();$orient = new OrientFoundationBinding(  $driver, 127.0.0.1, 2480, admin, admin, demo)...
you can start playing with a pure         GraphDB now
High-level PHP Component?
No
but we know that ;)
thats why in DNSEE we will build a basicObject Graph Mapper              for OrientDB
...because...
hey, you remember about our CMF?
codename: ConGoW
codename: ConGoWcontents gone wild   ( dont ask why )
ConGoW soon the real kickstart
ConGoW soon the real kickstart 10 devs at the starting point
ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub
ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack   Symfony2
ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack   Symfony2   Orient
ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack   Symfony2   Orient   Doctrine2
"My only friend, the end"
"My only friend, the end"David Funaro@ingdavidinodavidfunaro.com
"My only friend, the end"David Funaro      Alessandro Nadalin@ingdavidino                   @_odino_davidfunaro.com       ...
"My only friend, the end"David Funaro      Alessandro Nadalin@ingdavidino                   @_odino_davidfunaro.com       ...
"My only friend, the end"David Funaro                     Alessandro Nadalin@ingdavidino                                  ...
Creditshttp://www.flickr.com/photos/pagedooley/5313217918/sizes/l/in/photostream/  http://www.flickr.com/photos/zoetnet/55...
CMF: a pain in the F @ PHPDay 05-14-2011
CMF: a pain in the F @ PHPDay 05-14-2011
Upcoming SlideShare
Loading in …5
×

CMF: a pain in the F @ PHPDay 05-14-2011

6,340 views
6,274 views

Published on

A presentation about the problems you'll face when dealing with the relational model and highly customizable general-purpose projects, with a look at the NoSQL word focusing on a real solution, a graph database.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,340
On SlideShare
0
From Embeds
0
Number of Embeds
2,713
Actions
Shares
0
Downloads
26
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

CMF: a pain in the F @ PHPDay 05-14-2011

  1. 1. Long story short, a true tale about an hopeless DevTeamCMF: a pain in the F David Funaro & Alessandro Nadalin May, 14 2011
  2. 2. let us introduceThe Problem™
  3. 3. There comes the boss < theres a new project
  4. 4. There comes the boss < seems easy, lets use Joomla!
  5. 5. Dont open that door!
  6. 6. CMSThe 80/20 dilemma
  7. 7. 80% is cool at all
  8. 8. but asthe requirements grows
  9. 9. (you always knew that)
  10. 10. the project becomes a mess
  11. 11. There comes the boss < can I haz fotonic menthal scanner in the website
  12. 12. There comes the boss < cant do boss Joomla! is not meant...
  13. 13. There comes the boss < STFU NOOB DO THAT OR BLA BLA BLA
  14. 14. what if
  15. 15. There comes the boss < seems complex, no Joomla this time, lets use a framework!
  16. 16. There comes the boss < STFU NOOB WE SHOULD GO LIVE NEXT WEEK BLA BLA
  17. 17. FrameworkThe 20/80 dilemma
  18. 18. 20% is cool at all
  19. 19. the 80%is really funny
  20. 20. but has a cost Time To Market
  21. 21. again, what if
  22. 22. There comes the boss < seems complex, no Joomla this time, when should it go live?
  23. 23. There comes the boss < salesguy just told the client tomorrow
  24. 24. There comes the boss < ok, lets use a CMF
  25. 25. unfortunately, there is no stable CMF in the PHP ecosystem
  26. 26. so it has a cost initial development
  27. 27. investment = know how (team) + quality (framework) +velocity (that customers pretend) = CMF
  28. 28. "Our" CMF
  29. 29. Target: End User/Site adminshould be able to manage a site editing pages with different elements. Easily, like a CMS.
  30. 30. Target: Developer should be able to plug the CMF with NO PAIN,using the kick-ass framework at the bottom of the CMF.
  31. 31. 2things
  32. 32. Pages
  33. 33. Pages main structure of a web site organized as a tree a way to express our contents usually composed by a series of containers ( that well refer to as boxes )
  34. 34. Boxes
  35. 35. BoxesThe way to separatesingle elements of a pageLoosely coupled HMVCstructures
  36. 36. Boxes {Bridge}Entities Pages Boxes let entities be shown inside webpages
  37. 37. BoxesWe dispose of different types ofboxes: Common DSL
  38. 38. Common BoxSimple Input field Textarea Video Image File RichtextComposite Input field + image input field + image + richetex textarea + video
  39. 39. what about the DSL?
  40. 40. Domain Specific LanguageDSL = ! (Website 1 Website 2 )
  41. 41. Example: Identify DSL Event Page Event PageBooks Cars Users Review News Users News Contract Widget Widget
  42. 42. Web Elements = CMF Entities Page CarsBooks NewsReview Widget Contract DSL
  43. 43. back to the CMF
  44. 44. CMF StackFramework: symfony, for exampleCMF entities: Site, Pages, Menus, Users ...DSL entities: cars, contracts (for Hertz)Common Boxes : Textarea, image, video, ...Dsl boxes: Cars image gallery
  45. 45. Common Boxes + DSL Boxes =The main tool for the developer to build a page with the CMF
  46. 46. Example : page & box composition
  47. 47. The Page-Box Binding StorageData can change its structure fastly, according to the domain model. We need a technology supporting this fast change. Since we have N box type, to compose a page we should look for a few tables. We need a technology supporting this kind of lookups.
  48. 48. so, How will wemanage the data?
  49. 49. RDBMS , of course
  50. 50. because differentdata always have the same need
  51. 51. one size fits all
  52. 52. We tend to think at the way we should store data not how were gonna use it
  53. 53. and we think that theshit-it-always-happens migration will be easy
  54. 54. also if that column is useful for one and one only row
  55. 55. among milions
  56. 56. so we know were gonna facea few well-known problems...
  57. 57. Data Mapping Data Structure RDBMS id 10 name tom parent_id 19we have to find a way to re-organize the data-structre to insert ourtree inside a table.But is really different! Tree find a node in O(nlogn)
  58. 58. Data MappingData Structure RDBMS is better than before, but isnt the best way
  59. 59. Data MappingData Structure RDBMS ? ... and NOW??????
  60. 60. The Alienspend more time to normalizeyour models BUT you always find something that as a different Behaviour
  61. 61. Unpredictable TomorrowUser Story evolve continous Worse than the Alien
  62. 62. EAV formerly known as: normalization WTF?!?!?! or:how to do 70k joins to retrieve an entity of you model
  63. 63. Overnormalizeduser_id | mail | phone | address Large schema is broken into smaller ones user_id | mail_id user_id | phone_id user_id | address_id
  64. 64. Overnormalizeduser_id | mail_id LEFT JOINuser_id | phone_id LEFT JOIN...............user_id | address_id LEFT JOIN
  65. 65. Overnormalized Users ListLEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOINLEFT JOIN LEFT JOIN
  66. 66. OvernormalizedLEFT JOIN LEFT JOIN LEFT JOIN LEFT JOINList Users LEFT JOINLEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN LEFT JOIN What if you need to show only a single primary address?
  67. 67. back to the CMF
  68. 68. Bind Page with BoxesForeing Key -> TextareaBox
  69. 69. Bind Page with BoxesForeing Key -> ? AND NOW AND NOW ???
  70. 70. #1 Mantain the RI
  71. 71. #2 Not Mantain the RI
  72. 72. Maintain or not Maintain the RI; that is the question
  73. 73. we have to choose
  74. 74. have you ever thought"there should be another way!" ?
  75. 75. Can someone help us?
  76. 76. lets tryNoSQL world
  77. 77. Written in: C/C++ Main point: Blazing fast Protocol: Telnet-like Disk-backed in-memory database, but since 2.0, it can swap to disk. Master-slave replication Simple keys and values, but complex operations like ZREVRANGEBYSCORE INCR Values can be set to expire (as in a cache)$ redis-cli set mykey "my binary safe value" => OK$ redis-cli get mykey => my binary safe value
  78. 78. Best used: rapidly changing data with a foreseeable database size (should fit mostly in memory) For example: analytics real-time data collection caching MQ loghttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  79. 79. very fast
  80. 80. very fastObject != key -> value
  81. 81. very fast Object != key -> value Not reliableCan I haz otha nozql?
  82. 82. Written in: C++ Main point: Retains some friendly properties of SQL Protocol: Custom, binary (BSON) Master/slave replication speaks JavaScript Built-in sharding Performance over features After crash, it needs to repair tables> j = { name : david, surname : funaro };{ "name" : "david", "surname" : "funaro" }> db.things.save(t);> db.things.find();{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }
  83. 83. Best used: you need dynamic queries you prefer to define indexes, not map/reduce functions you need good performance on a big DB For example: all things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you backhttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  84. 84. Document Oriented
  85. 85. Document OrientedSchema Free
  86. 86. Document Oriented Schema Free Navigate ObjectCan I haz otha nozql?
  87. 87. Written in: Erlang Main point: DB consistency, ease of use Protocol: HTTP, some say REST Bi-directional replication => master-master conflict detection ( 409 Conflict ) MVCC: write operations do not block reads Previous versions of documents are available Crash-only (reliable) design Needs compacting from time to time Views: embedded map/reduce jQuery library includedcurl -d {"keys":["bar"]} -X POST http://host:port/foo/_all_docs?include_docs=true
  88. 88. Best used: accumulating occasionally- changing data pre-defined queries versioning is king For example: CRM/CMS systems APIhttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
  89. 89. Reliable
  90. 90. ReliableDocument Oriented
  91. 91. ReliableDocument OrientedSchema Free
  92. 92. Reliable Document Oriented Schema Free Navigate ObjectCan I haz otha nozql?
  93. 93. Why do we so hardly need a GraphDB like OrientDB?
  94. 94. "GraphDBs dont avoid relations but they embrace them ina way that they are not a computational problemanymore, by making them explicit instead of implicitthrough joins." Claudio Martella ( http://blog.acaro.org/entry/somebody-is-going-to-hate-me-nosparql )
  95. 95. Easy joinsselect from users where city.country.continent = Oceania
  96. 96. Same link semantics, different objects select links from menu >> 28:1, 17:5
  97. 97. Same link semantics, different objects select from [28:1, 17:5] 28:1 17:5 ExternalLink ProductLinkurl: http://www.google.com id: 4
  98. 98. 28:1 17:5 ExternalLink ProductLinkurl: http://www.google.com id: 4 $link->render()
  99. 99. Written in: Java Main point: Graph Protocol: Binary or HTTP ACID Disk-persisted or in-memory, Cluster 150k inserts per second Schema-less, *-full, *-mixed Speaks JSON Integrated GUI Documental SQL commandscurl -X PUT 127.0.0.1:2480/document/demo/1:2 --data "{@rid: 1:2}"
  100. 100. Reliable
  101. 101. ReliableGraph - Documental
  102. 102. ReliableGraph - DocumentalSchema-*
  103. 103. ReliableGraph - DocumentalSchema-*Easy traversal
  104. 104. ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendly
  105. 105. ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendlyUberFast ( binary protocol )
  106. 106. ReliableGraph - DocumentalSchema-*Easy traversalSQL-friendlyUberFast ( binary protocol )Universal ( HTTP protocol )
  107. 107. is so cool
  108. 108. but has a problem...
  109. 109. ver 1.0
  110. 110. rc1
  111. 111. but has a couple problems...
  112. 112. JAVA
  113. 113. but somebody
  114. 114. started writing the binary-protocol binding https://github.com/AntonTerekhov/OrientDB-PHP ( beta, 28 April 2010 )
  115. 115. and others
  116. 116. wrote the HTTP onehttps://github.com/odino/Orient
  117. 117. $driver = new OrientHttpClientCurl();$orient = new OrientFoundationBinding( $driver, 127.0.0.1, 2480, admin, admin, demo);$response = $orient->query("select from fellas where any() traverse(0,-1) ( @rid = 1:1 )");$output = json_decode($response->getBody());foreach ($output->result as $friend){ echo $friend->name; // other fun...}
  118. 118. you can start playing with a pure GraphDB now
  119. 119. High-level PHP Component?
  120. 120. No
  121. 121. but we know that ;)
  122. 122. thats why in DNSEE we will build a basicObject Graph Mapper for OrientDB
  123. 123. ...because...
  124. 124. hey, you remember about our CMF?
  125. 125. codename: ConGoW
  126. 126. codename: ConGoWcontents gone wild ( dont ask why )
  127. 127. ConGoW soon the real kickstart
  128. 128. ConGoW soon the real kickstart 10 devs at the starting point
  129. 129. ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub
  130. 130. ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack Symfony2
  131. 131. ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack Symfony2 Orient
  132. 132. ConGoW soon the real kickstart 10 devs at the starting point OSS on GitHub Stack Symfony2 Orient Doctrine2
  133. 133. "My only friend, the end"
  134. 134. "My only friend, the end"David Funaro@ingdavidinodavidfunaro.com
  135. 135. "My only friend, the end"David Funaro Alessandro Nadalin@ingdavidino @_odino_davidfunaro.com odino.org
  136. 136. "My only friend, the end"David Funaro Alessandro Nadalin@ingdavidino @_odino_davidfunaro.com odino.org
  137. 137. "My only friend, the end"David Funaro Alessandro Nadalin@ingdavidino @_odino_http://davidfunaro.com http://odino.org http://joind.in/talk/view/3008
  138. 138. Creditshttp://www.flickr.com/photos/pagedooley/5313217918/sizes/l/in/photostream/ http://www.flickr.com/photos/zoetnet/5520594473/sizes/l/in/photostream/ http://www.flickr.com/photos/mhxbhd/3962410821/sizes/o/in/photostream/http://www.flickr.com/photos/labyrinthx/1955594336/sizes/z/in/photostream/ http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis http://www.flickr.com/photos/salim/19426192/sizes/o/in/photostream/http://www.flickr.com/photos/mybluevan/3780363712/sizes/l/in/photostream/ http://farm4.static.flickr.com/3064/3086258014_f1925639e0.jpg

×