Devoxx 2010 | Tools In Action : Kauri and Lily

&Scalable and RestFul Web AppsAt the xroads of Kauri and Lily IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 1

Speaker Context Marc Portier, ±14 years of Java, XML based web development. Co-founder of Outerthought.• Outerthought? – Innovative Content Management system that follows ReST principles: “DaisyCMS” – New ReST supporting development framework: “Kauri project” – Scalable Search and Store: “Lily Project” IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Looking for You!• User, Contributor of our Open Source Projects• Teacher, University, School• Student• Talented worker – Job openings! IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

The Wonderful Web MachineIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Amazing WebUsable Scalable Heterogenous FRANTIC COMPLEX Functional LayeredSupercalifragilisticexpialidocious !Entertaining UNMANAGEABLE Elegant Just Works CHAOTIC Simple Decentralized Extensible IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

How does it keep up?• Diversity / Compatibility• Amount of Servers• Amount of Users• Amount of resources• Response Times / Latency Wouldnt you want to do that too? IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

In woodworking its important to work with the grain of the wood.The Web, too, has a grain, and a RESTful web service is one that works with it. － Leonard Richardson and Sam Ruby RestFull Web Services IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Meeting high expectations• Current web-experiences are spoiling us – Google Maps – Mash-ups – Atom – Amazon S3• It all looks so simple (and mostly works)• General customer feeling: “How hard could it be?”” IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Riding theWeb Project Wave IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

kauriproject.org• Resource Oriented WebDevelopment PlatForm• Spans WebServices (headless)• and WebApplications (human consumer)• Leverages modern and popular techniques and systems (+Jax-RS )• Currently 0.4-RC2 – ( 0.4 upcoming ) IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

HeritageKauri• Maven ➢ Build & Deploy• jQuery ➢ Front-end Lib• Spring Framework ➢ Dependency Injection• Restlet ➢ REST APIOuterthought• Apache Cocoon ➢ XML, URI Maps• XReporter ➢ Statemanagement• DaisyCms ➢ Runtime, RIA, ... IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Core Concepts• modules, – java/rest services, – service containers, – service wiring,• prototyping,• RIA,• techniques• routing, pages,• templates IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Reuse ParadoxIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Perfection is achieved, not when there is nothing more to add, but when there is nothing more to remove.－ Antoine de Saint-Exupery IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Plugable Systems• Interfaces• Implementations• Wiring IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Challenges• Jar Dependency Management – Maven POM >> Deployment Descriptoor – Runtime Shielding• Java interface aspect (obvious)• REST interface aspect – RestService Notion (instantly web-distributed) – URI-Templating: /service/{rsrc_id}/aspect/{subresource}.json – Wiring + internal protocol: service:/ IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Component Model: Modules • Unit of reuse • Maven managed JAR • Service Interfaces – Java aspect – REST aspect • Maps to One Spring Container • Has own classloading scope IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Runtime SystemIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Attitude• No Servlet container – (Although Kauri Apps can be deployed to...) – Servlets miss out on REST and IO threading• Own Application Container – Spring Based • Enhanced with own injection directives • Wrapped in managed classloading hierarchy • Allows wiring between modules – Maven Repository Based – Required support for the REST aspect – Advanced configuration IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Module vs. war• war != jar – Different layout – =deployment unit != reuse/component unit – Too high level for dependency shielding• Deployment Strategy: – conf/kauri/wiring.xml: deployment descriptor – Reuse maven repository layout – Different targets: • Standalone, rpm, war,... IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Routing and Representations• MVC-Split (Inspiration: Spring webmvc)• Routing – Groovy syntax support (builder @ init time) – Request time: Java performance – JaxRS support• Separate Representation Builder – Similar groovy syntax (again builder) – Produces Representation – Handle (presentation of) error conditions centrally IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Routing Samples• Groovy routerbuilder.router { directory(uri: "/resources", root: "module:/resources/") resource(uri: "/persons/{id}.html", ofClass: "com.foobar.Person")}• JaxRS@Path("/persons/{id}.html")public class PersonResource { @GET @Produces("text/html") public KauriRepresentation getPerson( @PathParam("id") String idParam) { .... }} IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Pages & Prototyping• Combination of – Instant reload: module-source from FS vs. jar – Intuitive FS based router • pages/person/{id}.html.xml – Advanced templates – Simple mock-dataservice • Entity modelling through sample json files • Loaded in the templates via service:/ • Easily re-wired to JPA based implementation. IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Web Development Process IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri PrototypingIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Continuous Prototyping• -p prototype mode – Possibility to blend and match production stuff with new try-out stuff• Pages router – Invite in design-aware HTML/CSS/JS client hacker with feeling for programming & FS based toolchain• -s module-source.properties – fast roundtrip mode directly mounted in the source IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Templating• XML (tag) Based – Conditionals, repeat, ... – Security – protect,• ${placeholder} replacement – through el or groovy – + Specific functions (i18n, config, publicUri, ...)• + Add your own• Noteworthy – Page inheritance – Local service loading IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Template Inheritance<?xml version="1.0"?><html xmlns:t="http://kauriproject.org/template"> < base “layout.xml” <head> <title> <t:block name="title">The title</t:block> </title> </head> <body> <t:block name="main">Main content</t:block> </body></html> usage > <?xml version="1.0" encoding="UTF-8"?> <html xmlns:t="http://kauriproject.org/template" “mypage.xml” t:inherit="module:/templates/layout/layout.xml"> <t:block name="title">My page</t:block> <t:block name="main">My interesting content</t:block> </html> IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Template Data-LoadingTemplate : “persons/{id}.html.xml” <?xml version="1.0"?> <html xmlns:t="http://kauriproject.org/template"> <t:variable name="person" src="service:/data/persons/${request.attributes.id}" overwrite="false" accept="application/json"/> <body> <h1>Person: ${person.name}</h1> <p>City: ${person.city}</p> </body> </html> Data : “mock/persons/mpo.json” { “name”: “Marc Portier”, URI : “/persons/mpo.html” “city”: “Bredene”, ... } IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Forms• Browser-based Data Value Interaction System• RIA approach• jQuery Based – UI effects & Controls• Adds – Standard rest-full ajax logic – Validation (local and remote) – Formatting IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Forms SampleJson Form-config: var fconf = { createURI: "${publicUri(service:/data/contact/)}", type: { members: { name: "string", email: { base: "string", "+validators": { isEmail: {} }, label: "e-mail" }, birthday: { base: "date", yearRange: "-100:+0", label: "What is your date of birth?" }, } }};HTML: <form id=”myform” /> IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Form Controls• Simple • Specific – Basic data-types – GoogleMaps-Location – Classic validations – Ajax based file-upload – HTML basics – ...• Aggregates – Collection – Composite – Choice• + Add your own IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

More Kauri• I18n support• XML & HTML loading (and cleaning)• Unified Configuration• Unified Security – Spring Security (ACEGI) – Support for various mehanisms • Basic, digest, https, form, ... • Belgian e-ID• Database resources (JPA)• ... IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Future Kauri• State management through temporary resources• More Advanced controls• Functional Modules IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Kauri Goals How What• All team members • ROA & RIA• All project stages• Efficiency • Modules – Prototype as a design mode • Templates – Enhance, not throw away • Data Interaction IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Observing the Obvious • REST-full stateless approach in web- access layer pretends “Ready for scale out” • Shifting the problem towards “Storage” • ie. Indexed StorageIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Scalable Store and Search IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

lilyproject.org• Scalable Store AND Search – Massively distributed searchable data store• Blended HBASE with SOLR (shard), adding – Schema with support for • versioning, varianting and Blob handling – 2ndary indices – Rowlog/Queue for sync & async updates – M/R batch re-indexing• 0.2 since October 2010, 1.0 expected March 2011 IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Scale on all endsIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Lily Core Concepts● Storage ● Hbase ● repository model ● versioning, varianting, mixins● Indexing ● Mapping● Search ● SOLR IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Repository ModelIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

MixinsIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Sample Schema (json)namespaces: { { /* Declaration of namespace prefixes. */ name: "b$name", "org.lilyproject.bookssample": "b", valueType: { primitive: "STRING" }, "org.lilyproject.vtag": "vtag" scope: "versioned"}, },fieldTypes: [ { { name: "b$bio", name: "b$title", valueType: { primitive: "STRING" }, valueType: { primitive: "STRING" }, scope: "versioned" scope: "versioned" }, }, { { name: "vtag$last", name: "b$pages", valueType: { primitive: "LONG" }, valueType: { primitive: "INTEGER" }, scope: "non_versioned" scope: "versioned" } }, ], { recordTypes: [ name: "b$language", { valueType: { primitive: "STRING" }, name: "b$Book", scope: "versioned" fields: [ }, {name: "b$title", mandatory: true }, { {name: "b$pages", mandatory: false }, name: "b$authors", {name: "b$language", mandatory: false }, valueType: { primitive: "LINK", multiValue: true }, {name: "b$authors", mandatory: false }, scope: "versioned" {name: "vtag$last", mandatory: false } }, ] }, ... IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Flexible content model• generic enough to accomodate many popular content schemas – HTML5, CMIS, RDF, NewsML, Dublin Core, … – academically verified (UGent MMLab / IBBT)• developer convenience – higher level constructs – schema reuse – versioning, linking, ... IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Lily Versioning `IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Indexer 2. multi-version 3. incremental 1. denormalize6. sharded 4. batch 5. extraction IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Lily and HBase• adds high-level content model – data types – versioning – blob storage on HDFS• focus on sparse (efficient) storage• RowLog for synchronous cross-table updates and async message queues IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Lily and SOLR• provides flexible mapping between HBase content model and SOLR index fields• interactive and batch (M/R) index maintenance• sharding• use(s) SOLR as-is: loose, flexible, extensible coupling• search access via SOLR (HTTP) API IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Lily RoadmapIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Time to take a ReSTQuestions?IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

http://feeds.feedburner.com	56
http://outerthought.org	46
http://docs.outerthought.org	4
http://localhost	2
http://www.outerthought.org	1
http://webcache.googleusercontent.com	1

Devoxx 2010 | Tools In Action : Kauri and Lily

by NGDATA on Nov 25, 2010

Accessibility

Categories

Upload Details

Usage Rights

6 Embeds 110

Statistics

Devoxx 2010 | Tools In Action : Kauri and Lily — Presentation Transcript