Your SlideShare is downloading. ×

Gaelyk - Guillaume Laforge - GR8Conf Europe 2011

3,390

Published on

Published in: Technology
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,390
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
45
Comments
1
Likes
7
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Gaelyk Skyrocketing Groovy into the cloud! Guillaume Laforge Groovy Project Manager SpringSource / VMwarejeudi 19 mai 2011
  • 2. Gaelyk Skyrocketing Groovy into the cloud! Guillaume Laforge Groovy Project Manager SpringSource / VMwarejeudi 19 mai 2011
  • 3. Guillaume Laforge • Groovy Project Manager • JSR-241 Spec Lead • Head of Groovy Development at SpringSource / VMWare • Initiator of the Grails framework • Creator of the Gaelyk toolkit • Co-author of Groovy in Action • Speaker: JavaOne, QCon, JavaZone, Sun TechDays, Devoxx, The Spring Experience, SpringOne, JAX, Dynamic Language World, IJTC, and more... 2jeudi 19 mai 2011
  • 4. Guillaume Laforge • Groovy Project Manager • JSR-241 Spec Lead • Head of Groovy Development at SpringSource / VMWare • Initiator of the Grails framework • Creator of the Gaelyk toolkit • Co-author of Groovy in Action • Speaker: JavaOne, QCon, JavaZone, Sun TechDays, Devoxx, The Spring Experience, SpringOne, JAX, Dynamic Language World, IJTC, and more... 2jeudi 19 mai 2011
  • 5. IaaS, PaaS, SaaS • Software as a Service – Gmail, SalesForce.com SaaS • Platform as a Service – Google App Engine PaaS • Infrastructure as a Service – Amazon EC2 IaaS 3jeudi 19 mai 2011
  • 6. IaaS, PaaS, SaaS • Software as a Service – Gmail, SalesForce.com SaaS • Platform as a Service – Google App Engine PaaS • Infrastructure as a Service – Amazon EC2 IaaS 3jeudi 19 mai 2011
  • 7. IaaS, PaaS, SaaS • Software as a Service – Gmail, SalesForce.com SaaS • Platform as a Service – Google App Engine PaaS • Infrastructure as a Service – Amazon EC2 IaaS 3jeudi 19 mai 2011
  • 8. Google App Engine • Google’s PaaS solution – Run your app on Google’s infrastructure • Initially just Python supported • Java support added too – Sandboxed JVM – Jetty servlet container • Several JVM-compatible language supported 4jeudi 19 mai 2011
  • 9. Key aspects • You can use most of your usual web frameworks for developping apps on App Engine Java – A WAR file, basically! (an exploded war actually) – Uploading to the cloud by sending deltas of changes • No OS image, or software to install – Unlike with Amazon EC2 • All the scaling aspects are handled for you – Database / session replication, load balancing... • There are quotas, but you need a high traffic application to start being charged – Free to get started 5jeudi 19 mai 2011
  • 10. Some of the available services • Memcache • XMPP – JCache implementation – Send / receive Jabber – Save on CPU and datastore messages (GTalk) • URL Fetch • User – Use Google’s user/ – Access remote resources authentication system – HttpUrlConnection – OAuth support • Mail • Cron & Task queues – Support both incoming and – Schedule tasks at regular outgoing emails intervals • Images – Queue units of work – Resize, crop, rotate... • Blobstore – For storing large content 6jeudi 19 mai 2011
  • 11. Limitations •30 seconds request duration limit – Task queues raised to 10 minutes – Notion of Backends servers since 1.5.0 • Forbidden to – write on the file system – create threads – use raw sockets – issue system calls – use IO / Swing / etc. directly •There’s a whitelist of classes allowed • Number of files and their size are limited 7jeudi 19 mai 2011
  • 12. Quotas & Billing 8jeudi 19 mai 2011
  • 13. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 14. Free tier Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 15. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 16. No infinite scaling! Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 17. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 18. No more usage based Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 19. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 20. Paid tier w/ free quota Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 21. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 22. Europe 2011 — Copenhagen, Denmark Support!jeudi 19 mai 2011
  • 23. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 24. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 25. Reduced quotas Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 26. Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 27. The Datastore Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 28. The datastore... • Distributed key / value store – Based on Google’s «BigTable» – Schema-less approach • Supporting – Transactions and partitioning – Hierarchies through entity groups • Data access APIs – JPA and JDO • but adds a big request load time factor – Direct low-level APIs 12jeudi 19 mai 2011
  • 29. ...and its «limitations» • You’re not using SQL –No joins – No database constraints – No aggregation functions (count, avg...) • In a query, you can only filter on one column for inequality • Transactions only available in entity groups • You can only update an entity once in a transaction • No full text search capability 13jeudi 19 mai 2011
  • 30. Nice dashboard Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 31. http://gaelyk.appspot.com F Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 32. http://gaelyk.appspot.com F Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 33. • Gaelyk is a lightweight Groovy toolkit on top of the Google App Engine Java SDK • Gaelyk builds on Groovy’s servlet support – Groovlets: Groovy scripts instead of raw servlets! – Groovy templates: JSP-like template engine – Both allow for a clean separation of views and logic • Gaelyk provides several enhancements around the GAE Java SDK to make life easier, thanks to Groovy’s dynamic nature 16jeudi 19 mai 2011
  • 34. Gaelyk 0.7 released • Right on time for GR8Conf • What’s new? – Upgraded to Groovy 1.8 and GAE SDK 1.5 – XMPP presence and subscription support – Support for the files service and initial support for the backends service – New variables for files service, backends service, and lifecycle manager – Datastore metadata querying methods 17jeudi 19 mai 2011
  • 35. Why Groovy? • Groovy is a dynamic language for the JVM – very flexible, malleable, expressive and concise syntax – easy to learn for Java developers • deriving from the Java 5 grammar – provides powerful APIs to simplify the life of developers • possibility to dynamically enrich existing APIs – support for Groovlets and its own template engine 18jeudi 19 mai 2011
  • 36. First steps... • Go to http://gaelyk.appspot.com • Download the template project • Put your first Groovlet in /WEB-INF/groovy • And your templates in /WEB-INF/pages • And you’re ready to go! • Launch dev_appserver.sh • Go to http://localhost:8080/ Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 37. The web.xml "> <web-app xmlns="http://java.sun.com/xml/ns/javaee" version="2.5 <listener> /servlet-class> <listener-class>groovyx.gaelyk.GaelykServletContextListener< </listener> <servlet> <servlet-name>GroovletServlet</servlet-name> <servlet-class>groovyx.gaelyk.GaelykServlet</servlet-class> </servlet> <servlet> <servlet-name>TemplateServlet</servlet-name> -class> <servlet-class>groovyx.gaelyk.GaelykTemplateServlet</servlet </servlet> <servlet-mapping> <servlet-name>GroovletServlet</servlet-name> <url-pattern>*.groovy</url-pattern> </servlet-mapping> <servlet-mapping> <servlet-name>TemplateServlet</servlet-name> <url-pattern>*.gtpl</url-pattern> </servlet-mapping> </web-app> 20jeudi 19 mai 2011
  • 38. MVC: Groovlets and templates Groovlets Templates (controllers) (views) Entities (domain)jeudi 19 mai 2011 21
  • 39. A groovlet • Instead of writing full-blown servlets, just write Groovy scripts (aka Groovlets) def numbers = [1, 2, 3, 4] def now = new Date() html.html { body { numbers.each { number -> p number } p now } } 22jeudi 19 mai 2011
  • 40. A groovlet • Instead of writing full-blown servlets, just write Groovy scripts (aka Groovlets) def numbers = [1, 2, 3, 4] def now = new Date() html.html { body { numbers.each { number -> p number } p now } di ng oa } -rel to Au 22jeudi 19 mai 2011
  • 41. A template <html> <body> <p><% def message = "Hello World!" print message %> </p> <p><%= message %></p> <p>${message}</p> <ul> <% 3.times { %> <li>${message}</li> <% } %> </ul> </body> </html> 23jeudi 19 mai 2011
  • 42. A template <html> <body> <p><% def message = "Hello World!" print message %> </p> <p><%= message %></p> <p>${message}</p> <ul> <% 3.times { %> <li>${message}</li> <% } %> di ng </ul> el oa </body> to -r Au </html> 23jeudi 19 mai 2011
  • 43. Shortcuts • Google services • Variables available –datastore, blobstore –request / response –backends, lifecycle –context / application –memcache –sessions –capabilities –params / headers –images –out / sout / html –urlFetch –localMode / app.* –mail –userService / user • Methods available –defaultQueue / queues –include / forward / –xmpp redirect –namespace –println / print 24jeudi 19 mai 2011
  • 44. Groovy sugar! Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 45. Sending emails with Gaelyk mail.send to: to@gmail.com, from: other@gmail.com, subject: Hello World, htmlBody: <bold>Hello</bold> 26jeudi 19 mai 2011
  • 46. ...compared to Java Properties props = new Properties(); ops, null); Ses sion session = Session.getDefaultInstance(pr try { Message msg = new MimeMessage(session); l.com"));     msg.setFrom(new InternetAddress("other@gmai O,     msg.addRecipient(Message.RecipientType.T o@example.com"));                      new InternetAddress("t     msg.setSubject("Hello World");     msg.setText("<bold>Hello</bold>");     Transport.send(msg); } catch (AddressException e) {} } catch (MessagingException e) {} 27jeudi 19 mai 2011
  • 47. Unfair comparision? 28jeudi 19 mai 2011
  • 48. Unfair comparision? 28jeudi 19 mai 2011
  • 49. Unfair comparision? 28jeudi 19 mai 2011
  • 50. Accessing the datastore • Direct interaction with the low-level datastore API Entity entity = new Entity("person")   map // subscript notation, like when accessing a entity[name] = "Guillaume Laforge"   // normal property access notation entity.age = 32 entity.save() // asyncSave() entity.delete() // asyncDelete() datastore.withTransaction { // do stuff with your entities // within the transaction } // use the asynchronous datastore service datastore.async.put(entity) 29jeudi 19 mai 2011
  • 51. Querying to be improved... import com.google.appengine.api.datastore.* der.* import static com.google.appengine.api.datastore.FetchOptions.Buil   // query the scripts stored in the datastore def query = new Query("savedscript")   // sort results by descending order of the creation date query.addSort("dateCreated", Query.SortDirection.DESCENDING)   author // filters the entities so as to return only scripts by a certain r) query.addFilter("author", Query.FilterOperator.EQUAL, params.autho   PreparedQuery preparedQuery = datastore.prepare(query)   // return only the first 10 results def entities = preparedQuery.asList( withLimit(10) ) 30jeudi 19 mai 2011
  • 52. ...into something groovier? def entities = datastore.query { select all from savedscript sort desc by dateCreated where author == params.author limit 10 } as List 31jeudi 19 mai 2011
  • 53. ...into something groovier? def entities = datastore.query { select all from savedscript sort desc by dateCreated where author == params.author limit 10 te d! } as List m en p le t im ill no S t 31jeudi 19 mai 2011
  • 54. URL Routing system • You can have friendly URL mappings with the URL routing system all "/aboutus", redirect: "/blog/2008/10/20/about-us" all "/blog/@year/@month/@day/@title", y@title=@title" for ward: "/blog.groovy?year=@year&month=@month@day=@da get "/blog/@year/@month/@day", @day" forward: "/blog.groovy?year=@year&month=@month@day= get "/book/isbn/@isbn", forward: "/book.groovy?isbn=@isbn", validate: { isbn ==~ /d{9}(d|X)/ } – RouteFilter should be configured in web.xml 32jeudi 19 mai 2011
  • 55. URL Routing system 2.5"> <web-app xmlns="http://java.sun.com/xml/ns/javaee" version=" ... <filter> <filter-name>RoutesFilter</filter-name> ss> <filter-class>groovyx.gaelyk.routes.RoutesFilter</filter-cla </filter> <filter-mapping> <servlet-name>RoutesFilter</servlet-name> <url-pattern>/*</url-pattern> </filter-mapping> ... </web-app> 33jeudi 19 mai 2011
  • 56. URL Routing system • You can also define caching times get "/aboutus", cache: 24.hours, forward: "/aboutus.gtpl" get "/breaking-news", cache: 1.minute, forward: "/news.groovy?last=10" – Nice for GAE’s infamous «loading requests» •less critical since GAE SDK 1.4 – warmup requests – and with paid tier, you can have instances pre-allocated 34jeudi 19 mai 2011
  • 57. URL Routing system • Special routes for specifying – incoming emails – jabber chat messages – jabber presence and subscription messages email to: "/incomingMail.groovy" jabber to: "/incomingXmpp.groovy" // synonym: jabber chat, to: "..." jabber presence, to: "/subs.groovy" jabber subscrpition, to: "/subs.groovy" 35jeudi 19 mai 2011
  • 58. URL Routing system • Namespace awareness: nice for multitenancy • Capability awareness: for graceful degradation // @cust customer variable could be « acme » post "/@cust/update", forward: "/update.groovy", namespace: { "ns-$cust" } es status // different destinations depending on the GAE servic get "/speakers", forward { to "/speakers.groovy" // default destination // when the datastore is not available to "/unavailable.gtpl" on DATASTORE not ENABLED // show some maintenance is upcoming to "/speakers.groovy?maintenance=true" on DATASTORE is SCHEDULED_MAINTENANCE } 36jeudi 19 mai 2011
  • 59. Capabilities • Google App Engine allows you to know the status and availability of the various services – DATASTORE, DATESTORE_WRITE, MEMCACHE... – ENABLED, DISABLED, UNKNOWN, SCHEDULED_MAINTENANCE – is() and not() methods if (capabilities[DATASTORE_WRITE].is(ENABLED)) { // write some content in the datastore } else { // otherwise, redirect to some maintenance page } 37jeudi 19 mai 2011
  • 60. Task queue API // access a configured queue using the subscript notation queues[dailyEmailQueue]   // or using the property access notation queues.dailyEmailQueue   // you can also access the default queue with: queues.default defaultQueue // add a task to the queue queue << [ countdownMillis: 1000, url: "/task/dailyEmail", taskName: "sendDailyEmailNewsletter", method: PUT, params: [date: 20090914], payload: content ] 38jeudi 19 mai 2011
  • 61. Jabber / XMPP support (1/3) • Sending instant messages String recipient = "someone@gmail.com"   // check if the user is online if (xmpp.getPresence(recipient).isAvailable()) { // send the message def status = xmpp.send(to: recipient, body: "Hello, how are you?")   // checks the message was successfully // delivered to all the recipients assert status.isSuccessful() } 39jeudi 19 mai 2011
  • 62. Jabber / XMPP support (2/3) • Sending instant messages with an XML payload String recipient = "service@gmail.com"   // check if the service is online if (xmpp.getPresence(recipient).isAvailable()) { // send the message def status = xmpp.send(to: recipient, xml: { customers { customer(id: 1) { name Google } } })   // checks the message was successfully delivered to the service assert status.isSuccessful() } 40jeudi 19 mai 2011
  • 63. Jabber / XMPP support (2/3) • Sending instant messages with an XML payload String recipient = "service@gmail.com"   // check if the service is online if (xmpp.getPresence(recipient).isAvailable()) { // send the message def status = xmpp.send(to: recipient, xml: { customers { customer(id: 1) { <customers> name Google <customer id=’1’> } <name>Google</name> } </customer> </customers> })   // checks the message was successfully delivered to the service assert status.isSuccessful() } 40jeudi 19 mai 2011
  • 64. Jabber / XMPP support (3/3) • Receving incoming instant messages – Once you’ve configured a route for jabber messages – Add the inbound message service in appengine-web.xml def message = xmpp.parseMessage(request) // get the body of the message message.body // get the sender Jabber ID message.from // get the list of recipients Jabber IDs message.recipients   // if the message is an XML document instead of a raw string message if (message.isXml()) { // get the raw XML message.stanza // get a document parsed with XmlSlurper message.xml } 41jeudi 19 mai 2011
  • 65. Memcache service • Map notation access to the cache } class Country implements Serialzable { String name   def countryFr = new Country(name: France)   in the cache // use the subscript notation to put a country object // (you can also use non-string keys) memcache[FR] = countryFr   // check that a key is present in the cache if (FR in memcache) { the cache using a key // use the subscript notation to get an entry from def countryFromCache = memcache[FR] } 42jeudi 19 mai 2011
  • 66. Closure memoization • Cache the return values for each dinstinct invocation (for a given arguments set) -> def countEntities = memcache.memoize { String kind datastore.prepare( new Query(kind) ) .countEntities() } // first call def totalPics = countEntityes(photos) // second call, hitting the cache totalPics = countEntityes(photos) 43jeudi 19 mai 2011
  • 67. Blobstore enhancements • The blobstore allows to store some large content – images, videos, etc. def blob = ... print blob.filename // contentType, creation, size // output the content of the blob to the response blob.serve response // read the content of the blob blob.withReader { Reader r -> ... } blob.withStream { InputStream is -> ... } // delete the blob blob.delete() 44jeudi 19 mai 2011
  • 68. File service • The file service provides a file-like read/write access over blobs from the blobstore services – programmatic creation of blobs was an issue before def file = files.createNewBlobFile("text/plain", "hello.txt") // writing with a writer file.withWriter { writer ‐>     writer << "some content" } // writing binary content file.withOutputStream { stream ‐>     stream << "some content".bytes } // file.withWriter(encoding: "US‐ASCII") // file.withWriter(locked: false, finalize: false) 45jeudi 19 mai 2011
  • 69. File service • Instead of using the blobstore reading capabilities, you can read from the file service // read from a buffered input stream file.withI nputStream { BufferedInputStream stream ‐>     // read from the stream } // read from a buffered reader file.withReader{ BufferedReader reader ‐>     // read from the reader } // locked and encoding params available too 46jeudi 19 mai 2011
  • 70. Images service • Readable DSL for manipulating images def blobKey = ... def image = blobKey.image.transform { resize 1600, 1200 crop 0.1, 0.1, 0.9, 0.9 horizontal flip // vertical flip too rotate 90 feeling lucky // image corrections } def thumbnail = image.resize(100, 100) 47jeudi 19 mai 2011
  • 71. Channel Service • For Comet-style applications def token = channel.createChannel(clientID) channel.send clientID, Hi! • Then in the view, in JavaScript... channel = new goog.appengine.Channel(token); socket = channel.open(); socket.onmessage = function(msg) { ... } 48jeudi 19 mai 2011
  • 72. Simple plugin system • Gaelyk features a simple plugin system for extending your apps and share commonalities • A plugin lets you – provide additional groovlets and templates – contribute new URL routes – add new categories – define variables in the binding – override existing binding variables – provide any static content – add new libraries – do any initialization – add before/after request hooks 49jeudi 19 mai 2011
  • 73. Simple plugin system • A plugin is actually just a zip file! – Basically, just a Gaelyk application, minus... •the Groovy / Gaelyk / GAE JARs •the web.xml and appengine-web.xml descriptors – But with a /WEB-INF/plugins/myplugin.groovy descriptor • A plugin must be referenced in /WEB-INF/ plugins.groovy with – install myplugin •shortcut to /WEB-INF/plugins/myplugin.groovy 50jeudi 19 mai 2011
  • 74. Simple plugin system • An example plugin descriptor – /WEB-INF/plugins/jsonPlugin.groovy import net.sf.json.groovy.*   // add new variables in the binding binding { le jsonLibVersion = "2.3"          // a simple string variab json = new JsonGroovyBuilder()  // 3rd party class }   // add new routes with the usual routing system format routes { get "/json", forward: "/json.groovy" }   // install a category youve developped categories jsonlib.JsonlibCategory before { req, resp ‐> ... } // or after   // any other initialization code youd need 51jeudi 19 mai 2011
  • 75. Summary • Easy access to a cloud solution –Deploying Java apps, as easily as you would with PHP • Familiar to Java folks –Your good old Servlet centric webapps style • Free tier • Gaelyk provides a simplified approach to creating Servlet centric webapps in a productive manner –Leveraging Groovy’s servlet / template support and dynamic capabilities 52jeudi 19 mai 2011
  • 76. What’s coming next? • Expect more sugar around the Datastore – An SQL-like query DSL • Testing facilities specific to Gaelyk –testing utility class, Spock-specific ones... • Anything that’ll come up in newer GAE SDK versions –in particular, Full Text Search! 53jeudi 19 mai 2011
  • 77. Not only me... Benjamin Muschko Vladimír Orany Marco Vermeulen Gradle GAE plugin Gradle Gaelyk Spock testing Gradle Gaelyk plugin Eclipse plugin integration Gaelyk datastore TDD Gaelyk viewer plugin template (w/ Spock & Easyb integration) Eclipse DSLD file 54jeudi 19 mai 2011
  • 78. One more thing... Europe 2011 — Copenhagen, Denmarkjeudi 19 mai 2011
  • 79. CloudFoundry Gaelyk variant 56jeudi 19 mai 2011
  • 80. CloudFoundry Gaelyk variant p orw are Va 57jeudi 19 mai 2011
  • 81. Thank you! e aforg pment ume L Develo Guilla Groovy om He ad of e@ gmail.c glaforg rge Email: @glafo : Twitter 58jeudi 19 mai 2011
  • 82. Image credits • Speed limit : http://www.morguefile.com/archive/display/18492 Warehouse : http://www.morguefile.com/archive/display/85628 Check mark: http://www.lnl.infn.it/~epics/WikiDumps/localhost/600px-symbol_ok.svg.png Puzzle: http://www.everystockphoto.com/photo.php?imageId=263521 Light bulb: https://newsline.llnl.gov/retooling/mar/03.28.08_images/lightBulb.png Weight balance: http://www.medicalscale1.com/wp-content/uploads/2010/11/balance-weight-scale2.jpg Clouds http://www.morguefile.com/archive/display/627059 http://www.morguefile.com/archive/display/625552 http://www.morguefile.com/archive/display/629785 Duke ok GAE http://weblogs.java.net/blog/felipegaucho/archive/ae_gwt_java.png Python logo : http://python.org/images/python-logo.gif Gaelic cross with clouds : http://www.morguefile.com/archive/display/37889 Snow foot steps : http://www.flickr.com/photos/robinvanmourik/2875929243/ Sugar : http://www.flickr.com/photos/ayelie/441101223/sizes/l/ Press release: http://www.napleswebdesign.net/wp-content/uploads/2009/06/press_release_11.jpg Steve Jobs http://blogs.lentreprise.com/auto-entrepreneur/wp-content/blogs.dir/233/files/2010/07/Steve_Jobs_WWDC07.jpg Duke Nukem Forever http://img.jeuxvideo.fr/photo/00704014-photo-duke-nukem-forever.jpg 59jeudi 19 mai 2011

×