Downloaded 12 times
Uploaded byGR8Conf
Jan reher may 2013
The document reflects on the use of Grails and Groovy for a real-world project, highlighting both the successes and mistakes encountered. Key lessons include the importance of investing in education and understanding of the technologies, engaging experts to collaborate on complex tasks, and properly surveying available solutions. The author emphasizes that while the framework allows for efficient web application development, challenges such as slow compilation and unoptimized code arose during implementation.
More Related Content
![[2020 git lab commit] continuous infrastructure](https://cdn.slidesharecdn.com/ss_thumbnails/2020gitlabcommitcontinuousinfrastructure-200827203956-thumbnail.jpg?width=640&height=640&fit=bounds)
Jan reher may 2013
- 1. We chose Grailsand Groovy for a real-world project. Was it wise? Jan Reher, Systematic
- 2. Topics • The systemwe built • Technologies that start with a G • Mistakes made, lessons learned • The presentation I would have liked to hear two years ago • Was it wise? 2
- 3. 3 Founded in 1985 430employees, 400 in Denmark • Defence • Healthcare • Public Sector • Intelligence and National Security
- 4. Jan Reher Does: • Softwaredesign • Programming • Technical leadership • Requirements elicitation • Workshop facilitation • Teaching • Knowledge networking • Process improvement Likes: • Most of what he does • C++ • Functional programming • Tolkien • Gardening • Karate • Desk-top role playing games jan.reher@systematic.com MS, Computer Science BS, Mathematics, Physics 15 years at Systematic 4
- 5.
- 6. Property Trading Process AccountSwitching Process 6 e-nettet.dk Two major features Work started August 2011 First version in production May 2013 Work started August 2012 First version in production 2014
- 7. Backend Business logic Common Component PortalActor System Internet Explorer HTTPS HTTPS HTTPS MQ Common Component Common Component Business logic Business logic HTTPS MQ Data base Architecture No security Session level OCES (X.509) security Session data only Per-call OCES (X.509) security Internet, no security SOAP, WSDL, XSD Java, JAXB, JPA Oracle WebLogic Oracle DB Grails Oracle WebLogic Internet Explorer 7
- 8.
- 9. <complextype name=”SavingsAccountRequest”> … </complextype> class SavingsAccountRequest{ // fields // constraints } <g:render template=”…”/> … savingsAccountRequest/_show.gsp class SavingsAccountRequest { // fields // getters and setters } JAXB manually manually <g:render template=”…”/> … savingsAccountRequest/_edit.gsp SavingsAccountRequest.java SavingsAccountRequest.xsd Product framework, types SavingsAccountRequest.groovy Specification View Model DTO 9
- 10. Product framework, objects SavingsAccountRequest Editform SavingsAccountRequest Java (JAXB) object SavingsAccountRequest Grails object bindData() IsomorphicMapper Back-end Web service operation 10
- 11. Product Framework, labels BetaltPALSkatForegaaende5AarFoerUudnyttetPALSkat BetaltPAL skat foregående 5 år før uudnyttet PAL skat 11
- 12. Testing • Unit testsin Spock • Integration tests in Spock • Automated GUI tests in Geb and Spock 12
- 13. Not surveying theGrails landscape Spring WS plugin ? JSON RESTful API for GORM plugin ? Something else? Therefore: • Look for third-party solutions • Invest in background knowledge Backend Business logic Common Component Portal Actor System Internet Explorer HTTPS HTTPS HTTPS MQ Common Component Common Component Business logic Business logic HTTPS MQ Data base 13
- 14. Believing that Groovyis just Java Programmer quote: The biggest mistake was to think that you can just ”write Java in Groovy”. You end up getting the worst of both. Lesson: Invest in learning Groovy. 14
- 15. Not investing ineducation 15 Programmer quote: The biggest mistake was lack of initial education, understanding and preparation for developing Grails applications. This can be due to a downside of another tool we are using: Scrum. The fact we had to deliver functionality a very short time after project start didn't leave time for proper education and understanding of the framework. We still have to live with code written at that time. Lesson: • Invest in learning Grails. • Grow experts Quote from grails.org: Build modern, sophisticated and robust Groovy web applications in record time!
- 16. Letting the consultantwork alone Programmer quote: The biggest mistake was to let a consultant do all the difficult stuff alone, instead of acting as a consultant, and helping us do the the difficult stuff. Lesson: Get an expert. Have him teach and coach. Do not let him work alone. 16
- 17. Using Groovy onthe back-end • Groovy/Java interaction works fine • Slow compilation • Long turn-around • Ungroovy code 17
- 18. The biggest mistake 18 Notsurveying the Grails landscape Believing that Groovy is just Java Not investing in education Letting the consultant work alone Using Groovy on the back-end Letting the consultant work alone
- 19. Happy notes Grails Groovy Spock Geb Online resourcesand community Books Project enthusiasm Project benefits page (#) Hooray!
- 20. 20 We chose Grailsand Groovy for a real-world project. Was it wise?
Editor's Notes
- #2 My name is Jan Reher. My title is LeadSystems Engineer, whichmeans I am a programmer ofsomeseniority. I alsoteachvarioussubjects and do process definition and processimprovement, technologystrategy and knowledgenetworking.I willleavesome time at the end of thispresentation for questionsand comments.
- #3 I want to talk about the system we have built, and the project organisation thatbuilt it. And I want to highlight a fewaspects of the systems to demonstratehowwe have usedGroovy and Grails and otherthingsthat start with a G. I want to talk abouthowweintroducedthesetechnologies, and about the choiceswe made.I willbeshowing a bit of codealong the way.I want to highlight some of the mistakeswe made. This in the hopethatyouwilllearn from thesemistakes, and not repeatthem.My intendedaudience is twogroups: One consists of software professionals likemyselfthatuse or areconsideringusingGrails and things for developing and delivering software solutions to customers. I am trying to give the presentationthat I would have liked to heartwoyearsagobeforewestartedusingGrails. The othergroup is the Grails and Groovycommunity. I want to give you an example of howthis wonderful thingyou have built is being put to work in the industry, hpoingthatyoucanusethis feedback to furtherimprove the technologies. I will assume some knowledge of Grails and Groovy. But not much. If you attended this conference yesterday, you know enough.Finally, I want to answer the question in the title.
- #4 A little bit aboutourcompany, just for background information.Systematic is Denmark’slargestprivatelyowned software company.Weare a software development house.We have a few products but spend most of oureffortexecutingdevelopmentprojects for customers.Ourprimaryareas of expertice: defense, healthcare, integration services as well as intelligence and national security.Projects spans from 3 developers doingmaintenance and support on smaller system, to large complex systems with projectsizes up to 70 persons. Wework in different domain using a variety of platforms and technologies.One thingwetakepride in is ourability to consistently deliver projects on time. One of the factors thatenablethis is thatwe have a levelfivecertificateaccording to the CapabilityMaturity Model. If youknowwhatthat is, youknowthatveryfewcompaniesare at levelfive.Visit us at Systematic.com
- #5 Professional experienceincludesC, C++, C#, Visual Basic, Java, Groovy. LikeVenkatsays, new languages force you to learn to think in new ways, which is healthy.Favourite: C++. Experience, expressibility, evolvinglnguage. One of the reasons I likeGroovy is that it is almost as rich as C++, but different. I likevariety and change.
- #7 The system we have built has two major feature. I just demoed a simple success scenario in one of them.This is a profoundly Danish systems, built for a Danish customer, to serve in a Danish context. So the user interface and the domain language is Danish.It is a dry and no-nonsense integration solution with a dry and no-nonsense back-officeuser interface on top. It is about data validation and transport.The customer is e-nettet, a company made and owned by the Danish banks and mortgage banks to serve as a commonground for servingcommon problems.The storydoesn’t end here. Weareworkingtowards the nextdeliveries as we speak.
- #8 This shows the architecture of system 1. A fairly standard 4-tier architecture.Weareresponsible for all of the components here, except the actorsystems.Theyareprobably not builtusingGrails.The back-end is a system-to-system broker in thatitsresponsibility is to route information amongactor systems. There is verylittle data stored in the data base, reallyonly the states of the processes. Actor systems must maintaintheirowncopies of the data contents of processes.Actor systems call web services, and exposetheirown web services called from back-end.[ANIMATION STEP] The services arespecifiedcontractfirst, SOAP, WSDL, XSD. Back-end is written in Java. Web service specification is translated to Java using Java Architecture for XML Binding, JAXB.The database is Oracle DB. Access is via the Java Persistence API.The Portal is almost but not quite anactor system. It is a Grailsapplication, and I willbefocusing on it inthispresentation[ANIMATION STEP] Security is based on the open-source implementation of X.509 called OCES. It is in origin a Danish standard, and is in wide-spreaduse in Denmark. All Danes in thisroomuse it. It is loosely the thingcalledNemID.It is a requirementthat all data acess must use per-callauthentication and must belogged. Therefore, the Portal cannot store anyprocess data but must use the back-end for storage. So the Portal does not useGORM/Hibernate. It uses the public web services on back-end plus portal-only web services for storageetc on back-end. So I waslying a bit beforewhen I saidthat the data base containedverylittle: It actuallycontains all data aboutcurrent processes, but that data belongs to the Portal. Thus wesatisfy the data accessrequirement, and alsogainthat a first-orderbreach of the Portal’ssecuritywill not cause data to beexposed.So, there is lessGrails and Groovytechnology in thisarchitecturethantheremight have been. Grails is the newcomerhere to a technologystackthat is otherwiseproven and well-known to the customer and to us, becausewe have used it for a previous non-Grailsproject. But, hey, Grails, let’strysomething new for this new Portal component. Thatwaswhat the project’s original architectdecided. Sometimesyouneed to take a leap to move forwards.This arcitecture is fine. Not veryflashy but tried and true. It has serveduswell.
- #9 I want to highlight a part of the system’s design. We have builtsomethingthatwethink is prettyneat.Pull out product slips. Show UI. System must support editing and display of a large number of product types.Show Product classhierarchy. Thesearedefined in XSD. XSD types specify the product types in the domain. Thereare ”lots” of them, currently 15. Theywillchange and grow over time. Theyusecommon patterns likechoicesbetween elements and collections of elements. Common sub-elements likeaccountnumbers and otherfinancial primitives areused in all products.Show createrequest page. Show somerequest types.Show XSDSo the task is this: For everyproduct type we must provide an edit form and a display for the request and for the nota. Somearequite simple (IndlånAnmodning, IndlånNota). Othersare large: SikkerhedWhy not ask Grails to generate views? Well, wetried. It seemed to usthat the resulting GUI wouldbetoo primitive. It alsoseemed to usthat component data types likeaccountnumbers and suchwere not handledproperly. So we gave it up, and implemented a solution ourselves. Therearethreepossibleconclusionshere. 1. Weweretoo stupid or did not tryhardenough to usegeneratedviews. 2. Grails has a mechanism for solvingthis problem but it is toodifficult to understand and use. 3. Our problem is not so commonthatanyone has published a general solution. I willnow show our solution.
- #10 From the XSDwe generate the deriveddescriptionshownhere. And wedo it recursively for the constituent types in the XSD, e.g. for an account element, which is used in severalplaces in the product types.Thesedescriptions of a savingsaccountrequestareisomorphic in the sensethatthey have elements with the same names and with types thatareisomorpic but expresseddifferently in XSD, Java, Groovy, and GSPShow code
- #11 bindData(): Spring Data BindingIsomorphicMapper: custom-made generic mapper.If time and moodallows:Show code from ProductController.saveRequest() to call to ProductMapperSo this is ourproductframework. It allowsus to add and modifyproduct types to the system, and since the handling of product types is a core feature of the system, it is wellworthcreating a general solution.An obviousimprovementwouldbe to generate the Grails model type and the view from XSD instead of codingthemmanually. Well, maybewewill do this.But is there a library or tool out there that does all this already? In a way, I hope not, because then have wasted effort. I have looked but did not find anything. Maybe Peter McNeil’spresentationtodaycontainssomething pertinent to this problem.
- #12 Here is the final element of the productframwork.The XSD element namesexpressdetailed domain knowledge and arealso the origin of all labels thatweneed to put into the user interface to tell the userwhat data to write in a given field. Thereare 520 of these element names, and 145 enumerationvalues. These must beturnedintofield labels and comboboxvalues.Here’s an element name [ANIMATION STEP]. If you understand Danish (I do) and have sufficient domain knowledge (I don’t) youknowwhich label it wouldbeappropriate to put next to the input fieldthatcorresponds to this element in the XSD. Wedon’twant to enter all these labels in the messages.properties file. Wewant to generate them. And in most cases, the element name is directlyusable as a label on the user interface. It just needs a bit of polish.The algorithm is simple:Split at numbers and camelcasing [ANIMATION STEP], takingintoaccountthatsequences of capital letters areacronymsDowncase everythingexceptacronyms [ANIMATION STEP]Replacecertaintwo-vowelcombinations with special Danish characters. [ANIMATION STEP]BecauseourGrails domain types areisomorphic to the XSD types, the fields in the Grails domain types have the same names as the elements in the XSD types. The element namesarepassed to our GSP templates so thatGrailscan bind them to fields in the Grails domain types.Here is how it looks in our template GSP file thattakescare of rendering labels: [SHOW _label.gsp]. Note thatwecan override the generated label in case we do not like it by setting a valueexplicitly in the messages.properties file.The LabelGenerator looks likethis. [SHOW LabelGenerator]. It was a real pleasurecoding it in Groovy, due to Groovy’s support for regularexpressions and itsrich and neat sub-language for manipulatingsequences. For instancesequence[0..-2] means all elements in the sequenceexcept for the last twoones.
- #13 In project 2, most of our tests areautomated. Theycome in threecategories: Integration tests, GUI tests and unit tests.WeuseSpock as a testingframework.Working with Spock is an unconditionaldelight. It is the besttestingframework for anylanguage I have evermet; vastlysuperiour to JUnit.There is noSpockpresentation at Gr8Conf thisyear but ifyou do not knowSpock, youshould look into. As an example, here is one of our unit tests: IsomorphicMapperSpec. It is a test of the mapper codethattranslates back and forthbetweenGroovy types and JAXB types. Notice: Spec, methodnames, when/then, nonoisyannotiations, nocalls to assert.And here is anotherone: LabelGeneratorSpec. Notice: expect/where.Sadlywe have veryfew real unit tests in our systems. And wemakenouse of Spock’s excellent facilities for mocks and stubs.Second is the set of tests thatwecall the integration tests. Theseareblack-box tests thatexecute as unit tests but call the web service operations on a deployedinstance of the backend. Some of themareacceptance tests, that is the tests thatwe must execute to demonstrate to the customerthatour system works. As an example, see ProductSendRequestTest."T2310 Send product then case with 2 products then product. Then a case with 2 products are stored in DB"(). Notice: title, blocks.Becausewe have to write the integration tests to getacceptance, and because the back-end as designed in project 1 is not very amenable to unit testing, we have never gottenaround to writingverymany true unit tests. This is a problem sinceexecuting integration tests takequite a long time. Long turn-aroundFinally, the automated GUI tests. See TestIndlaanProductCreate. "T3620 Select Indlaan product and fill out all fields and save product“. Start executing it. Pros:.Likeanyother set of automated tests, theseact as a safety net under uswhenwecode. Ourbuild server executesthemregularly, likeotherautomated tests. Automated GUI tests are still a relatively new thing in our organisation. I wouldn’twant to be on a projectwithoutthem.Technique: Spock. Test, Page, Module. Jquery. Problems: Waits, hardwaits, IE/Firefox, execution time: slow and sensitive to environment.It is time to look at some of the mistakeswe made on the project. I asked people on the project what they thought was the biggest mistake. Here’s the firstone:
- #14 So the architectdecidedthatwecannotuse GORM/Hibernatebecause data bustbestored on the back-end.Therefore the service layer of the Portal explicitlyconvertsbetweenGrails domain types and JAXB types, and calls web service operations on Backend to store and retrieve data. The backendimplementation of the web service operation stores the JAXB object in the Data base using JPA. This is a bit tedious but it works fine.The mistake is this: WeonlyrecentlydiscoveredthatthereareGrailspluginsthatprobablycould do this for us. There is somethingcalled the Spring WS plugin. ANIMATION STEP. There is somethingcalled the JSON RESTful API for GORM. GraemeRochertalkedabout it yesterday. ANIMATION STEP. And theremightbeSomething Else out there. We did not event knowwhat to Google for. ANIMATION STEP. It doesn’treally matter whichpluginweought to use, and wehaven’treallystudiedthesecandidates. The point is thatweconcludedthatwewould have to codethisourselveswithouteventrying to discover ifsomebody had alreadybuiltsomethingwecouldreuse. And really, weought to have realisedthatwewere not the onlyones with thisparticularsecurityconcern.The lesson is twofold: [ANIMATION STEP] First, always look for third-party solutions to common problems beforeyoudecide to code the solution yourself. [ANIMATION STEP] Second, invest in acquiringbackgroundknowledgeabout a new technology suite likee.g. Grails. Youwill not use all youlearn but youwilllearnsomethinguseful.Maybethiswasourbiggestmistake
- #15 Now I will do Groovy an injustice, and show you how not to introduce Groovy to a seasoned Java programmer. GO TO TEXT EDITOR.Strip the code: semicolons, type annotations, visibility markers, getters and setters, constructors.[Use map constructor with misspelled parameter to construct object:def w = new Whatever(date : 0)]Some of our programmer s still look at Groovy as a sort of wobbly, dysfunctional form of Java. They are puzzled and suspicious: “OK, so no semicolons. Well, whatever. But no type checking, just run-time errors? And visibility markers have no effect. Really, what is the point? Why would we want to exchange Java for this?”The result is Groovycodethat is not verygroovy-like. Meaningwegetnobenefit from the powerful features in thatlanguage. We just getwobbly Java and somefrustrated and unhappy programmers. I askedour programmers whattheythoughtourbiggestmistakewas, and one of themwrotethis: [ANIMATION STEP]Lesson: Don’tthinkGroovy is almost Java. Almostanyprogramminglanguage is almostanyotherprogramminglanguage but only to a firstapproximation. Youneed to invest in learninghow to codeGroovy in a native, groovyway. And on projecttwo, we have done so, with the resultthat the groovynes of ourcode has muchimproved.Maybethiswasourbiggestmistake
- #16 Another programmer quotethatalso points to a possiblerootcause. READ IT ALOUD.In a broaderperspective, this points to a potential pitfall of working in an agile way. But that’s a wholeotherpresentation, possibly at anotherconference.The lesson is the same. Learning a new technology is not something you can do while running a project at full speed. You have to set aside time for learning, and for remedying the beginner’s mistakes that beginners make.This is not Grails’ problem. Grails is fine. But it is a problem for all who wish to take on a new technology. And the problem is bigger the more radical the new technology is.Maybethiswasourbiggestmistake
- #17 Project 1’s plan called for an early release of a skeleton Portal and Back-end. The project team had very little knowledge of Grails. To remedy this, we hired a Grails consultant.With deadlines looming, and the project facing other problems (unstable requirements, team building, development environment construction), the consultant was charged with implementing most of the portal, and the rest of us were asked not to disturb. The result was a functioning Portal, which only the consultant understood. When he left, we had to struggle to catch up.[ANIMATION TO REVEAL QUOTE]Lesson: Get an expert. Then tie his handsbehind his back.Maybethiswasourbiggestmistake
- #18 Using Groovy on the back-end. We decided to code the account switching business logic on the back-end in Groovy, instead of Java as the rest of the back-end.The reason for thiswasreallythathavingoneprograminglanguage on a project is superior to havingtwo, especiallysince the new language is much more expressive and cool than the old one.Compilation of code base in twolanguages is slow. We have beenunable to get hot deploy of the backend to work. We have few unit tests because the system 1 back-end architecture is not very amenable to unit testing. And wemake stupid mistakes in Groovythat a compiler with static type checkingwould have caught, not the leastbecauseprogramers with Groovyskillstend to work on the Portal wharethatskill is most needed. This meansthatourseasoned Java programmers do most of the back-end programming, and theyare not partivularlygood at or fond of Groovy. So we have to build and deploy to a web server and then run our web service tests to get feedback. All thismakes for a long turnaround time whencoding. This particularmistake is mine. It wasmyidea to code the back-end in Groovy. Oh well. And, yes, I knowaboutGroovy’s@TypeChecked annotation; we just never have gottenaround to using it. Thatmightbe a mistake.Maybethiswasourbiggestmistake
- #19 What do youthinkwas the biggestmistake?My mistake of choice it this: Letting the consultant work aloneThe reason is this: If we had used the consultant as a consultant, not just a hiredhand, wecould have usedhim to survey the Grails landscape for us, looking for tools and techniques. Wecould have gotten his advice on critical decisions, hecould have taughtushow to codeGroovy in the Groovyway. And so on. We did usehim for thesethings but not enough. And many of ourothermistakes have theirroot in thisone.So if I could do theseprojects all over again, this is the onething I woulddefinitelywant to change.
- #21 Now, the conclusion: READ IT.YesAll in all, selectingGrailswas a goodchoice. We have had problems,we have made mistakes and bad decisions. But thisgroup of technologieswhosenames start with a G has been a great asset. I would do it againanyday, but do it better, based on what I have learned in theseprojects.Meanwhile back at Systematic, right nowpeopleare still working on thesetwo systems. And we have gottwootherprojects in the company, for othercustomers, thatarealsousingGrails and Groovy. The story does not end here.
- #22 Questions and Comments?