Groovy & Grails eXchange 2012 - Building an  e-commerce business with gr8 technologies in Latin America
Upcoming SlideShare
Loading in...5
×
 

Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America

on

  • 1,813 views

Presentation given in the Groovy & Grails exchange 2012 at London.

Presentation given in the Groovy & Grails exchange 2012 at London.

Statistics

Views

Total Views
1,813
Slideshare-icon Views on SlideShare
1,771
Embed Views
42

Actions

Likes
3
Downloads
20
Comments
2

4 Embeds 42

https://twitter.com 20
http://lanyrd.com 17
http://www.linkedin.com 4
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • if we have millions of product details as static generated html file in document root then does it affect the server performance as we have millions of file in single folder?
    Are you sure you want to
    Your message goes here
    Processing…
  • Thank you for sharing this slide. I'm impressed with the way you implement it. Generating static page is very very crazy idea!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Hi my name is Domingo Suarez, I'm from Mexico City, and this afternoon I'm going to talk about my experience building the largest e-commerce website in Mexico with grails. I will talk about what worked well, what didn't, some lessons learned and so on. I only want to share my experience in my context
  • Before I start, let me tell you a little about my background. I've been using Grails since late 2007, my first website built with grails went live in 2008. For all my professional career I've been a java developer, mostly on the server side. In Mexico I founded the Groovy/Grails user group. In the Grails48 hackaton my team won the second place for developing a kanban board for github issues. Let’s get started...
  • For the last two years I've been working at clickOnero, an ecommerce company with a national presence in Mexico. Here is the story, In 2010 a friend of mine asked me if I knew an available Grails developer for hire. In Mexico is very hard to find Grails developers, very hard... At the end I accepted the job and joined clickOnero, a company with Swiss investors. The clickOnero investors made more investments around the world with the same business model.
  • In he beginning, The clickOnero's business model was to sell coupons for daily deals. The CEO asked us to develop an in-house solution for this.
  • We built the solution in 2 months, with a team from all over the world, 4 developers from: Germany, UK, Greece and Mexico. I went to Germany to work together with them and then came back to Mexico to deploy and launch the company. The first version was built using the common Grails artifacts, controllers, gsps, taglibs, domain classes and services. This was deployed to tomcat with MySQL as database. When we started to build cooldeals, we made some crucial technical decisions.
  • For example, to cache heavily with ehcache, backed with terracotta in production.
  • From the very beginning we had some requirements related to massive emails campaigns and chose RabbitMQ for long running operations like sending emails.
  • We went live in Mexico the first week of September 2010, one week later we launched the website in Argentina, both websites with the same branding and the same platform. Two weeks later, the Dubai version went live with a different branding but same platform. The platform was built with multi currency and multilingual support, for 2 different continents and 6 different countries (the Dubai version runs in 4 different countries in the Middle East)a
  • Meanwhile in Mexico, I help put together the dev team. I hired 4 local Java developers and converted them to Groovy/Grails developers, it was really easy to do. The developers were very excited about the new possibilities and simplicity of the language and frameworks.
  • But, we made bad things. In the same application, we built the website for the customers and the administration backend. We wrote only test cases for the critical pieces (checkout, user registration). The internal backend was really a mess!
  • We started to have issues. Slow response times. We didn’t know where the problem was We used JavaMelody to detect bottlenecks and long running code. It was really helpful to fixed the issues we had.
  • JavaMelody generates this graphs, where you can see problem in your app. Memory problems, jdbc problem, threads
  • This is an example of jdbc queries. You can see the hit count, mean time response, max, min Of course you can use this data to identify bad code in your app
  • As an IT department, we were responsible for the hardware infrastructure as well. No sysadmin and no dba in the company, only 5 developers. The hardware we had was 4 app servers(tomcat) and one database(MySQL) server. Here is a bit more technical detail on these servers: With this infrastructure we were able to support the company's growth. By late 2011 we had 2 million users, with 50k concurrent users. Show logstalgia visualization
  • As an IT department, we were responsible for the hardware infrastructure as well. No sysadmin and no dba in the company, only 5 developers. The hardware we had was 4 app servers(tomcat) and one database(MySQL) server. Here is a bit more technical detail on these servers: With this infrastructure we were able to support the company's growth. By late 2011 we had 2 million users, with 50k concurrent users. Show logstalgia visualization
  • The company worked very well for several months, in fact the last year we won the prize for the best Internet company in Mexico, a prize granted for the Mexican Internet Association (AMIPCI)
  • At December 2011, the company was acquired by Mexican investors and the company started to change to improve the business performance.
  • At this time the investors acquired other e-commerce startups to increase the clickOnero market:-MiPlan.com was a ecommerce company specialized in offering national and international trips. When this company was acquired, the clickOnero user base increased to 3 Million users-PezUrbano is a daily deals website from Brazil, merging the two platforms is on the works. The clickOnero's user base will increase by 1.5 million users
  • We divided all the system in several components:-API, Grails application only for data exchange. This app exposes http services for getting data with basic auth http. In this app, we built the checkout, stock, user registration, mailing list subscription services.-Hipstore. A totally static HTML application built with Chaplin, an abstraction over backbone, with a templating engine. This app is served from the web server, with gzip and cached in the client with etag if available. This app consumes the API services-Admin. Grails application for internal use, the production team uses this app to create items campaigns, the customer service department uses this app for customer support. The sales team uses this app for tracking sales
  • The admin apps users made some changes in the database items, when the change is committed to the database, the admin app sends a JSON message to a RabbitMQ A node app listen that message and with ZombieJS navigate the
  • We saw the users spend more time in the store, we reduced the bounce rate and guess what, We went down from 4  to 2 app servers. The users didn't notice the reduction in hardware. (show the google analytics image)

Groovy & Grails eXchange 2012 - Building an  e-commerce business with gr8 technologies in Latin America Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America Presentation Transcript

  • Building ane-commerce businesswith gr8 technologies in Latin AmericaGroovy & Grails eXchange London14th December 2012Domingo Suarez Torres@domixlunes, 29 de abril de 13
  • Agenda• About the Business• Version 1• Version 2• Lessons learnedlunes, 29 de abril de 13
  • About me• Java & Grails developer from México City• grails.org.mx Mexican G&G communityFounder• My team won the second place in theGrails48 hackaton.• TrollBoard. Kanban Board for GitHub Issues• http://trollboard.rs.af.cmlunes, 29 de abril de 13
  • clickOnero• Started in 2010 as a Daily Deals website.• German/Swiss investors• Late 2011 acquired by Mexican investors• Today it is a 100% Mexican company witha pure ecommerce modellunes, 29 de abril de 13
  • clickOnero 1.0Grails, Terracotta, RabbitMQ & mySQLlunes, 29 de abril de 13
  • clickOnero 1.0• CodeName: CoolDeals• Typical Grails application• MySql,Tomcat, ehcache (Terracotta),RabbitMQ• Built in 2 months by 4 devs from Germany,UK, Greece and Mexicolunes, 29 de abril de 13
  • Caching• Prevent excessive database access• ehcache, backed with Terracotta• Stored in cache• Domain classes• HTML pages & fragments• JSON, XML documentslunes, 29 de abril de 13
  • Long running code• RabbitMQ• Messaging with AMQP• Asynchronous processing• Mainly to use STMP services. Millionsemails sent daily• Financial reports (long complex queries)lunes, 29 de abril de 13
  • CoolDeals Deploymenthttp://www.springstar.comlunes, 29 de abril de 13
  • CoolDeals Deploymenthttp://www.springstar.comlunes, 29 de abril de 13
  • CoolDeals Deploymenthttp://www.springstar.comlunes, 29 de abril de 13
  • Dev Team @ Mexico• Very hard to find Grails developers• Java developers -> Grails developers (4)• It’s easy to do!• New Grails developers excited about newpossibilities and simplicity of the languageand frameworkslunes, 29 de abril de 13
  • Very Bad stuff• Test cases only for critical pieces• checkout, user registration, mailing listsubscription• Very low code coverage.• No Continuos integration• Internal backend was a mess, a lot of code incontrollers• Bad GORM querieslunes, 29 de abril de 13
  • Enters JavaMelody• The goal of JavaMelody is to monitor Javaor Java EE application servers in QA andproduction environments.• It is a tool to measure and calculatestatistics on real operation of an applicationdepending on the usage of the applicationby users.• ¡Grails Plugin!lunes, 29 de abril de 13
  • lunes, 29 de abril de 13
  • JDBC Querieslunes, 29 de abril de 13
  • JavaMelody• It helps us improve our app!lunes, 29 de abril de 13
  • More Plugins• RabbitMQ• Spring Security• Rendering• Quartz• Feeds• Export• Recaptcha• AWS• Joda Time• Java Melodylunes, 29 de abril de 13
  • Hardwarelunes, 29 de abril de 13
  • Boxes• 4 WebServers• DBServer• Physical Firewall• LoadBalancerlunes, 29 de abril de 13
  • Numbers• 200k-300k visits daily• 1.5M > page views daily• 80K concurrent users• 20K-50K new users daily• 1000-3000 purchases daily• 2.5M registered userslunes, 29 de abril de 13
  • Logstalgia-httpd log vizlunes, 29 de abril de 13
  • “Ecommerce Companyof the year in Mexico”lunes, 29 de abril de 13
  • Lessons learned• Cache is critical to scale• The content in our app is ‘static’, fewchanges, sometimes no changes.• If the content is ‘static’, why do we alwaysexecute code to render that content, evenif the content is cached?lunes, 29 de abril de 13
  • clickOnero 2.0Single Page Application (chaplin, backbone)lunes, 29 de abril de 13
  • Motivation• December 2011 clickOnero was acquiredby Mexican investors• The company started to change.• The new CEO asked to develop a brandnew store to sell different items like:clothes, shoes, trips, services (hair cuts,massages, cinema tickets, etc)lunes, 29 de abril de 13
  • New markets• clickOnero acquired other e-commerce startupsto increase the market• Miplan.com: was a e-commerce companyspecialized in offering national and internationaltrips. 500k users• PezUrbano is a daily deals website from Brazil,merging the two platforms is on the works.1.5M users• Very close to 5M userslunes, 29 de abril de 13
  • Improvements• Split the complete system in severalcomponents• API: Grails app for exchange information• Admin. Grails application for internal use• Hipstore.A totally static HTMLapplication built with Chaplin. Not aGrails app.The store for the userslunes, 29 de abril de 13
  • lunes, 29 de abril de 13
  • Grails Apps• API• Typical Grails application, with no GSPs.• Speaks only JSON• Used by partners (remote services) andHipStore• Admin• Typical Grails applicationlunes, 29 de abril de 13
  • Developmentenhancements• Increased test cases in both Grails apps• Spock• Jasmine for JavaScript code• Introduced Jenkins• Jenkins Jobs to deploy automatically to QA &Production environments• Bash shell scriptslunes, 29 de abril de 13
  • HipStore• Static HTML application built with Chaplin• Chaplin is an architecture for JavaScriptapplications using the Backbone.js librarylunes, 29 de abril de 13
  • HipStore• Developed in CoffeeScript• Uses PushState• RequireJS (AMD Support)• HandleBars (Template Engine)• JQuery• Underscore• Twitter Bootstrap• Build and packaged with Jakelunes, 29 de abril de 13
  • HipStore• Single Page Application• Chaplin consumes JSON from the API torender the store items.• Uses PushState to update the URL in thebrowser.Very useful for bookmarking andsocial media sharing, even for SEO.lunes, 29 de abril de 13
  • Sample URLs• http://{server}/store/• http://{server}/store/{category}• http://{server}/store/{category}/{campaign}• http://{server}/store/{category}/{campaign}/{item}• Above urls valid for the Chaplin router, but invalidfor the browser, because them doesn’t exist. Ifyou try to use it, you’ll get a 404lunes, 29 de abril de 13
  • HashBang• http://{server}/store/#!/{category}• Not very friendly with search engines.• Google will try something like:• http://{server}/store?_escaped_fragment_=• Please read: https://developers.google.com/webmasters/ajax-crawling/lunes, 29 de abril de 13
  • Our approach• Write to disk all the possible links inHipStore. Crazy?• We use ZombieJS to navigate the websiteand then write to disk the generated HTML• Put those static files (HTML) in thewebserver document root• The best cache everlunes, 29 de abril de 13
  • Why do this?• When the user visit our website, thewebserver will respond with completelystatic HTML, CSS, JavaScript files• Very fast• The user never hits the Tomcats, we reducethe load in the app servers.• The webserver always responds very fastlunes, 29 de abril de 13
  • ZombieJSlunes, 29 de abril de 13
  • HTML harvesting• The Node app receives the JSON message• Navigates to HipStore with ZombieJS• Executes the JavaScript (Chaplin app)• Extracts the generated HTML with jQuery• Saves to disk in the web server documentrootlunes, 29 de abril de 13
  • Things to consider• When an item in the Store is modified, we need toregenerate the appropriate HTML file only once.• Then all the users will receive the same file• The user only hits the Tomcat when really need it(CheckOut, Registration)• When the user click in some action in the app, theinteraction is handled by Chaplin controller if thebrowser supports JavaScript• Remember in our website the content is almost static?lunes, 29 de abril de 13
  • Results• The load in our Web Servers was reduce alot.• The load in the database reduced a lot.• The users can share the links.• Store becomes very search engine friendlylunes, 29 de abril de 13
  • Some statslunes, 29 de abril de 13
  • Thanks!• Q&Alunes, 29 de abril de 13