Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Make your app idea a reality with Ruby On Rails
Next
Download to read offline and view in fullscreen.

Share

ActiveWarehouse/ETL - BI & DW for Ruby/Rails

Download to read offline

Presentation delivered at the Singapore Ruby Brigade meetup 6-Jan-2010 (at hackerspace.sg). Discusses BI and DW in the Rails context, and test drives ActiveWarehouse and ActiveWarehouse/ETL with a "Cupcakes Inc" sample application.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

ActiveWarehouse/ETL - BI & DW for Ruby/Rails

  1. 1. NB: This presentation was delivered at the Singapore Ruby Brigade meetup 6-Jan-2010 (at hackerspace.sg)
  2. 2. BI & DW for Ruby/Rails “!???”
  3. 3. Why should we care about this enterprisey stuff? <ul><li>Have you heard a client ask for.. </li></ul><ul><ul><li>A “dashboard”? </li></ul></ul><ul><ul><li>Management reports? </li></ul></ul><ul><ul><li>Operational statistics? </li></ul></ul><ul><li>..in addition to the actual site? </li></ul>
  4. 4. Or maybe you want to pitch for the dashboard/BI projects themselves? ..using your rails skills of course BI Business Intelligence CPM Corporate Performance Mgmt BPM Business Performance Mgmt B&P Budgeting and Planning EPM Enterprise Performance Mgmt Dashboard Enterprise Dashboards
  5. 5. BI Basics No, BI is not (always) an oxymoron
  6. 6. BI = Business Feedback & Control Systems Keeping the doors open Uptime on the servers; alerts Infrastructure & Systems
  7. 7. BI = Business Feedback & Control Systems Keeping the doors open Optimising in the short term intra-day Focus on systems in isolation Need extra call centre staff on shift? Daily sales numbers? Infrastructure & Systems Operational Management
  8. 8. BI = Business Feedback & Control Systems Keeping the doors open Optimising in the short term intra-day Focus on systems in isolation Strategic performance monthly, quarterly, yearly Across all systems Profitability by product Utilisation and sales performance Infrastructure & Systems Operational Management Executive Management
  9. 9. Traditional Rails perspective.. e.g. NewRelic Custom AR reports Someone else’s problem (opportunity) Infrastructure & Systems Operational Management Executive Management
  10. 10. Someone Else’s Problem.. Your Rails Storefront App Fulfillment (maybe a third party) To report on sales fulfillment.. AR/AP/GL To report on revenue and profitability.. To report on sales revenue, actuals and forecast.. And don’t forget all those other systems.. CRM MRP FA
  11. 11. Who is “Someone Else”? The gigaohm network: “ 5 Free Business Intelligence Crunchers for Your 2010 Arsenal ”
  12. 13. ETL ODS Your Rails App Other Transactional Systems Data Sources DBoR, relational reporting BI & DW A copy of transaction data specifically structured for query and analysis Extract – Transform – Load Or, Extract – Load – Transform Or, Transform – Extract – Load (depending on the technology)
  13. 14. “cubes” Sales = $22 Customer ID Product ID Date ID … Customer dimension Date dimension Product dimension Fact categorisation “ Fact”
  14. 15. MOLAP, ROLAP, HOLAP MOLAP: proprietary format to optimize for analytical queries ROLAP: use relational database to mimic multi-dimensionality HOLAP: hybrid. Drive analytics from MOLAP, drill down to relational Star schema Snowflake
  15. 16. Why?? What’s wrong with.. select a.name, sum (b.amount) from products a join order_items b on a.id = b.product_id group by product_id Product.sum (:amount, :include => :orders, :group => ‘ product_id’) <ul><li>Every question needs it’s own query </li></ul><ul><li>Can’t predict all the questions in advance </li></ul><ul><li>Un-scalable grunt work </li></ul>
  16. 17. ActiveWarehouse ActiveWarehouse-ETL
  17. 18. ActiveWarehouse <ul><li>Rails plugin by Anthony Eden </li></ul><ul><li>ROLAP solution based on ActiveRecord </li></ul><ul><li>Features </li></ul><ul><ul><li>Generators for Facts, Dimensions, Cubes and Bridges </li></ul></ul><ul><ul><li>Supports calculated fields </li></ul></ul><ul><ul><li>View helpers for reports with drill down </li></ul></ul>
  18. 19. ActiveWarehouse-ETL <ul><li>Rails gem/plugin by Anthony Eden </li></ul><ul><li>DSL for extract – transform – load </li></ul><ul><li>Source/sink: file, db, xml, .. (extensible) </li></ul><ul><li>Features </li></ul><ul><ul><li>Pre/post processors </li></ul></ul><ul><ul><li>Transformations </li></ul></ul>
  19. 21. The Cupcakes Store Use Activewarehouse-etl to load seed data from csv to app db (mysql) 1 The Cupcakes BI Dashboard 2 Use Activewarehouse-etl to load dimension and fact data to the warehouse (mysql to mysql) 3 Use Activewarehouse to build a simple analytical “dashboard” and reporting tool Follow the documentation at http://github.com/tardate/cupcakesinc to see how this works (and try it yourself)
  20. 22. Product listing at Cupcakes Inc..
  21. 23. Customer listing at Cupcakes Inc..
  22. 24. Order listing at Cupcakes Inc..
  23. 25. Order detail at Cupcakes Inc..
  24. 26. Sales By Product AW Report
  25. 27. Sales By Product (drill to 2009)
  26. 28. Reasons to be Cheerful..
  27. 29. Language ETL processing, cube rules etc typically use custom languages (often archaic and limited) BI Suites It’s … ruby!
  28. 30. UI Customisation and Presentation Integration Web delivery typically very constrained. Often rely on strong integration with office software (Excel). Leads to “custom application development in Excel” syndrome. BI Suites It’s … ActionPack! Google maps mashups, social graph links. .. you get full UI control, as long as you have the development budget.
  29. 31. Speed of development Basic deployments can be very fast. But UI inflexibility can lead to either lots of time wasted trying to shoe-horn, or need to “reset customer expectations” BI Suites It’s … Ruby & Rails. Say no more ;-)
  30. 32. TCO Top-tier suites can come with a hefty $ tag. And prices are going up.. But some analysts are predicting 2010 to be the year BI gets FLOSS momentum (see gigaohm review of 5 well established alternatives) BI Suites It’s … Ruby & Rails. Say no more ;-) Trade-in software license costs for more development.
  31. 33. Caveats..
  32. 34. Native MOLAP Generally good support for database MOLAP features. Can be platform specific though – e.g. Microsoft MDX, SQL Server Analytical Services BI Suites A gap. No real support currently available. ActiveWarehouse uses relational model to “fake” MOLAP (ROLAP)
  33. 35. Performance Generally, all established analytical engines (and backing databases) have great performance track record. Huge scalability (millions of rows) BI Suites Unproven. ActiveWarehouse/ETL does not have many (public) proof points. Given that it is tied to AR performance, expect scalability could be an issue.
  34. 36. Take-aways ~ActiveWarehouse <ul><li>It’s an impressive codebase. When you get it working, it works well.. but </li></ul><ul><ul><li>Virtually no documentation! </li></ul></ul><ul><ul><li>No contemporary examples </li></ul></ul><ul><ul><li>Not under very active development </li></ul></ul><ul><ul><li>A “textbook” data warehouse implementation. May or may not be exactly what you want.. </li></ul></ul><ul><li>Remember: </li></ul><ul><ul><li>data is batched. Not realtime. </li></ul></ul><ul><ul><li>Rails 2.x : install the plugin (gem is 1.x) </li></ul></ul>3
  35. 37. Take-aways ~ ActiveWarehouse-ETL <ul><li>Neat tool. In addition to feeding AW: </li></ul><ul><ul><li>Generate and load seed/test data </li></ul></ul><ul><ul><li>Move data between systems </li></ul></ul><ul><li>But again, </li></ul><ul><ul><li>Poor documentation </li></ul></ul><ul><ul><li>When it fails, can do so silently (makes sure filename paths are delimited correctly for your platform!) </li></ul></ul>2
  36. 38. Take-aways ~ BI on Rails Solutions <ul><li>Plain AR </li></ul><ul><ul><li>just avoid the rabbit hole </li></ul></ul><ul><li>AR + ETL </li></ul><ul><ul><li>get all the data you need in one place </li></ul></ul><ul><li>AW+ETL </li></ul><ul><ul><li>traditional ROLAP, make Rails the focus of the BI effort </li></ul></ul><ul><li>Go the BI suite route </li></ul><ul><ul><li>When you need to adapt to many transactional systems at scale, and customer has the $$ </li></ul></ul><ul><ul><li>(Rails remains just for transactional apps) </li></ul></ul><ul><li>Or… (discussion point;-) </li></ul>1
  37. 39. Thank you! <ul><li>Questions? </li></ul>0
  38. 40. Some References <ul><li>ActiveWarehouse: http://github.com/aeden/activewarehouse </li></ul><ul><li>ActiveWarehouse-ETL: http://github.com/aeden/activewarehouse-etl </li></ul><ul><li>Cupcakes Inc sample site(s): http://github.com/tardate/cupcakesinc </li></ul><ul><li>Singapore Ruby Brigade (SRB): http://groups.google.com/group/singapore-rb </li></ul>
  • praharshinireddy

    May. 5, 2017
  • YigangSong

    Dec. 21, 2016
  • LongHuynhPhuoc

    Jun. 8, 2016
  • rubyotero

    Jan. 19, 2016
  • freefrancisco

    Nov. 18, 2014
  • thomaskobert

    Oct. 4, 2014
  • orendon

    Jun. 26, 2014
  • bestlong

    Jun. 19, 2014
  • jonatasdlp

    May. 9, 2014
  • kewin2010

    Oct. 24, 2013
  • jhigginbotham

    Jan. 25, 2013
  • pharrell

    Jan. 13, 2013
  • joelvh

    Dec. 28, 2012
  • sunilrkarkera

    Dec. 22, 2012
  • levmatta

    May. 15, 2012
  • fabriziomachado

    Jul. 5, 2011
  • campeterson

    Aug. 5, 2010
  • ikbear

    May. 17, 2010
  • munn

    May. 14, 2010
  • xds2000

    May. 13, 2010

Presentation delivered at the Singapore Ruby Brigade meetup 6-Jan-2010 (at hackerspace.sg). Discusses BI and DW in the Rails context, and test drives ActiveWarehouse and ActiveWarehouse/ETL with a "Cupcakes Inc" sample application.

Views

Total views

11,427

On Slideshare

0

From embeds

0

Number of embeds

754

Actions

Downloads

170

Shares

0

Comments

0

Likes

20

×