• Save
Spreadmart To Data Mart BISIG Presentation
Upcoming SlideShare
Loading in...5

Spreadmart To Data Mart BISIG Presentation



Presentation at the North Central BI Special Interest Group (BISIG) going over a case study of converting an Excel Spreadmart solution to a SSAS data mart solution

Presentation at the North Central BI Special Interest Group (BISIG) going over a case study of converting an Excel Spreadmart solution to a SSAS data mart solution



Total Views
Views on SlideShare
Embed Views



1 Embed 4

http://www.slideshare.net 4



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Spreadmart To Data Mart BISIG Presentation Spreadmart To Data Mart BISIG Presentation Presentation Transcript

  • Spreadmart to Data Mart Conversion Joe Beeck – GfK Custom Research Dan English Principal Consultant Principal Developer/Team Lead dane@magenic.com Joe.beeck@gfk.com
  • Who are we? – Dan and Joe Dan English Joe Beeck http://denglishbi.spaces.live.com/ • • Developing with Microsoft technologies for over Principal Developer/Team Lead at GfK Custom 10 years Research North America. • • Over 5 years experience with Data Warehousing Has been working with Microsoft technologies for and Business Intelligence over 10 years • • Experienced in ETL and Analysis Services Current role primarily focuses on working with development, requirements gathering and data business users to identify requirements and modeling managing the project team • • Microsoft Certified IT Professional (MCITP) and Microsoft Certified Solution Developer (MCSD) Microsoft Certified Technology Specialist (MCTS)
  • Who is Magenic?  Founded in 1995, Magenic is a technical consulting firm focused exclusively on Microsoft technologies and has designed and delivered more than 500 Microsoft-based applications  Headquartered in Minneapolis, with offices in Chicago, Boston, Atlanta and San Francisco  2005 Microsoft Partner of the Year, Custom Development Solutions – Technical Innovation  2007 Microsoft Partner of the Year Finalist, Data Management  Microsoft Gold Certified Partner and National Systems Integrator  40 Enterprise Data Services (EDS) consultants
  • Who is GfK? Founded in1934 and headquartered in Nuremberg, Germany Size • $1.43B + in annual revenue • 9,300+ full-time employees (USA – 700+) • 2nd largest custom research company in North America • 2nd largest custom research company worldwide Full Service • Knowledge and resources to meet any client need • Global databases and custom research expertise • Qualitative and quantitative practices Global Coverage • 130 offices located in more than 70 countries
  • Today‟s Agenda • Market Research Overview • The Original Spreadmart Solution • What is the BI Maturity Model? • Spreadmarts vs. Data Marts • Case Study and Demo • Lessons Learned • Questions?
  • Market Research Overview
  • Why Do Market Research? To reduce the risk of decision making: • What hidden opportunities exist in the current market? • To whom should we target our advertising? • What product should we market next? • Should we change the formula of an existing product?
  • Case Study – Reversing Category Decline Industry: • Dairy Industry Business Problem: • How to stop and reverse declining dairy sales Background: • Dairy sales slipping • Negative publicity about dietary fats from dairy • Fewer servings per day recommended • The client, Dairy Trade Association, needed to understand consumer attitudes toward dairy products to direct strategy
  • Case Study – Reversing Category Decline Approach • Attitudinal segmentation • Identify how narrowly or broadly people view dairy • Understand/quantify the consumer perception that dairy is unhealthy • Measure consumers attitudes on: • Dairy category overall • Individual products • Health and lifestyle issues • Cross this attitudinal information with consumption patterns, lifestyle habits, and demographics • Combine and model results to create in-depth profiles of the respondent
  • Case Study – Reversing Category Decline Results • Major recommendation: It‟s about milk! Milk should be at the core of the communication message. • Results: Very successful campaign to reverse the trend and make milk cool again. • Milk sales rose • Public perception changed
  • Ways to Collect Data Type Situation • A moderate number of questions Telephone • A lot of people • No visual or sensory stimuli needed • A few questions – simple • A lot of people Mail • Few security concerns • Visual and/or sensory stimuli • Fewer questions • Simple to complex Online • A lot of people for relatively little money • Visual stimuli • More questions In-person • More complex • Visual and/or sensory stimuli
  • Types of Questions Closed End Open End Provides choice for the respondent. Respondent answers in own words; no responses for respondent to choose from. Good for “What do you do, Example: What, if anything, do where is it done, who uses it” you like about the product? type questions Please clarify. Should generally be used when Example: Why do you say that all (or most) of the possible you [respondent‟s answer to responses can be determined question 3]? beforehand.
  • Market Research Process Define Survey and Measures Conduct Survey Collect Data Process and Clean Data Report Results
  • The Original Spreadmart Solution
  • Business Requirements 300 000 survey responses per year – 25 000 per month 12 report templates 1500 reports generated per month Ability to generate historical reports 24-hour turnaround after receipt of data Perfect data
  • Speadmart Solution Run PERL Run VBA Pre- Use Adobe Manually script to Load and script to “stitch” aggregate Distiller to post files to validate data generate data and convert SharePoint together using 1,500 individual split into 17 everything to according to PostScript tabulation separate PDF a predefined reports software files (250 at Excel tabs documents file structure according to a time) the hierarchy 16 Hours 16 Hours 30 Hours 1 Hour 1 Hour 1 Hour
  • Spreadmart Issues Data had become decentralized over the course of 3+ years Excel became unusable due to increasing data volume and memory errors Unable to run historical reports without returning to saved versions of Excel documents Prone to error because of so many manual updates, lack of versioning control, and lack of integrity-checking software Custom updates increased reliance on individual developers. Transfer of knowledge became very difficult System/process became so slow that even small issues would cause delays in delivery to the client Solution had become fragmented and new report requests were no longer cost efficient Errors and delays were beginning to put contract in jeopardy
  • There must be a better solution…
  • BI Maturity Model – where are you at? STRUCTURE: Mgmt Reports Spreadsheets Data Marts Data Warehouses Enterprise DW BI Services System Individual Department Division Enterprise Inter-Enterprise SCOPE: By Wayne Eckerson, Director of Research, TDWI
  • Spreadmart BI – Infant (2nd) Stage Are the users What happens when Did they extract all How long does it extracting and the person responsible of the necessary take to extract reporting on the for the report goes on data to allow the data and how right data? vacation or is sick or management to ask clean is it once it leaves the company? further questions? is extracted? MS Access MS Excel MS PowerPoint Business Users Do they have enough What logic is Source Data data collected to being applied and perform yearly Is all of the data is this common comparisons or available in the logic within the trends over time? source system? organization?
  • Data Mart BI – Child (3rd) Stage OLAP Engine Data Mart Source Data Business Users
  • Spreadmart vs. Data Mart BI Spreadmart Data Mart • High end-user control • Shared/consistent view of data • Easy to create and use • Centralized logic • Can be pieced Pros • Highly interactive (slice-and- together Pros dice) • Highly customizable for • Secured the intended audience • Very Flexible • Low cost solution • Extremely Fast response time • Inconsistent view of the data • • No centralized logic Takes time to generate Cons • • Typically no security applied Less end-user control Cons • • Silos of data throughout Costs more to develop organization • Could potentially introduce new tools (training)
  • Spreadmart to Data Mart Case Study Spreadmart • Excel file report system • Lots of embedded business logic and conditional formatting • Generated over 1500+ files (most contained multiple reports) with macro • Process took approximately 30 hours to run • Initial Excel file was created and tested over a 6 month time period • If there were any data issues or report creation errors process had to be re-run • Not easy to implement additional change requests Data Mart • Star schema database engine designed • Analysis Service database created with centralized logic • Reporting Service reports created and data driven subscription setup • Generated same reports in approximately 30 minutes • Entire database along with reports was created and tested in 2 month time frame • Database and reporting structure extremely flexible to change requests
  • Data Mart Case Study
  • Reporting Services with SSAS data SSAS Designer within SSRS • Keep measures in the columns • Flattened hierarchy information • Very nice drag-n-drop feel and parameter setup MDX Query within SSAS data source • No drag-n-drop designer • Custom MDX scripting capability SSIS data source • OLE DB Source or DataReader (ADO.Net) • Ability to customize output • Join multiple datasets SQL Server Stored Procedure • Similar capabilities like SSIS • Custom formatting and data merging logic within stored procedures • OPENQUERY commands with linked server (SSAS)
  • Data Mart Conversion Steps 1. Received the business requirements for the deliverables 2. Reviewed the reporting deliverables, data files, and calculations required for the reports 3. Created the star schema database model 4. Created the ETL process to import the data file and load the star schema 5. Created the Analysis Service database 1. Setup the necessary dimensions, attributes, hierarchies 2. Produced the cube with necessary measures, measure groups, and calculations 6. Setup the linked server within SQL Server to access the SSAS database 7. Created the stored procedures to be used by Reporting Services 8. Created the Reporting Service reports 9. QA reports and all data associated with them 10.Setup data driven subscription to generate all of the reports to be delivered to the client
  • SSAS data to SSRS Demo DEMO
  • Lessons Learned The client needs to understand how their hierarchical data is applied ( re-casted each month or applied using type 2 dimension ) The benefits of the future BI solution need to be emphasized throughout the project Automate, Automate, Automate Stick to your process Business users are „key‟ – keep them involved throughout the process and use them for Q&A and validation Data is never as clean as you would expect – „trust but verify‟ Nothing is ever as „easy‟ as you think – even rounding can cause issues Document and comment on all processes with reasons why
  • Resources Microsoft BI Site http://www.microsoft.com/bi/ SharePoint BI Features Introduction http://office.microsoft.com/en-us/sharepointserver/HA100872181033.aspx PerformancePoint Home Site http://www.microsoft.com/business/performancepoint/default.aspx PerformancePoint Developer Portal http://msdn.microsoft.com/en-us/office/bb660518.aspx Channel9 MSDN BI Screencasts http://channel9.msdn.com/Showforum.aspx?forumid=38&tagid=277 SQL Server 2008 Home Site http://www.microsoft.com/sqlserver/2008/en/us/default.aspx Microsoft Virtual Labs (TechNet and MSDN) http://www.microsoft.com/events/vlabs/default.mspx Magenic Blogs http://blog.magenic.com/blogs
  • Source Information BI Maturity Model – http://www.dmreview.com/issues/20041101/1012391-1.html or http://www.tdwi.org/publications/display.aspx?ID=7199 Dan‟s Blog postings – Using Reporting Services (SSRS) with SSAS data and SSAS MDX Round = Banker‟s Rounding DateTool - http://www.sqlbi.eu/datetool.aspx and http://sqlblog.com/blogs/marco_russo/archive/2007/09/02/datetool-dimension-an-alternative- time-intelligence-implementation.aspx
  • Contact Information – Thank You! Dan English - dane@magenic.com Dan‟s BI Blog - http://denglishbi.spaces.live.com Dan‟s Videos - http://www.youtube.com/user/denglishbi or http://video.msn.com/video.aspx?mkt=en-us&user=- 3657354010876223112 Magenic - info@magenic.com Joe Beeck - Joe.beeck@gfk.com