Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The AMB Data Warehouse
A Case Study
Mark Gschwind
VP, Business Information Architecture
AMB Property Corporation
mark@gsch...
Agenda
• Background on AMB’s business, its data and users  BI
requirements
• How we solved for it
-- Data Architecture
--...
10/8/2010 The AMB Data Warehouse - A Case Study
AMB’s Business
10/8/2010 The AMB Data Warehouse - A Case Study
AMB’s Business
10/8/2010 The AMB Data Warehouse - A Case Study
10/8/2010 The AMB Data Warehouse - A Case Study
10/8/2010 The AMB Data Warehouse - A Case Study
AMB’s Data
Data Objects
Buildings
…which are (usually) separated into units which…
…which have tenants with leases that us...
AMB’s Data
Source Systems
• Four ERP systems: CTI+MRI+Yardi+FAS and one forecasting
sys
• Coding conventions are not unifo...
AMB’s Users
Sr. Executives (3) and Managing Directors (10)
0 of 10 Managing Directors use the cubes
Need hi-level KPI’s th...
BI Requirements
Operational
Integrate all the source systems, conforming the coding conventions
Must support Regulatory Re...
10/8/2010 The AMB Data Warehouse - A Case Study
10/8/2010 The AMB Data Warehouse - A Case Study
-- Identify MRI INSERTs
SELECT * FROM [dbo].tbl_MRI_now
EXCEPT
SELECT * FR...
10/8/2010 The AMB Data Warehouse - A Case Study
CDC via Triggers
-- Code snippet of trigger
CREATE TRIGGER [dbo].[trg]
ON ...
10/8/2010 The AMB Data Warehouse - A Case Study
CDC based on update_datetime
10/8/2010 The AMB Data Warehouse - A Case Study
Alerting
• G/L Totals Checked
Nightly
• “Listener” processes
check for ina...
Hardware
10/8/2010 The AMB Data Warehouse - A Case Study
DW ETL and SQL Servers
SQL Server 2005 SP2
Compaq ProLiant BL460c...
Hardware
10/8/2010 The AMB Data Warehouse - A Case Study
Our failover solution uses PolyServe (HP)
Hardware
10/8/2010 The AMB Data Warehouse - A Case Study
Our failover solution uses PolyServe (HP)
Hardware
10/8/2010 The AMB Data Warehouse - A Case Study
Analysis Services Server
AS 2008 (SP1)
Compaq ProLiant BL460c G1
...
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
What Is Master Data?
Lists of data that are shared ...
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
10/8/2010 The AMB Data Warehouse - A Case Study
10/8/2010 The AMB Data Warehouse - A Case Study
Master Data Management
10/8/2010 The AMB Data Warehouse - A Case Study
EDM (MDS) Allows us to
• Get a single version of th...
Data Quality
Demo of MyData
10/8/2010 The AMB Data Warehouse - A Case Study
Reporting and Analysis - BOXI
10/8/2010 The AMB Data Warehouse - A Case Study
Data Quality
Demo of Cubes
10/8/2010 The AMB Data Warehouse - A Case Study
Analysis – Reasons for OLAP Success
• Updated incrementally, so it can become part of a business process
• Active steering...
Generalizing
• Keep users in a sandbox to avoid bad queries,
but make it a good sandbox
• Hard part: finding the sweet spo...
Q&A
10/8/2010 The AMB Data Warehouse - A Case Study
Upcoming SlideShare
Loading in …5
×

The AMB Data Warehouse: A Case Study

7,842 views

Published on

In this presentation to BAADD (SF Bay Area), BI Consultant Mark Gschwind shows one of the leading analytic platforms in the Real Estate industy, AMB Property Corporations data warehouse. Mark gives the attendees a tour of the infrastructure, explaining the challenges faced and the ways he solved them. He discusses how he achieved near-real-time data latency that helped drive user adoption. He demos the cubes, and an innovative custom application called MyData that helped ensure data quality. The presentation is a good example of how one organization achieved success using BI.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

The AMB Data Warehouse: A Case Study

  1. 1. The AMB Data Warehouse A Case Study Mark Gschwind VP, Business Information Architecture AMB Property Corporation mark@gschwindconsulting.com 10/8/2010 The AMB Data Warehouse - A Case Study
  2. 2. Agenda • Background on AMB’s business, its data and users  BI requirements • How we solved for it -- Data Architecture -- CDC -- Master Data Management -- Data Quality -- Reporting -- Analysis • Generalizing, about what you can learn from us • Q&A 10/8/2010 The AMB Data Warehouse - A Case Study
  3. 3. 10/8/2010 The AMB Data Warehouse - A Case Study AMB’s Business
  4. 4. 10/8/2010 The AMB Data Warehouse - A Case Study AMB’s Business
  5. 5. 10/8/2010 The AMB Data Warehouse - A Case Study
  6. 6. 10/8/2010 The AMB Data Warehouse - A Case Study
  7. 7. 10/8/2010 The AMB Data Warehouse - A Case Study
  8. 8. AMB’s Data Data Objects Buildings …which are (usually) separated into units which… …which have tenants with leases that usually span 10 years We make money when… Tenants pay rent, or we develop a building to spec for a tenant. Changing over time: • Tenants come and go • Units change as walls get torn down and changed • Buildings transfer ownership from the REIT to funds • People managing the property change over time • Buildings change status from “Development” to “Operating” to “Sold” 10/8/2010 The AMB Data Warehouse - A Case Study
  9. 9. AMB’s Data Source Systems • Four ERP systems: CTI+MRI+Yardi+FAS and one forecasting sys • Coding conventions are not uniform • About 400K g/l transactions/month, peaking at qtr end • Leasing Activity much smaller, but no less important (4,100 units, about 3,000 tenants) • ERP’s do not have ESB messages/queues/brokers. In fact there is no accepted standard for an XML Namespace for Real Estate.  CDC impact, no MDM “hubs” possible • Limited reporting capabilities in ERP’s to the point that they cannot support reporting to US and Int’l regulatory agencies such as the SEC and IRS  SOX impact • They allow bad data to get entered  Data quality impact 10/8/2010 The AMB Data Warehouse - A Case Study
  10. 10. AMB’s Users Sr. Executives (3) and Managing Directors (10) 0 of 10 Managing Directors use the cubes Need hi-level KPI’s that clearly show alerts, w drill-down to an actionable level Need everything in one place – do not want to go to LMS+OLAP+BPC+BOXI Data is heavily massaged to allocate costs, split operational variance from FX variance, etc. Want to do “what if” analysis on interest rates, macro-economic variables, organizational changes impacting G&A They are under-served - still learning what these people want Regional Managers (30) Only 12 of 30 Operations RM’s use the cubes. Probably only 6 actively These people are trying to keep our buildings leased up, and actively finding new deal Require external data on market rents, market absorption, etc Need to see what is going to happen on the leasing side, less interested in reporting history Power Analysts (5) These people provide the Sr. Executives and Managing Directors their reports and analysis Do a lot of massaging in Excel for what-ifs, etc. Intimately familiar w the data Power Excel users building elaborate models for Sr. Executives, love OLAP Accounting and Operations Analysts (12) 19 of 20 Operations Analysts use the cube 12 of 12 Accounting Managers use the cube Mostly concerned with reporting history, and the mechanics of the forecast. But they deal w these things at a consolidated level. Need to see results no more than 30 minutes after data entry – as part of their process They are controlling the Master Data, and need these changes reflected w low latency as well Power Excel users Accounting and Operations Staff (170 cube users 250+ BOXI users) Includes first-line accounting which has been outsourced. Mostly concerned with reporting actual results at the most granular level of detail Need to see results no more than 30 minutes after data entry – as part of their process Big Excel users 10/8/2010 The AMB Data Warehouse - A Case Study
  11. 11. BI Requirements Operational Integrate all the source systems, conforming the coding conventions Must support Regulatory Reporting (SOX Compliant) – Systematic processes ensuring data quality – Source control (VSS) – Change control: Formal testing process w signoffs – Ability to re-create history the way it was reported High-availability / Low Latency / 24-6 support for global usage Analytic Allow fast, easy access to aggregate data, returning it to Excel Show trends going all the way back in history+30 yrs out in forecast Overall Consistent metadata, single version of the truth, cleanse data to enforce data quality 10/8/2010 The AMB Data Warehouse - A Case Study
  12. 12. 10/8/2010 The AMB Data Warehouse - A Case Study
  13. 13. 10/8/2010 The AMB Data Warehouse - A Case Study -- Identify MRI INSERTs SELECT * FROM [dbo].tbl_MRI_now EXCEPT SELECT * FROM [dbo].tbl_MRI_before UNION ALL -- Identify MRI DELETEs SELECT * FROM [dbo].tbl_MRI_before EXCEPT SELECT * FROM [dbo].tbl_MRI_now -- If there is an INSERT and DELETE, -- remove the DELETE and -- make the INSERT an UPDATE CDC via Table Compare
  14. 14. 10/8/2010 The AMB Data Warehouse - A Case Study CDC via Triggers -- Code snippet of trigger CREATE TRIGGER [dbo].[trg] ON [dbo].[FinancialsAccountDetail] FOR INSERT, UPDATE, DELETE AS DECLARE @ichange_type CHAR(1) IF EXISTS(SELECT * FROM inserted) IF EXISTS(SELECT * FROM deleted) -- --update = inserted and deleted tables both contain data -- BEGIN SET @ichange_type = 'U' SELECT * FROM inserted END -- U ELSE -- --insert = inserted contains data, deleted does not -- BEGIN SET @ichange_type = 'I' ELSE -- --delete = deleted contains data, inserted does not -- BEGIN SET @ichange_type = 'D'
  15. 15. 10/8/2010 The AMB Data Warehouse - A Case Study CDC based on update_datetime
  16. 16. 10/8/2010 The AMB Data Warehouse - A Case Study Alerting • G/L Totals Checked Nightly • “Listener” processes check for inactivity hourly • If totals do not tie, or inactivity is detected, emails go out.
  17. 17. Hardware 10/8/2010 The AMB Data Warehouse - A Case Study DW ETL and SQL Servers SQL Server 2005 SP2 Compaq ProLiant BL460c G1 OS: Windows Server 2003 Enterprise x64 CPU: 8 Xeons @3.00GHz RAM: 32 GB Storage: EMC SAN, about 100 GB for .mdf’s, 100 for .ldf’s
  18. 18. Hardware 10/8/2010 The AMB Data Warehouse - A Case Study Our failover solution uses PolyServe (HP)
  19. 19. Hardware 10/8/2010 The AMB Data Warehouse - A Case Study Our failover solution uses PolyServe (HP)
  20. 20. Hardware 10/8/2010 The AMB Data Warehouse - A Case Study Analysis Services Server AS 2008 (SP1) Compaq ProLiant BL460c G1 OS: Windows Server 2008 Enterprise x64 SP2 CPU: 8 Xeons @3.00GHz RAM: 24 GB Storage: EMC SAN, 2 cubes taking up 10 GB
  21. 21. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study What Is Master Data? Lists of data that are shared and used by several of the applications that make up the system. For example, a typical ERP system as a minimum will have a Customer Master, an Item Master, and an Account Master. AMB uses EDM, (predecessor to SQL 2008 R2 MDS) for its G/L account hierarchies. Good demo of MDS: http://www.youtube.com/watch?v=g1fZez5AJHY&feature=related
  22. 22. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study
  23. 23. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study
  24. 24. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study
  25. 25. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study
  26. 26. 10/8/2010 The AMB Data Warehouse - A Case Study
  27. 27. 10/8/2010 The AMB Data Warehouse - A Case Study
  28. 28. Master Data Management 10/8/2010 The AMB Data Warehouse - A Case Study EDM (MDS) Allows us to • Get a single version of the truth for our Master Data • Satisfy our SOX-compliance requirements… • Improve data quality by enforcing standards and workflows (used to use an .xls for this) • Improve operational, reporting consistency • Enable Users to Securely Manage Master Data with Web- based access • Provide an audit trail of changes for regulatory compliance • Flexibly add new data, hierarchies MDS Wish List • Versioning by hierarchy, not model
  29. 29. Data Quality Demo of MyData 10/8/2010 The AMB Data Warehouse - A Case Study
  30. 30. Reporting and Analysis - BOXI 10/8/2010 The AMB Data Warehouse - A Case Study
  31. 31. Data Quality Demo of Cubes 10/8/2010 The AMB Data Warehouse - A Case Study
  32. 32. Analysis – Reasons for OLAP Success • Updated incrementally, so it can become part of a business process • Active steering committees adjusting it as business processes change • Same data as BOXI (not re-ETL’ing) • Easy to use, especially with Excel users • Nice stream of upgrades, particularly w 2005 and 2008. • OLAP better suited to these calc’s:  Any-any currency, Budget vs Actual FX rates, Operating vs Ttl Variance  Occupancy %, Projection,  External Aggregate-level data  Traffic-lighted KPI’s • It’s a good sand box 10/8/2010 The AMB Data Warehouse - A Case Study
  33. 33. Generalizing • Keep users in a sandbox to avoid bad queries, but make it a good sandbox • Hard part: finding the sweet spot for analytics in your organization. But after you address it, move on the next one • Try to do statutory reporting out of the source system – or you are getting into a big headache that will take you away from analytics • Question requests that data be real-time 10/8/2010 The AMB Data Warehouse - A Case Study
  34. 34. Q&A 10/8/2010 The AMB Data Warehouse - A Case Study

×