Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automation First as Strategy for Data Warehouse Modernization


Published on

Data warehouse teams are under increasing pressure to prototype sooner, deploy solutions faster, create designs that more flexibly adapt as the business changes, and achieve better alignment with business goals.

Watch this recorded webcast to hear how data warehousing teams are getting the most out of their data warehouses by modernizing the tools and methods they use through an Automation First approach.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Automation First as Strategy for Data Warehouse Modernization

  1. 1. WEBINAR Automation First as a Strategy for Data Warehouse Modernization Achieving productivity and flexibility via modern tools and methods Philip Russom, Ph.D. September 19, 2019
  2. 2. SPONSOR 2
  3. 3. PHILIP RUSSOM Senior Research Director for Data Management, TDWI
  4. 4. AGENDA • Definitions – Data warehouse modernization – Data warehouse automation • Modernization Best Practices – Why is modernization important? – What are the strategies for it? – Automation First strategies • Use Cases for DW Mod/Automation – Benefits of Automation First • Recommendations Automation First for DW Mods @prussom, #TDWI, @WhereScape
  5. 5. DEFINITION OF DW Modernization • “Modern” means “up to date” or “recent” • We modernize to keep pace with evolving biz & tech requirements – To realign the DW with current business goals – To provision data for existing and future business use cases – To leverage new data platforms and data-driven tools – To adopt new data mgt best practices and to adjust DW teams & skills – To remediate limitations of scale, speed, functionality, agility • Synonyms for “data warehouse modernization” – Data warehouse augmentation, automation, optimization
  6. 6. MANIFESTATIONS OF Data Warehouse Modernization • Replatforming is the most hyped right now – Deploying new data platforms: cloud, Hadoop, NoSQL – Rip-and-Replace: New platform replaces old one – Augmentation: Old & new platforms coexist & integrate • Many warehouses need to be redesigned to… – Improve data models, data quality, metadata – Improve architecture for multi-platform & logical DWs • Modernizing the DW to support related disciplines – Advanced analytics – This is leading driver of DW mod – New data structures, sources. New biz use cases
  7. 7. Related disciplines need modernization, too • Analytics • Reporting • Self-service data • Data integration • Data quality • Metadata mgt
  8. 8. DEFINITION OF DW Automation • Data warehouse automation is a strategy for DW modernization – It’s mostly about modern tooling for DW design, development, admin • The point of data warehouse automation is… – To give data and operations professionals modern productivity, flexibility, innovative designs, and warehouse-to-biz alignment – To support agile, rapid prototyping, and collaborative methods • Data warehouse automation achieves these goals via… – Tools with high ease of use, graphical user interfaces, wizard driven – Modern pipelining, modeling, dataset creation, cloud support, etc. – Metadata-backed automation. Smart tool algorithms.
  9. 9. Automation First As a strategy for DW modernization • In the context of DW modernization, Automation First means… – Give priority or preference to automation – Project should start with tools for automation • Multiple DW mod strategies can be done in series – Automation First improves data schema and quality before migrating the data to a new platform – When you introduce a new data platform into a DW, that’s a “green field” opportunity to introduce new DW development methods
  10. 10. Why Automation First? • Because DW automation enables DW mod tasks – Modern data modeling and modern metadata mgt – Automatic documentation generation and update • Because DW automation fixes DW problems – Outmoded development methods and processes • With automation, agile typically becomes the norm – Limited productivity, flexibility, reuse, standards • Leading driver for adoption of DW automation • Because DW automation accelerates later DW mod tasks – Get a productivity boost that applies to whole mod project
  11. 11. Metadata is more relevant than ever • All data-driven action goes thru metadata – Browsing data, running query, inserting data, making transactions, updating records, modeling data, making virtual views, refreshing rpt, etc. • Still mission-critical for data-driven biz – Operations, analytics, compliance, discovery – Enables modern info capture, flow, processes • More data, more platforms, more apps… – Each needs solid metadata mgt
  12. 12. Why DW automation and modern metadata? • DW automation & metadata support common uses – Self-service, better modeling, logical DW… • Consolidate metadata silos into an automation tool – For single view, standards, governance • DW automation & metadata are codependent – You cannot automate without solid metadata – Automation tools help modernize metadata mgt – Metadata is the documentation that keeps a data warehouse from becoming a data swamp
  13. 13. • Business Modernization should be ultimate goal – Modernize to adapt to a changing market, economy, customers, competition, etc. • Ideally, upper management should set the goals – Communicate biz goals to whole organization – IT and Data Mgt teams must support biz goals • DW Automation assists with many business goals – Reduces time to biz use of data solutions – The pace of modern business demands this • DW Automation produces quick, aligned solutions – Else, dep’ts build their own ungoverned silos UPPER MANAGEMENT ROLE IN DW Modernization and Automation
  14. 14. • Obviously, data management professionals are required – Specialists in warehousing, integration, analytics, reporting. Data modeling, architecture, metadata • People & processes for data governance & stewardship – Align the DW with biz goals, compliance, and data standards, as you modernize and automate it – Adjust governance policies as you go • Affected parties must be part of the process – Miscellaneous data consumers and user constituencies • In some cases: partners, clients, customers • Create a multi-phase plan, not “big bang.” STAFFING and COORDINATING DW Modernization and Automation
  15. 15. Many of the top priorities for data management in 2019 that users selected in a recent survey are tool functions and practices associated with data warehouse modernization and automation. Market Demand for DW Mod/Automation
  16. 16. Users Surveyed say that it is: Very Important & Important to have Tools & Practices that enable DW Automation Source: TDWI 2019. Based on 190 Respondents. Market Demand for DW Mod/Automation
  17. 17. Recommendations • Know the many manifestations of DW modernization – Select the ones that match your biz/tech priorities – E.g., replatforming, augmentation, redesign, automation • Create a plan for your DW modernization project – New tooling usually comes first, data migration last – Coordinate with related teams: analytics, reporting… – Coordinate with affected parties: users, managers… • Consider modernizing your dev/admin tool set – E.g., tools for DW automation, metadata, quality, modeling • Kick off DW modernization with an Automation First strategy – New automation tools will boost the whole project’s productivity – Their additional functionality will aid many modernization tasks
  18. 18. MICHAEL TANTRUM Director of Alliances, WhereScape
  19. 19. Modernizing with Data Warehouse Automation Michael Tantrum Director of Alliances, WhereScape
  20. 20. WhereScape helps IT teams fast- track projects and deliver more through data infrastructure automation. GLOBAL REACH Portland, OR Asi a Auckland, NZ Europ e PROVE N 700+ customers From small organizations to large global enterprises. 20 years Of automation innovations for IT teams of all sizes. EXPERIENCE D 5x developer output By automating the routine steps that slow developers down. ROI
  21. 21. The Traditional Approach to Data Infrastructure Delivery Hand-Coding and Manual Processes • Lengthy, expensive and problematic projects • Lack of agility, standardization and best practices • Documentation and technical debt • Tribal knowledge
  22. 22. Automation Modernizes Data Warehousing  By automating data warehousing patterns, teams can deliver faster  Reduce complexity of new technologies and data platforms  Lessen risk, adopt best practices  Quickly evolve and agilely react to change  Deliver at less cost
  23. 23. Automation Leverages Data Warehousing Patterns
  24. 24. The Conventional Way • Over-complicated • Inefficient • Difficult to address change
  25. 25. A Better Way ✓ Metadata-driven ✓ Built-in methodologies ✓ Best practices, standards ✓ Documentation and lineage ✓ Full lifecycle management Simplified and Automated
  26. 26. An Automation First Approach with WhereScape  Modernization requires automation  Boost team productivity to fast-track delivery  Limit project risk and lower modernization initiative costs  Minimize complexity – migration to the cloud, Data Vault, new data platforms and sources, hybrid environments  Use automation to support DevOps and agile continuous integration and delivery
  27. 27. Thank You More information on Automation Follow us on LinkedIn or Twitter @wherescape
  28. 28. © Copyright 2019, WhereScape Software Ltd. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. About WhereScape WhereScape helps IT organizations of all sizes leverage automation to design, develop, deploy and operate data infrastructure faster. More than 700 customers worldwide rely on WhereScape automation to eliminate hand-coding and other repetitive tasks to deliver data warehouses, vaults, lakes and marts in days or weeks rather than months or years. WhereScape has global operations in the USA, UK, Singapore and New Zealand.
  29. 29. CONTACT INFORMATION If you have further questions or comments: Philip Russom, TDWI Michael Tantrum, WhereScape