• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
White Paper - Data Warehouse Project Management
 

White Paper - Data Warehouse Project Management

on

  • 6,346 views

 

Statistics

Views

Total Views
6,346
Views on SlideShare
6,346
Embed Views
0

Actions

Likes
3
Downloads
211
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Great approach.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    White Paper - Data Warehouse Project Management White Paper - Data Warehouse Project Management Document Transcript

    • Data Management & Warehousing WHITE PAPER Data Warehouse Project Management DAVID M WALKER Version: 1.0 Date: 14/10/2008 Data Management & Warehousing 138 Finchampstead Road, Wokingham, Berkshire, RG41 2NU, United Kingdom http://www.datamgmt.com
    • White Paper - Data Warehouse Project ManagementTable of ContentsTable of Contents ...................................................................................................................... 2!Synopsis .................................................................................................................................... 3!Intended Audience .................................................................................................................... 3!About Data Management & Warehousing ................................................................................. 3!Introduction................................................................................................................................ 4!Common Problems.................................................................................................................... 5! Setting the wrong expectations ............................................................................................. 5! Managing the Technical Architecture.................................................................................... 7! Resource Turnover ............................................................................................................... 8!The building blocks of a project control ..................................................................................... 9! Phases .................................................................................................................................. 9! Milestones ........................................................................................................................... 10! Activities .............................................................................................................................. 10! Tasks................................................................................................................................... 10! Issues.................................................................................................................................. 11! Enhancements .................................................................................................................... 11! Test cases........................................................................................................................... 12! Defects ................................................................................................................................ 12! Risks ................................................................................................................................... 12!The Event Horizon................................................................................................................... 14! Short Term .......................................................................................................................... 14! Medium Term ...................................................................................................................... 14! Long Term........................................................................................................................... 14!Tools and Technology ............................................................................................................. 15! Project Management Software............................................................................................ 15! Ticketing.............................................................................................................................. 15! Version Control ................................................................................................................... 16! Wiki ..................................................................................................................................... 17!Methodologies ......................................................................................................................... 18!Project Leadership .................................................................................................................. 19! The Executive Sponsor ....................................................................................................... 19! Project Sponsor................................................................................................................... 19! Project Manager.................................................................................................................. 20! Technical Architect.............................................................................................................. 20! Senior Business Analyst ..................................................................................................... 21! Leadership Style ................................................................................................................. 21! Organisational Learning ...................................................................................................... 22! Team Rotation..................................................................................................................... 23!Estimating and Effort ............................................................................................................... 24!Tips and Tricks ........................................................................................................................ 26!Summary ................................................................................................................................. 28!Appendix 1 – Governance Procedures ................................................................................... 29! Change Management Processes........................................................................................ 29! Data Warehouse Processes ............................................................................................... 30! Data Management Processes............................................................................................. 31!Appendix 2 – Project Services ................................................................................................ 32!Copyright ................................................................................................................................. 32! © 2008 Data Management & Warehousing Page 2
    • White Paper - Data Warehouse Project ManagementSynopsisData warehouse projects pose a specific set of challenges for the project manager. Whilstmost IT projects are a development to support a well defined pattern of work a datawarehouse is, by design, there to support users asking ad hoc questions of the data availableto the business. It is also a project that will have more interfaces and more change than anyother system within the organisation.Projects often have poorly set expectations in terms of timescales; the likely return oninvestment, the vendors’ promises for tools or the expectations set between the business andIT within an organisation. They also have large technical architectures and resourcing issuesthat need to be handled.This document will outline the building blocks of good project control including the definition ofphases, milestones, activities, tasks, issues, enhancements, test cases, defects and risks andwill discuss how they can be managed, and when, using an event horizon, the projectmanager can expect to get information.To help manage these building blocks this paper will look at the types of tools and technologythat are available and how they can be used to assist the project manager. It also looks athow these tools fit into methodologies.The final section of the paper has looked at how effective project leadership and estimatingcan improve the chances of success for a project. This includes understanding the roles ofthe executive sponsor, project manager, technical architect and senior business analyst alongwith the use of different leadership styles, organisational learning and team rotation.Intended AudienceReader Recommended ReadingExecutive Entire DocumentBusiness Users SynopsisIT Management Entire DocumentIT Strategy SynopsisIT Project Management Entire DocumentIT Developers SynopsisAbout Data Management & WarehousingData Management & Warehousing is a specialist consultancy in data warehousing, based inWokingham, Berkshire in the United Kingdom. Founded in 1995 by David M Walker, ourconsultants have worked for major corporations around the world including the US, Europe,Africa and the Middle East. Our clients are invariably large organisations with a pressing needfor business intelligence. We have worked in many industry sectors but have specialists inTelco’s, manufacturing, retail, financial and transport as well as technical expertise in many ofthe leading technologies.For further information visit our website at: http://www.datamgmt.com © 2008 Data Management & Warehousing Page 3
    • White Paper - Data Warehouse Project ManagementIntroductionData warehousing projects are notorious for both being delivered late and over budget. Whyis this and what can be done to prevent it?The common problems that affect the data warehouse can be grouped by the expectationsthat are set, the technology used and the management of resources. There is also a need todefine the components of project control and to develop understanding of the level to whichthese can be planned and forecast.With this understanding it is possible to look at tools, technologies and methodologies thatcan assist the project manager to control, support and deliver a solution.Finally this paper looks at the impact of project leadership techniques and the effect that goodestimate efforts can have on the delivery of a project.This paper does not set out to be an exhaustive manual for the management of a datawarehouse project, rather a guide to help project managers understand the differencebetween data warehousing projects and others that they may have been responsible. It alsosuggests some things that should be considered when managing such projects. © 2008 Data Management & Warehousing Page 4
    • White Paper - Data Warehouse Project ManagementCommon ProblemsData warehouse projects often suffer from poorly set expectations, problems with thetechnical architecture and resource turnover. These common categories have multiple andvaried underlying causes: Setting the wrong expectations The failure to set the right expectations of a data warehouse is part of every project and the reason that they are often perceived not to deliver. This failure to set expectations occurs at many levels: • Timescales For most large organisations a data warehouse project will have many phases, take years to build and have to be supported for many more years afterwards. Development and production phases will overlap and once in production the environment will be subject to massive amounts of change. When a Data Warehouse project is conceived it will not be discussed in these terms but as just another project with a finite timescale and a simple set of phases. • Return on Investment The business case generated to justify a data warehouse project will always make a number of promises about return on investment, which are deemed necessary to get the budget. However, the real benefit will either come from an unexpected quarter (e.g. a fraud being discovered or the ability to drop a costly product range) or be from intangible benefits (e.g. business analysts that stop creating their own small database and spreadsheet applications and start analysing the business and delivering indirect business benefit). Business cases should be looked upon as a reason for starting rather than a defined return on investment. The deliverable, and therefore the return on investment should be measured in terms of what benefit was derived rather than against some planned benefit. Some phases will deliver more, and some less than expected. • Vendor set expectations of tools Data Warehousing and Business Intelligence tool are purchased with an expectation of productivity gains and reduced total cost of ownership however the reality is often more complex: o One reporting tool will not be sufficient for all your reporting needs. o An ETL tool provides a well-defined development environment but often does not provide the productivity benefits promised and requires more expensive, specialist resources. o It is often difficult to integrate the tools in the advertised way. o Changing from one tool to another often does not fix the underlying issue, or does so only at significant cost. o Technology sets leapfrog one another; a unique feature in one tool will often be in another with the next release. © 2008 Data Management & Warehousing Page 5
    • White Paper - Data Warehouse Project Management • IT set expectations to the business IT are often guilty of being over-optimistic about the point at which the benefit will occur: o The initial phases of a project are about understanding and infrastructure. It is only when these are in place that the business facing solution can be deployed. o Quick-win solutions, whilst valuable in the short-term, have a long-term cost that has to be accounted for. This is because there is both an on- going management cost and a de-commissioning cost that the business will have to pay later. Quick-win solutions often also compromise data quality that in turn damages user confidence in the system o The final solution can never be reconciled exactly against the current solution, this is because there will be unknown bugs in the currently delivered solution, new sources of data and different treatments of that data. It should, however, be possible to explain all the differences. o The testing and issue resolution of a system will take longer than expected. • Business set expectations to IT The business will often point to IT for its failings but it too will fail to manage expectations. Commonly: o The business will not invest the time required to deliver requirements and analysis. o The data quality will be much worse than the business is prepared to acknowledge and the business will be slow to respond to the need to change working practices to improve the data quality. o The business will not spend enough time or dedicate enough resource to testing. o The business will not spend enough time or dedicate enough resource to training. o There will be phases of the project where they lose focus as other business critical events take place. o When the time comes to de-commission existing solutions and tactical solutions built along the way the business will be unwilling or unable to do so, often fearing that some piece of data has been overlooked.© 2008 Data Management & Warehousing Page 6
    • White Paper - Data Warehouse Project Management Managing the Technical Architecture The technical architecture of a data warehouse is the framework on which the solution is delivered. Data Warehouses are often described as ‘complex’ without further explanation. It is not that the process itself is difficult (copy data from the source systems, format it and write a report on it), nor does it need unusually complex technology to deliver. Instead the complexity comes from the scale of the solution. A reasonable sized data warehouse will have thousands of individual ETL mappings and hundreds of reports all serving a large community of users quickly. In order to do this it is important to observe the following principle: Keep the architecture simple Whilst this may seem obvious, simplicity is hard to do and has to be regularly worked on. Keeping things simple requires a conscientious decision. Some common examples include: o Small is beautiful. Creating more simple ETL mappings rather than large single blocks of code makes change and maintenance easier. Have a larger number of smaller data marts rather than one single ‘super- mart’ which results in complex change management. Have multiple reports for different people instead of one that tries to satisfy everyone. A change to meet a requirement from one user will undoubtedly leave another user unhappy. o Have discreet ‘componentised’ functionality. Use the right tool for the job. For example don’t store data in the ETL tool, don’t use a reporting tool to perform ETL, extract data for downstream sources in a consistent way, etc. o Ensure that the data load process is complete and auditable. It is common for data warehouses to allow direct updates to the database by DBAs etc to maintain reference data. This should be prohibited and an 1 application provided to the business for them to perform this functionality. o Minimise components. There is often a temptation to make tweaks to the technical architecture to have a larger number of ‘best of breed’ tools to accommodate certain requirements. In practice this increases the maintenance costs and support skills required whereas the existing toolset can often provide a sufficient solution. o Keep it clean. A data warehouse whose data is not clean quickly falls into disrepute and becomes unused. The technical architecture has to support data quality monitoring and data cleansing and there must also be processes to deliver change in the source system.1 This may seem counter-intuitive, how can writing a whole separate application to maintain data be simpler than justwriting the SQL? The reason for this requirement is that there is no audit trail and therefore no way of knowing whatupdates have been done. If instead a business user has to maintain the data via an application and this is loaded intothe data warehouse via ETL then the business takes responsibility for maintaining the data and there is an audit trailof where the data change came from. Writing SQL also creates a large number of small, ad-hoc SQL jobs. Thistakes time and sooner or later someone will make a mistake with the SQL that may take a lot of work to correct. © 2008 Data Management & Warehousing Page 7
    • White Paper - Data Warehouse Project Management o Tactical Solutions. Every project has them and there is often a very good case for them but they always cost more to remove than to create and often de-rail the ‘keep it simple’ principle. If tactical solutions become necessary then they should have an implementation, an expected lifetime, maintenance cost for the lifetime, a lifetime-overrun cost and a decommissioning cost. When presented with these costs tactical solutions often look a lot less appealing. o Have an active de-commissioning policy. Data Warehouses have a lot of code that needs to be maintained and takes time to run. If database entities are no longer used then de-commission them and the ETL that feeds them. The same is true for a report that has been replaced or just fallen into disuse. Resource Turnover Data Warehouses are long-term projects and are likely to engage internal and external staff for several years. For a variety of reasons staff will come and go during that time. It is also possible for the new arrivals to bring new ideas and ways to improve what is being done. Whilst this is to be welcomed it also presents a very specific and potentially expensive risk. This often manifests itself when new members of staff come in and say ‘I wouldn’t have done it this way – let’s start doing it another way’ For whatever reason a significant change in direction most often results in one of two outcomes; expensive re-work to keep a consistent architecture or a complex architecture that requires more 2 support and documenting the route to the current point can avoid such re-work. In addition ensure that there is sufficient briefing material available so that new starters can get up to speed quickly and before launching into the ‘I wouldn’t have done it this way’ discussions.2 Key Design Decisions – a template for documenting critical decisions in the design process. See the ‘DataWarehouse Documentation Roadmap’ white paper from Data Management & Warehousing © 2008 Data Management & Warehousing Page 8
    • White Paper - Data Warehouse Project ManagementThe building blocks of a project controlThe following section offers a description of the components of the project control (the projectplan in conjunction with project risks, issues and defects). Whilst all the terms will be familiarto readers they are described here to ensure consistent use and discuss the interactionbetween them. Phases The phase of the program is a set of work with a defined outcome that delivers benefit to the project as a whole. Note that there should be phases of the project that do not deliver direct business benefit but are required for the project to succeed. It would be common for a data warehouse project to have the following initial phases: • Phase 1: Mobilisation (2 weeks). This short phase at the start of a project aims to get everything ready for the team to begin. It includes getting building passes, office space, network access, systems permissions and other basic structures in place. Too often 3 there is a perceived need to see everyone on site and working on day one. In reality this normally wastes a significant amount of resource that will be required in the later phases of the project. • Phase 2: Technical Architecture & Initial Business Requirements (4-8 weeks). This second phase is about putting a sound basic infrastructure in place. The technical architecture provides a solid platform on which to build whilst the initial business requirements are used to develop a list of priorities for development. These also provide the basis for developing the initial resourcing plans for enlarging the team. • Phase 3: Data Warehouse Infrastructure Deployment (4-8 weeks). With this phase sufficient hardware and software is deployed and the initial data warehouse logical data model created and a plan for the subsequent phases of data mart deployment is developed. • Subsequent Phases: Data Mart Delivery (3-6 months each). From this point on the iterative development of data marts and reporting solutions can begin. Each phase represents a deliverable of direct business benefit. Phases can overlap but only in a controlled way such as not to cause resourcing or technical conflict between phases. The above outline means that whilst the speed of delivery gets faster later in the project the initial delivery will be six to nine months from inception. 4 Phases are often driven by an enhancement packaging process that groups together a series of requirements and enhancements into a phase scope definition.3 This is particularly common in system integrator run projects where the client often says “How big is your team andwhen will they arrive?” because the client want to see movement on the project.4 See Appendix 1 for a description of governance processes for a data warehouse environment © 2008 Data Management & Warehousing Page 9
    • White Paper - Data Warehouse Project ManagementMilestonesMilestones are the checkpoints within a phase of the project. For example each datamart phase may have milestones that indicate the completion of RequirementsGathering, Analysis, Design, Build, Test, Deployment, Training, etc.ActivitiesActivities describe what needs to be done to complete a specific milestone. Thereforethe milestone ‘Analysis Complete’ may have activities such as ‘Identify PotentialSource Systems’, ‘Perform Data Quality Analysis’, ‘Perform Source System Analysis’,etc. These are normally carried out by a team or group of people and normally haveestimated elapsed durations associated with them. Whilst some activities aredependent on others many will have dependencies at a lower level. For example whilstthe activity ‘Indentify Potential Source Systems’ is going on there may be no reasonwhy the ‘Perform Data Quality Analysis’ for the first identified system cannot start.It is often difficult to represent this in a project management tool and common forproject managers to reflect the situation as a number of activities of fixed duration thatare dependent on the completion of one before the next starts, or not to have anydependencies between activities. Both methods cause the plan to slip.TasksTasks are the specific items of work to be carried out by an individual. It is importantthat a named individual is responsible for the task, not a group or a team. Where a taskis not yet assigned to an individual it should be assigned to the team leader so that theyare aware that they have to (re-)assign it on to someone to actually do the work.It should also be noted that tasks can have sub-tasks, for example the task ‘DocumentTechnical Architecture’ may require sub-tasks of ‘Document Database Architecture’and ‘Document Hardware Architecture’. All three of these tasks should also have anindividual owner who may or may not be the same person as the person responsiblefor bringing the whole document together.Tasks also have real dependencies on other tasks being completed. Individuals canwork very effectively if they have a bank of outstanding tasks, each with a priority and ifthey cannot work on one task can move to the next one quickly and easily. Issues mayhold up tasks.© 2008 Data Management & Warehousing Page 10
    • White Paper - Data Warehouse Project Management Issues An issue is something that affects the progress of the project. Issues can occur at a project level or affecting an individual task. For example: • A team member going off unexpectedly on long-term sick leave is a project level issue that needs to be resolved. • The inability to get a username and password for an individual system that prevents someone completing a source systems analysis is a task level issue. Issues introduce delays in the project and so it is important to see the impact of issues as soon as possible. • Project level issues should be driven down to task level as quickly as possible. Using the example above of long term sickness the process of driving it down to task level would be to re-assign tasks that belonged to that individual to others. • Task level issues often mean that the person responsible moves onto the next task in the task bank and can come back to the other task when the issue is resolved. Mistakes and consequently issues are inevitable on a project. As such active management and an environment where mistakes are accepted, learnt from and documented is to be favoured over one in which people try to hide their mistakes. An understanding of the types of issues over the lifecycle of the project is the first step towards improving both the quality and speed of delivery. An Issue Management 5 Process should manage issues Enhancements An enhancement is a change to the system with the aim of improving it. In a development such as a data warehouse where new requirements are continuously being developed it usual that an enhancement is captured, considered and if appropriate worked into the requirements documentation for inclusion in a subsequent phase via the enhancement packaging process. Not all enhancements need to go through the requirements process. Some, such as an improved navigation paradigm for the user interface, may be considered and passed directly as a task to the person who can affect the change. This rapid response process is very useful if there are separate teams responsible for issues and maintenance as well as the main development teams.5 See Appendix 1 for a description of governance processes for a data warehouse environment © 2008 Data Management & Warehousing Page 11
    • White Paper - Data Warehouse Project Management Test cases Test cases are a special type of task associated with testing. A test case is a set of tasks supported by scripts and routines that help automate and manage the testing of software. Every piece of code developed should have an associated test case. Where an issue is found with the code it becomes a defect that has to be rectified. A test case may reveal many defects, or none and test cases may have to be repeated a number of times until the code is defect free. In some environments it is useful to consider not only code but also the review of documents as test cases and the issues raised from the review as defects. This allows a separation in the reporting between tasks originally associated with the plan and work carried out because of the incompleteness of the deliverable (i.e. defects). The test case for a document would simply be to review it. As with issues above, mistakes are inevitable in the development process and therefore should be accepted and documented as part of improving the process rather than being hidden away. Defects Defects are a special type of issue associated with testing. A defect is a material error with a deliverable. In the case of code this may be as a result of an undelivered but promised requirement, a misinterpretation of the requirements, unexpected results (common when testing boundary conditions and exceptional usage) etc. A defect should be resolved before the code is deployed (even if the resolution is to say that the functionality will be delivered in a subsequent phase). This can also be applied to review comments for documentation that are, in effect, defects in the deliverable. Each defect should be reviewed and changes applied where appropriate before the defect is closed. Risks There may be external circumstances or events that cannot occur for the project to be successful. If you believe such an event is likely to happen, then it would be a risk. Identifying something as a risk increases its visibility, and allows a proactive risk 6 management plan to be put into place. If an event is within control of the project team, such as having testing complete by a certain date, then it is not a risk. If an event has a one hundred percent chance of occurring, then it is not a risk, since there is no "likelihood" or risk involved but it is an issue. Examples of risks might be a "Re-organization may result in key people being reassigned," or "The new hardware may not be able handle the expected volume." Risks are managed by developing mitigations or ways in which to avoid the risk or prevent it from becoming a reality.6 http://www.mariosalexandrou.com/definition/risk.asp © 2008 Data Management & Warehousing Page 12
    • White Paper - Data Warehouse Project Management A risk can be quantified in two dimensions: • The first is the probability of it happening which is a measure of how likely it is that a risk will become an issue. • The second is the impact that is a measure of the impact in terms of resource, time or scope. Projects usually measure both of these dimensions on a High/Med/Low type scale. This can produce a grid as illustrated. The grid can then be used to assess the level of mitigation (or actions to prevent the risk from becoming an issue) that should be associated with each of the risks. Obviously the ‘hotter’ the risk the more attention that should be paid to developing mitigation for that risk.This can all be summarised in thediagram below:Figure 1 - Project Control © 2008 Data Management & Warehousing Page 13
    • White Paper - Data Warehouse Project ManagementThe Event HorizonProjects are often expected to have detailed plans from start to finish; in practice this is bothunrealistic and impossible to maintain for data warehouse projects. Instead projects shouldaim to deal with the plan depending on when events are likely to happen. Short Term Current or near term phases need to be tracking every aspect of the work from the scope (enhancements) through milestones, activities, risks, tasks, test cases and for the current and next phase issues and defects. It is the issues and defects that will cause the current phase to be delayed or have other resource impact. The financial implications in the short term are over-spending or use of contingency because there is no space in which to manoeuvre. Medium Term Phases further out need to have the scope, milestones, activities, risks and where possible tasks planned out. These plans, their timescales and the budget for these phases should be adjusted in the light of experience gained on current and previous phases should influence the work being defined for these phases. Long Term Long-term plans are normally limited to rough scopes of work and timescales (for example we hope to add market segmentation to the data warehouse and expect it to take 5 people 3 months). This allows estimated budget requirements to be calculated but is highly dependent on the timescales and successful delivery of the previous phases. Learning from current work will improve the estimating over time.Using the above definitions it is possible to indicate what information a project managershould ideally have available in the following graphical way:Figure 2 - Information available to a project managerIn practice managers normally have even less information available (indicated by the redline). This means that both the short-term and medium-term horizons each cover two phasesrather than the three phases of the ideal situation. The long-term view is then restricted toknowing what enhancements are needed. © 2008 Data Management & Warehousing Page 14
    • White Paper - Data Warehouse Project ManagementTools and TechnologyIn order to manage such a large-scale project it is important to use a number of tools to helpkeep track of all the aspects of the project. 7There are six major groups of software that are needed for corporate programme/projectmanagement: 8 • Project Management Software 9 • Collaborative Software 10 • Ticket Tracking Systems 11 • Project Portfolio Management 12 • Resource Management 13 • Revision ControlFor a data warehouse it would be common to use the following tools in the following way: Project Management Software Most organisations either have a large centralised project management system such as Primavera P3 or desktop project management systems such as Microsoft Project. These provide the high-level Gantt charts and resource management. In the components outlined above it is most common to put the phases, milestones and activities into this type of tool. The detailed and changing nature of the tasks makes most of the tools unsuitable for the level of detail required. Also the tools tend to be inaccessible to those doing the work for a number of reasons: • Organisations are unwilling to invest in licences for every team member. • Applications require a desktop/laptop install. • Developers are unwilling to engage in using a project management tool. • Project managers don’t share the plan with developers for fear of contradiction about the proposed timescales As a result these tools are useful for holding the phase, milestone and activity level information but are rarely useful for task level information in data warehouse projects due to the large number of interlinking tasks required to manage the environment. Ticketing Ticketing systems are often used to track issues with software but their use can be much broader. Most ticketing systems allow the classification of tickets as well as dependencies between tickets and a workflow element that allows tickets to be assigned to others. A well-configured system is therefore the ideal tool for managing tasks, issues, risks, test cases, defects and individual enhancement requests. Tasks are initially raised; as the team member works on the task they may create dependant or sub-tasks (often as simple reminders to themselves or others). When issues arise they are captured against the task that is being affected allowing project7 http://en.wikipedia.org/wiki/List_of_project_management_software8 http://en.wikipedia.org/wiki/Project_management_software9 http://en.wikipedia.org/wiki/Collaborative_software10 http://en.wikipedia.org/wiki/Issue_tracking_system11 http://en.wikipedia.org/wiki/Project_Portfolio_Management12 http://en.wikipedia.org/wiki/Resource_Management13 http://en.wikipedia.org/wiki/Revision_control © 2008 Data Management & Warehousing Page 15
    • White Paper - Data Warehouse Project Management managers to see what is impacting the project timelines. The same is true for the other ticket types (enhancement, risk, test case and defect). Deploying a ticketing system, and opening it to everyone involved in the project from business user to developer, allowing everyone to see everything, can have dramatic effects on the speed and quality of development. It is important that the culture around the system is one of mutual trust and respect. It is easy for a poor manager to go and count the number of defects in an individual developer’s code as a KPI of that developer’s performance. This is very naïve for two reasons: it discourages honesty and it does not measure performance at all. Discouraging honesty is a disaster, because the manager will never know the true state of the project again. The number of defects against one developer may be high because that person is the most skilled developer and therefore gets the most difficult work to do, because the specification was wrong, because the source system has changed, or because the requirements changed when the users saw the initial 14 results. An effective ticketing system will also highlight delays and bottlenecks in the process where, for example, critical blocking tasks go unanswered. Careful review of this information allows risks and issues to be addressed early, thus reducing impact and for an understanding of the root cause of delays to be gained which can be used to speed up subsequent phases. It also removes the process whereby the project manager each week goes around and disturbs everyone else on the project to get an update on the status of each task, which wastes huge amounts of time for those disturbed. A project manager can get the status from the system and then talk to those individuals whose tasks and issues have a material effect on the status of the project. This is management by exception rather than micro-management of the process. Version Control A version control system is vital to the development of a data warehouse, not just for the code developed but every document used throughout the life of the project and subsequent maintenance. Whilst there are products especially built for document rather than code version control it is often sufficient just to use the same piece of software. The advantages of source code control have long been established but documentation control is often not addressed. If there is a requirements document on a shared drive that is updated when a new requirement emerges and (if the author remembers) the version number on the front page or in the file name is updated this does not mean that everyone working on the project has the same document. If someone copies it to their laptop to work on it at home over the weekend and on their return copies back to the shared drive all would appear well. The issue arises if two people have had the same idea, when the second person copies their work back to the shared drive the first person’s work is simply overwritten.14 This is a failing in the requirements capture process that should be picked up by the project manager from thesetickets and corrected © 2008 Data Management & Warehousing Page 16
    • White Paper - Data Warehouse Project ManagementUsing a versioncontrol system alsomeans that thedevelopment teamcan work against afixed version ofrequirements for aperiod of time, even ifothers in theorganisation are stillupdating therequirements. Thisstability allows thedevelopment processto be smoother and agap analysis performed at a later stage to understand what has changes allows thedevelopers to catch up with the requirement (normally in another phase).Finally the repository provides an auditable trail of the changes in all aspects of theproject documentation that again can help with improving future phases.WikiThe introduction to this section described collaborative software that covers a broadswathe of technology. In practice the most useful software that one can use is a wiki.This provides a number of web pages that any user can update with a syntax that issimpler than standard HTML. The pages that get created are often part of the informalstructure that helps the programme move along.Wikis can be used for frequently asked questions, staff lists, announcements, socialevents, guides to using other parts of the project, etc. Since they are a sharedresource, editable by all they can provide a productive communication tool that is farmore effective than email for encouraging collaboration.Wikis are more common than most people realise with websites such as Wikipediausing the same quick, cheap, easy software to build an encyclopaedia that can bedeployed as the team’s main source of information for the project. It can also have the‘water-cooler’ effect where people from IT and the business collect to share informationin an informal manner rather than through formal documentation or review processes.© 2008 Data Management & Warehousing Page 17
    • White Paper - Data Warehouse Project ManagementMethodologiesMost organisations will have a preferred IT development methodology. It is likely that the 15larger the organization the more structured the methodology (for example PRINCE2 or other 16waterfall approaches). These processes are often assessed by mechanisms such as CMMI. 17Few large organisations are willing to adopt agile approaches despite both the businessside saying that IT should deliver faster and meet requirements more completely and the ITside saying that they are trying to be flexible and meet that demand.As a result the organisations’ methodologies will not be well suited for the development of adata warehouse. This is because most methodologies have a fixed scope and a clearobjective. The deployment of a new financial package covers certain well-defined businessprocesses, has well understood inputs and outputs and bears scrutiny by use cases toexamine the steps.A data warehouse is a system that will have a number of fixed outputs but also the intangible‘just load up the data and I will know what I need when I see it‘ type requirement. It has a 18huge number of changing inputs from the source systems and often has large parts that willbe in production whilst others are under development causing a crossover between thedeployment and maintenance phases.Data warehouses are therefore inherently risky ventures and yet assessments such as CMMI 19set out to avoid projects that carry risk. In Peopleware it suggests of CMMI assessments: The projects most worth doing are the ones that will move you DOWN one full level on your process scaleTrying to deploy large-scale data warehouses in large organisations is what has lead to theconcepts described in this document, whereby phases, milestones and activities can bemanaged within the organisation’s standard structures but allowing more agile developmentmethods at the task level as well as detailed tracking of the relationship between issues andtasks which improves time and resource management.Remember that it is the deliverable and not the methodology used to get there that isimportant. It is all too easy to get trapped in trying to complete a methodology rather than thedeliverable.15 http://en.wikipedia.org/wiki/Prince216 http://en.wikipedia.org/wiki/CMMI17 http://en.wikipedia.org/wiki/Agile_software_development18 Imagine an organisation that has 25 source systems and two software upgrades/patches a year to each of them.This means that there are 50 changes a year or one a week to be analysed, designed, build and tested as well asmanaging maintenance and other issues that arise.19 Peopleware – Productive Projects and Team: Tom DeMarco and Timothy Lister, Dorset House Publishing, SecondEdition 1999, Chapter 29: Process Improvement Programmes © 2008 Data Management & Warehousing Page 18
    • White Paper - Data Warehouse Project ManagementProject LeadershipThe single biggest impact on any project comes from the leadership it receives. Since a datawarehouse is such a long-term commitment the leadership staff should understand that theyare entering into a project that they may be engaged in for several years. A revolving doorpolicy of leaders will only slow the project down.The project leadership comes in two parts: • The sponsors: An executive sponsor and his delegate, the project sponsor, who commission the project and will ultimately, control its use. • The project team: Leadership within the team normally consists of three individuals, a project manager, a senior technical architect and a senior business analyst who must work together to deliver, each taking responsibility for their own area but deferring to the others for their respective areas of responsibility.The roles are outlined in more detail below: The Executive Sponsor The Executive Sponsor is a senior manager or executive with demonstrable interest in the outcome of the project, who is ultimately responsible for securing spending authority and resources for the project. Ideally, the Executive Sponsor should be the highest-ranking manager possible, in proportion to the project size and scope. The Executive Sponsor acts as a vocal and visible champion, legitimizes the project’s goals and objectives, keeps abreast of major project activities, and is the ultimate decision-maker for the project. The Executive Sponsor provides support for the Project Sponsor and the Project Manager. They have final approval of all scope changes, and signs off on approvals to proceed to each succeeding project phase. The Executive Sponsor may elect to delegate some of the above responsibilities to the Project Sponsor (sometimes known as the Executive Sponsor’s Agent). Project Sponsor 20 The Project Sponsor is someone who strongly supports the objectives of the project and is willing to act as a champion and advocate on behalf of the project. To this end they are the primary source of business input and take ownership from a business and governance perspective. This can be broken down into six main responsibilities: • Accountability The project sponsor is responsible for holding the project manager to account and consequently keeping the project on track. The project sponsors must also make themselves available to provide support and address risks and issues.20 Adapted from:http://www.stanford.edu/dept/its/projects/PMO/files/linked_files/guidelines_sponsors.pdfhttp://ithelp.lincoln.ac.nz/site/story_images/3021_project_sponsor_s8869.pdfhttp://www.cit.cornell.edu/computer/robohelp/cpmm/Project_Roles_and_Responsibilities.htm © 2008 Data Management & Warehousing Page 19
    • White Paper - Data Warehouse Project Management • Strategic Fit Assure that the project is aligned with the organization’s strategic goals. • Resources Provide or locate resources for the project especially where these are outside the control of the project manager and protect resources from being pulled away. • Project Finances Provide or locate funding for the project and track the budget in conjunction with the project manager. • Lead Political Charge Help the project manager navigate the organization’s political environment. • Own the Final Product Be clear on what is expected in the end and help celebrate & transition.Project ManagerThe Project Manager is the person responsible for ensuring that the Project Teamcompletes the project. The Project Manager develops the project plan with the teamand manages the team’s performance of project activities. It is also the responsibility ofthe Project Manager to secure acceptance and approval of deliverables from theProject Sponsor and other stakeholders. The Project Manager is responsible forcommunication, including status reporting, risk management, escalation of issues thatcannot be resolved in the team, and, in general, making sure the project is delivered inbudget, on schedule, and within scope.Technical ArchitectThe Technical Architect is the single point of responsibility for the technical solutionfrom an application and system perspective.The main responsibilities of the Technical Architect are to: • Define the technical architecture • Solve technical issues • Ensure that all components of the technical architecture are properly integrated and implemented • Define the development tools and environment • Coach the technical team in the development of the technical architecture • Provide technical support and technical quality control throughout all stages of the project • Co-ordinate vendor services related to technology selection and implementation • Work with the Senior Business Analyst to determine how the technology can be applied to meet the business needs • Work with the Project Manager to ensure the smooth running of the project • Evangelise the technical solution to the rest of the IT community and the business© 2008 Data Management & Warehousing Page 20
    • White Paper - Data Warehouse Project Management Senior Business Analyst The Senior Business Analyst is responsible for: • Analysing and Defining the business requirements and solution • Quickly and clearly understanding the business issues and data challenges • Identifying the organizations strengths and weaknesses and suggesting areas of improvement. • Reviewing and editing requirements, specifications, business processes and recommendations related to proposed solution. • Leading the testing efforts • Working with the business to identify required changes • Communicating the required changes to the development team • Work with the Technical Architect to determine how the business need will be met by the technology • Work with the Project Manager to ensure the smooth running of the project • Evangelise the business solution to the rest of the IT community and the business Leadership Style Managing a small team to develop a large complex environment requires a degree of skill and the ability to adjust the style of leadership to match the situation. This type of flexibility is known as situational leadership and is a necessary quality in a data warehouse project. 21 Kenneth Blanchard wrote “... managers should work for their people, ... and not Supporting Coaching the reverse. ... If you think that your people are responsible and that your job Supportive Behaviour is to be responsive, you really work hard to provide them with the resources and working conditions they need to Delegating Directing accomplish the goals youve agreed to. You realize your job is not to do all the work yourself or sit back and wait to catch them doing something wrong, but to roll up your sleeves and help them Directive Behaviour win. If they win, you win.” As staff develop on a data warehouse project it is typical for a good leadership team to move from a directive style through the coaching style and supporting style and ultimately reaching the delegating style. For example the technical architect may start with ‘Build the data warehouse this way’ moving through ‘We build the data warehouse this way because …’ and ‘I like what you have done but have you considered …’ and ending with ‘Thanks, that’s just what I envisioned’. To this end a strong relationship and shared vision between project manager, technical architect and senior business analyst is essential.21 "Leadership and the One-Minute Manager", Kenneth Blanchard, Patricia Zigarmi, and Drea Zigarmi, p18. This is arefinement of Path-Goal Theory developed by Robert House in 1971 © 2008 Data Management & Warehousing Page 21
    • White Paper - Data Warehouse Project Management Organisational Learning Most people working on a project will learn in some way, but this knowledge, locked away in an individual, is of little value to the business as a whole. The team have to become a learning organisation that actively creates, captures, transfers and mobilizes 22 knowledge to enable it to adapt to a changing environment. 23 “Knowledge is not knowledge until someone else knows that one knows.” For a data warehouse project organisational learning is critical, as, not only is the environment rapidly changing but the number of places from which information is sourced is larger than any other IT project the business is likely to engage in. In this context information is more than data being transferred inside systems, it also includes information about how business processes work, key contacts both inside and outside the organisation, critical business requirements, data quality issues, etc. Organisational learning is a cultural phenomenon that requires team members to actively participate in developing knowledge. To this end a ticketing system, wiki and document versioning system provide tools but it is the people who have to engage in the concept and as a result be willing to: • Share information in a timely fashion • Capture even the most incidental information • Provide access to work in progress • Be open about issues and mistakes • Submit everything to a rapid peer review processes • Re-use rather than re-invent • Research and capture research for others to use • Have largely unrestricted access to the entire project knowledge repository • Accept change as inevitable and be prepared to adapt A team that has a culture capable of organisational learning will be: • More productive because they have the information to do the job • Happier because they are learning and their contribution is recognised • Less prone to expensive re-work as issues are addressed earlier All of this leads to faster, more successful projects.22 http://en.wikipedia.org/wiki/Organizational_learning23 Gaius Lucilius c180Bc-103BC © 2008 Data Management & Warehousing Page 22
    • White Paper - Data Warehouse Project ManagementTeam RotationIt is common practice for the most experienceddevelopers to be put in change of majorenhancements and then in a sliding scale ofexperience to assign people to maintenance workand issue resolution. This has initial benefits ofgetting the best developers to do the largestchunk of work but projects can also benefit fromrotating people so that after the enhancement isdeveloped those resources move to the issuehandling and other developers move up tomaintenance from issues and to enhancementsfrom maintenance.This has a number of very positive aspects. Firstlythe most skilled developers have done the firstdevelopment, and have therefore laid down the principles and standards for futuredevelopment that the others will follow. By moving these people into issue handlingthey also become aware of the consequences of their work and any quality or designissues that have been introduced.For the other less experienced teams they see career development as they can learnnew skills and have an opportunity to show that they can move up to the next level. Ifproblems occur they can still be supported by more senior developers but it is better toallow people learn early.This practice is also helpful in staff retention because it creates varied work and anopportunity to learn. It also means that people will work across subject areas andtherefore have a greater understanding of the whole. This in turn makes it easier whenindividuals do leave as combined with the organisational learning discussed above theteam is greater than the sum of its parts.© 2008 Data Management & Warehousing Page 23
    • White Paper - Data Warehouse Project ManagementEstimating and EffortInitial estimating on a data warehouse project will always be very subjective and thereforewrong. Plans will often start with what the developer feels is required (and therefore very high)and then adjusted down to meet what the project manager feels is acceptable (and thereforevery low). The final value will be somewhere between the two.It is important that everyone understands that the timescales will change in light ofexperience. Subsequent re-assessments of effort will become more accurate if theorganisation is willing to learn. This is often done by the organisational learning processdescribed above, combined with end of milestone or phase reviews and detailed reviews ofthe ticketing system – what issues arose, will similar issues arise in the next development andhow to allow or compensate for them.Tasks are also non-uniform, some tasks will take a short amount of time – often those at thestart and the end of a phase, others will take a longer period – often those in the middle of the 24phase. Project monitoring systems that determine completeness by the number of taskscomplete as a percentage of the total number of tasks will see the project initially getting‘ahead of schedule’ before ‘falling massively behind schedule’ and finally ‘recovering some ofthe lost time’. This is exacerbated by the fact that if a ticketing system is used additional taskswill be created and therefore the percentage complete will appear to drop at some points inthe project. Better measures are around the ratio of tasks to issues and the completion ofactivities to planned activity duration.Agile and other similar methodologies provide several techniques for improving theestimation. 25The first of these is velocity (or sometimes better known as load factor). This is the ratiobetween the developers’ estimate and what is actually achieved. So if a task is estimated astaking 4 days and then takes 10 days the “velocity” is 10/4 or 2.5. For the next phase thedevelopers’ estimates are taken and multiplied by 2.5 to determine the likely timescale. Thevelocity is measured for each estimate and whilst initially it will oscillate widely after a fewiterations it will settle to a factor that can regularly be used calculate elapsed time.The accuracy comes from allowing for two factors. The first factor is the developers owntendency to over or under estimate the effort required to do any given piece of work. Thesecond factor is the distraction that the work environment provides with telephone calls, e-mails and meetings, etc. These work distractions usually have an organisation specific patternthat is not normally factored in by developers but needs to be consistently applied to plansOver a period of time developers will come to understand their own estimating and theirvelocity will tend to a constant. The consistency of this number is what allows projectmanagers to be able to estimate accurately. 26Agile development processes also introduce the concept of technical debt . This is theobligation that an organization incurs when it chooses a design or construction approach thatis expedient in the short term but that increases complexity and is more costly in the longterm.24 See the Data Management & warehousing white paper “How Data Works” for a discussion of volume vs.complexity in data and how this affects the timescales in a data warehouse project.25 Version One discussion of Velocity: http://www.versionone.com/Resources/Velocity.asp26 First proposed by Ward Cunningham in 1992 at the OOPSLA92 conference (http://c2.com/doc/oopsla92.html) © 2008 Data Management & Warehousing Page 24
    • White Paper - Data Warehouse Project ManagementTechnical debt can occur accidentally (when a mistake is made) or deliberately (when aconscience decision is made to put something off). The secret to a successful implementationis to continually work to manage and reduce the debt, just like debt in real life. It is technicaldebt, so often unaccounted for, that often derails large projects as they run out of resourceand don’t understand where it has gone.Debt can be classified just as in real life financial situations: • Debt incurred un-intentionally o Usually due to low quality work (the uninsured loss) • Debt incurred intentionally o Short-term debt, usually incurred reactively, for tactical reasons such as numerous tiny shortcuts (credit card expenditure) o Medium-term debt incurred by individually identifiable shortcuts such as the creation of a tactical data mart (the car loan) o Long-term debt, usually incurred proactively, for strategic reasons such as the delay in purchasing significant technical infrastructure (the mortgage) • Non Debt o Some things are not technical debt such as feature backlog, deferred features, cut features, etc. Not all incomplete work is debt. These arent debt, because they dont require resource to correct them (the interest payment).Debt requires re-payment with interest, i.e. whatever has been done has cost something (thedebt) and correcting it requires additional resource (the interest) to put it to how it should be. 27As in financial circumstances delaying repayment always costs more in the long term.It should be noted that a well-defined technical architecture as discussed above is needed inorder to identify technical debt. This is because there must be a blueprint and baselineagainst which to draw comparisons against. This also re-enforces the need for a simplearchitecture as measurement is more difficult against complex architectures.It is often thought that time-boxing project phases either cannot be done for data warehouseprojects or will force developers to deliver something. Neither is true. A time-boxed project willdeliver something but can incur technical debt for those things that get omitted orcompromised. The delivered solution may be of little value if the time allowed is insufficient. Ifthe techniques described above are employed then early phases should be run as scope orbenefit driven rather than relying on time boxing. As velocity and technical debt are betterunderstood in the environment the project can switch to time-boxed activities to allow thebusiness to see tangible time-scaled development and deployment roadmap.Finally the project manager must understand the cost of their actions. By putting goodcommunication, processes and systems in place the project manager can effectively monitorand manage a project by exception, allowing the senior business analyst to handle thebusiness aspects and the technical architect to manage the technology and together progressthe delivery. If a team of eight people is brought together once a week for a two-hour statusmeeting with the project manager, then two working days out of the forty working days (eightpeople for five days a week assuming eight hour days) available in the week are lost. Thatweekly status meeting reflects 5% of all the available resource! 28 The ultimate management sin is wasting people’s time27 Note: The original draft of this paper was written before the 2008 financial crisis. The failure to manage debt in thereal world caused significant institutional melt down. This is mirrored by the meltdown of large data warehouseprojects within large organisations that have failed to manage technical debt.28 Peopleware – Productive Projects and Team: Tom DeMarco and Timothy Lister, Dorset House Publishing, SecondEdition 1999, Chapter 33 © 2008 Data Management & Warehousing Page 25
    • White Paper - Data Warehouse Project ManagementTips and TricksThe following section lists some items that are useful for a project manager to remember. It isnot exhaustive but provides some high level reminders: • Phase 1: Mobilisation Setting up a project has a cost, having a Mobilisation phase allows time and resource to be allocated to the setup. If there is no allowance for it then at the end of the first phase the cost of setup will be at least equal to the delay in the delivery of the phase. • Team Members Wiki Page If the project has a wiki (which it should) take photographs of everyone and add them to a page that has everyone on it, order it alphabetically by first name rather than in any organisational structure, include their contact details. This is again something that makes people feel involved and is a useful resource when someone remote to the core team or a new starter wants to contact another member of the team. • Develop a no blame culture Mistakes happen – they come from a lack of understanding or knowledge or poor communication and everyone on the team will make them at different times. It is therefore important that people are willing to admit to their mistakes and learn from them and that the project manager understands why mistakes are occurring and then take corrective action (e.g. training, improved communication channels, etc.) Where mistakes are hidden there is a build up of issues and technical debt that appear, inevitably, at critical points in the project. • Large Teams & Parallel Streams Avoid large teams and large numbers of parallel streams of work. Data warehouses are complex intertwined systems and the impact of a large number of teams trying to work simultaneously across the system will out-weigh any benefit from parallel working. Parallel teams work where there are discreet non-interacting areas, otherwise the contention between components and resources becomes too much of an overhead. Also as a team gets larger the degree of effective communication within the team drops. The balance is to have sufficient people to do the work and yet few enough to ensure effective communication between them. • Transparency By default let everyone involved with the project see everything. Exceptionally restrict access to sensitive documents. Always challenge any request to categorise a document as “sensitive”. This helps for many reasons; delays are not a sudden surprise to users, others on the project often volunteer help or expertise about issues in unexpected areas, it develops an inclusive aspect to the team dynamic, it provides an opportunity for people to learn from the work of others, etc. • Document and version everything As this document has discussed, having a ticketing and version control system in place makes management of a rapidly changing environment possible. No individual or group of individuals can effectively manage all the tasks, risks, issues, changes and documents of a large project with manual systems and if the system is easy to use then documenting everything is preferable to trying to decide what might be needed in the future. © 2008 Data Management & Warehousing Page 26
    • White Paper - Data Warehouse Project Management • Use Storyboards but avoid Use Cases 29 Many formal methodologies for OLTP systems have use-cases that describe how a user must get through a system. This is fine for such solutions where a pre-defined path is essential. For a data warehouse the individual use case is irrelevant as the user is unlikely to follow the same route regularly. Instead a story is required that talks of a user being asked to produce a specific report and then having to do analysis of another set of data and incorporate it in the results, etc. The storyboard should describe the process of moving around the system but not prescribe specific routes for users to take, instead focusing on how to transition between aspects of the system. • Key Design Decisions Whilst the idea of documenting everything has already been described it is particularly important to document key design decisions such that when a problem occurs it is easy to follow the intended route rather than revert to a re-evaluation of possible solutions. • Manage Technical Debt Ensure that everyone can quantify the cost implications of the decisions they make and document them in such a way that the project can assess the technical debt they are accruing and schedule the debt reduction appropriately.29 “UML Distilled” Martin Fowler “The more I see of use cases, the less valuable the use casediagram seems to be. With use cases, concentrate your energy on their text rather than onthe diagram. Despite the fact that the UML has nothing to say about the use case text, it isthe text that contains all the value in the technique.” © 2008 Data Management & Warehousing Page 27
    • White Paper - Data Warehouse Project ManagementSummaryData warehouse projects pose a specific set of challenges for the project manager. Projectsoften have poorly set expectations in terms of timescales, the likely return on investment, thevendors’ promises for tools or the expectations set between the business and IT within anorganisation. They also have large technical architectures and resourcing issues that need tobe handled.This document has outlined the building blocks of good project control including the definitionof phases, milestones, activities, tasks, issues, enhancements, test cases, defects and risksand discussed how they can be managed, and when, using an event horizon the projectmanager can expect to get information.To help manage these building blocks this paper has looked at the types of tools andtechnology and how they can be used to assist the project manager. These tools includeproject management software, ticketing systems, version control and wikis. It has also lookedat how these tools fit into methodologies.The final section of the paper has looked at how effective project leadership and estimatingcan improve the chances of success for a project. This has included understanding the rolesof the executive sponsor, project manager, technical architect and senior business analystalong with the use of different leadership styles, organisational learning and team rotation. © 2008 Data Management & Warehousing Page 28
    • White Paper - Data Warehouse Project ManagementAppendix 1 – Governance ProceduresEach organisation will create its own processes for the governance of the data warehouseproject. Data Management & Warehousing have defined and used processes in three majorgroups: • Change Management Processes Identifying and controlling enhancements, maintenance and issues with the system • Data Warehouse Processes The processes associated with the build and deployment of the data warehouse itself • Data Management Processes The processes associated with the management of the data held within the data warehouseOutlined below are the major components of each of the processes. These are given asexamples only and are shown only with the high level summary diagrams and shortdescriptions as indicators to what should be developed by an organisation to service itsenvironment Change Management Processes Enhancements and maintenance processes are required to ensure the communication of the organisation’s requirements of the data warehouse and to ensure that those requirements are costed and prioritised in such a way as to deliver maximum benefit. The issue management process is required to ensure that all issues found with business intelligence outputs are captured and resolved consistently, allowing for KPI capture and process visibility for active management. © 2008 Data Management & Warehousing Page 29
    • White Paper - Data Warehouse Project ManagementData Warehouse Processes The requirements process is required to ensure that the user requirements are met effectively by creating a statement of what is required and a process for changing the definition of what is required The enhancement request packaging process is required to ensure that the scope is limited to user requirements that deliver benefit at reasonable cost and project timescales are predictable. The ETL design & build process is required to ensure that a reliable, robust, accurate and performant data loading process is designed and build. The report design and build process is required to ensure that a reliable, robust, accurate and easy to use reporting solution is designed and build. The testing process is required to ensure that the solution is a robust and accurate reflection of the requirements; that changes and issues are handled promptly, and performance is adequate.© 2008 Data Management & Warehousing Page 30
    • White Paper - Data Warehouse Project ManagementData Management Processes The data quality process is required to measure, control and improve the quality of data held in the data warehouse, reacting to new issues quickly and ensuring source systems are responsible for the data they provide. The data model process is required to develop and maintain both the logical and physical data models against set standards, and aligned with metadata. The data lifecycle process is required to give structure to the adding and removing of data from the data warehouse, and maintaining a balance between business, hardware, cost and compliance considerations. The data security process is required to ensure that the organization complies with legal requirements related to data protection, and that the data warehouse complies with all the organizations internal data policies. The metadata process is required to ensure that metadata is captured consistently as part of the data warehouse development, such that it can be effectively used in applications including data quality checking, data profiling and data security.© 2008 Data Management & Warehousing Page 31
    • White Paper - Data Warehouse Project ManagementAppendix 2 – Project ServicesData Management & Warehousing have a web based project management service configuredfor the development and management of data warehouses.The solution provides a complete environment including version control, ticketing, wiki andmany other features using free, widely available, open source software.A full description of the service can be found at: http://projects.datamgmt.comA demonstration system can be found at: http://projects.datamgmt.com/demoThe Data Management & Warehousing internal project management (screenshot above) canbe found at: http://projects.datamgmt.com/datamgmtA description of how to build the system on your own server can be found at: http://projects.datamgmt.com/howtoThe company also hosts the service for clients who require a fast start on their projects.Copyright© 2008 Data Management & Warehousing. All rights reserved. Reproduction not permittedwithout written authorisation. References to other companies and their products usetrademarks owned by the respective companies and are for reference purposes only.Some terms and definitions taken from Wikipedia © 2008 Data Management & Warehousing Page 32