Data Warehouse Project Management

Listen, Listen, Listen to the Stakeholders

By Hari Mailvaganam

A paramount determining factor in the success of data warehousing is the input of stakeholders. Data warehousing is very unique to an organization, its business processes, systems architecture and decision support needs. 

Project management for data warehousing allows for large amounts of user input and at all phases of the project. There are commercial software products tailored for data warehouse project management. A good project plan lists the critical tasks that must be performed and when each task should be started and completed. It identifies who is to perform the tasks, describes deliverables to be created and identifies milestones for measuring progress. 

There are a number of publications on data warehousing project management. The standard bearer publication is Ralph Kimball's Data Warehouse Lifecycle Toolkit.

Over the years I have worked with different project management methodologies and processes for data warehousing. In this article I have listed a summary of  a project management methodology that I have developed that also ties in with the Rational Unified Process (RUP). This is especially useful if the data warehousing project is part of a greater development effort which follows RUP.

The program management framework for data warehousing follows tested methodologies. The methodology described below follows the project management workflow of the Unified Process.

Figure 1. Rational Unified Process; Copyright IBM

The project plan should include presentations at regular intervals, say monthly, to management and stakeholders. The presentation will include:

  • Discussion on any challenges and determine if project is on schedule;
  • Review of activities and priorities to be achieved before next meeting;
  • Contingency plans to make up time and address problems;
  • An open question-and-answer period followed by a summary.

Phases of the Unified Process

The phases in project management following UP, Figure 1, are:
  1.  Inception
  2. Elaboration
  3. Construction
  4. Transition

The project lifecycle is composed over time into four sequential phases, each concluded by a major milestone, Figure 2.

The RFP is produces at the end of the inception phase.

Figure 2. Phases in the Unified Process; © IBM

1. Inception Phase

  • Project planning and evaluation.
  • Requirements gathering.
  • Define features the system must support.
  • Identify the stakeholders who oversee the system use.
  • Users and other applications (i.e. actors) that will interface with the system.
  • Define framework to identify business rules.
  • List the events that the system must be aware of and respond to.
  • List the constraints and risks place on the project.
  • Define product selection criteria.
  • Define project management environment.
  • Prepare and obtain approval of sponsors for project mandate and approach.
  • Define data warehouse objectives.
  • Define query library/location matrix.
  • Define query libraries.
  • Establish meta-data.
  • Prepare migration specifications.
  • Develop query usage monitoring.
  • Technical infrastructure assessment.
  • Current technology assessment.
  • Release of RFP.

The project plan must also place importance to:

  • Change control.
  • Risk assessment.
  • Training.
  • Scope agreements.
  • Resources.
  • Communication.

2. Elaboration Phase

  • Review of RFPs and selecting winning candidate.
  • Determine size of the data warehouse.
  • Determine complexity and cleanliness of the data.
  • Identify number of source databases and their characteristics.
  • Select DSS tools.
  • Determine network requirements.
  • Conduct training.
  • Integration of data.
  • Plan quality assurance testing procedures.

3. Construction Phase

  • Implement components.
  • Systems integration.
  • Conduct testing.
  • Plan deployment.
  • Develop support material.

4. Transitions Phase 

  • Roll-out of modules.
  • Managing change-control requests.
  • Conduct beta testing to validate the new system against user expectations.
  • Perform converting operational databases.
  • Evaluate performance and benchmarking.
  • Evaluate that deployment baselines are complete.
  • Obtain stakeholder feedback on deployment baselines consistency with evaluation criteria.
  • Fine-tune product based on feedback.
  • Release product to users.
  • Product release milestones.

Please contact us if you have any questions or suggestions.