Datawarehousing and Business Intelligence (DWBI) needs usually vary by customer. However, there are a common set of evaluation criteria that can be applied while evaluating the proposals from various vendors that usually include boutique as well as established IT service vendors. Irrespective of the size, complexity or scope of the DWBI project, the below list of 10 things expected from vendors in their proposals can decide on the success or failure of your project.
- DWBI Project Planning: Does the bid consider an End to End (E2E) impact on the DWBI platform in the proposed project approach? Have the requirements been peeled into Source System changes, Data Integration changes and Business Intelligence layer changes? Were delivery dates independently planned or dependencies considered for each of them?
- Working with non-DWBI teams to accomplish project goals: If the bid is for a System Re-Platform project that involves core technology layer changes, has a system cutover plan been included with the proposal?
- DWBI Business Continuity and High Availability Mechanism: Has a rollback mechanism or running a parallel system been considered as part of the effort estimates?
- Software Configuration Management Strategy: Has a “code freeze” time been mandated in the proposal to effectively push changes to production systems?
- Streamlining/Governing non-DWBI development efforts: Is there a mechanism to streamline and govern the usage of new data available in the Data Warehouse? Has metadata been published on reporting / adhoc query tools effectively to be used by power users who need to use the data? Do the power users have access to a documented guideline to develop reports or query the newly published data and functionality?
- Data Lineage and Impact Analysis: Will the business glossary terms, their relation to columns in the database and report attributes including derivations, consolidations and aggregations be documented?
- System Performance: Is there a mechanism to point out which line of code, in the newly introduced functionality, attributed to degradation in performance?
- DWBI Skill Requirements: Does the proposed staffing model consist of BI personnel who understand data and Data Integration personnel who understand BI? Is the team led by a DWBI certified Project Manager/Lead?
- Code Quality: Can code quality be measured? Can some of the metrics below be inferred from the metrics the vendor is capturing during the project life cycle?
- Right First Time
- Defect ratio per change (Quality Index)
- Performance metrics of the system
- Coding Metrics like Review comments per change, Code Quality, Documents for a change, etc.
- Productivity Gains: How does the vendor plan to measure productivity gains and efficiencies as his team gets familiar with your DWBI environment? Are there any reusable tools, solutions, components that are considered to be built as a part of the bid and who owns the IP rights for them?
I welcome your comments on the relevance of this list as you benchmark them against responses you receive for RFP’s.