A data warehouse is a centralized repository where company-wide data is consolidated and stored in a format that enables clear, accurate, and user-friendly reporting in tools such as Excel, Power BI, or other analytical platforms.
At some point in every company’s lifecycle, you inevitably realize that you are operating too many business systems and facing more information demands than you can efficiently—meaning correctly and quickly—extract, transform, and convert into meaningful reports. This moment comes at a different time for every organization.
It depends on the company’s nature, the number of employees, organizational structure, requirements for data accuracy, the maturity of the analytics/controlling department, and other factors. In many cases, you also need to integrate multiple business systems because each one holds part of the information needed to complete the overall picture. Although it’s hard to generalize, I would say the right time for a change (or at least for an expert consultation) is when you start to feel that you can no longer manage the volume and complexity of information in the company.
I know many medium and large companies that rely entirely on Excel and still produce sophisticated and fully sufficient reporting. However, even this approach requires skilled people who design the reporting structure. An Excel reporting concept can be created very well—or very poorly. Smaller businesses can sometimes operate without structured information or rely on occasional, fragmented data simply because the owner keeps key facts in mind. But for medium and large companies, such an approach quickly becomes unsustainable. 🙂
Phases of a Data Warehouse Project:
- Selecting an implementation partner
- Preparing project documentation, including major project pillars, timelines, and contractual documentation — a critical step
- Project execution, ongoing monitoring, acceptance/testing of deliverables, and phased release of funds
Contacting and Selecting a Consulting Company
Introduction: Once you decide to build a data warehouse in your organization, you will likely approach a specialized consulting firm. They will send a consultant to assess the scope of work and prepare an initial proposal. Experienced companies that have delivered many data warehouse projects can usually estimate the required effort fairly accurately based on structured discussions with you.
It’s crucial to come well-prepared to this meeting and clearly understand your expectations — for example, which financial reports you need, whether analysts require direct data access, and so on. It’s common for clients to know they want a data warehouse but still be unsure what exact functions they expect from it or which problems it should solve. What information do you ultimately want to report? In such cases, expect the consultant to guide you through the discussion. This often signals that managing the project — preparing a solid briefing, monitoring progress, validating results — may be challenging.
An unreliable implementation partner will notice this immediately. They may promise a lot, communicate well, but deliver slowly and with poor quality… and you will pay for it — with money, time, and nerves. Many such projects never reach completion.
What Matters When Selecting a Consulting Company
-
- Choose a company with strong references — ideally one that has worked on a similar project in an organization with a comparable business model.
- Ensure you have an in-house expert involved. Not a basic IT technician who installs printers, but someone with a business intelligence or controlling background and ideally with experience in databases. Alternatively, hire independent project oversight — similar to hiring a supervisor when building a house.
- Ask for hourly rates in advance and consider them when choosing a partner. Typical examples (note both unusually low and unusually high prices):
- Company consulting rates: 500–1000 USD/day (in the Czech Republic)
- Freelancer rates: 250–600 USD/day
- Do not make any payments until both the project documentation and pricing are approved. A reputable company will attend the initial meeting free of charge.
Processing Project Documentation and Data Warehouse Price
Project documentation is a critical deliverable that defines the project structure. Well-prepared documentation also protects both sides in potential disputes over scope or output quality. It should be part of the contractual documentation, especially for larger projects.
Some people believe documentation isn’t worth the effort because unexpected situations will inevitably arise during the project and be solved on the fly. While that’s partly true, many of these situations can still be anticipated, defined, and addressed within the documentation. This prevents scenarios where the contractor claims: “You didn’t tell us this,” or “What do you expect for this budget?”
Preparing documentation takes time and requires collaboration between the client and the contractor. Ideally, an independent third party should also be involved. Project documentation should be divided into major pillars that are significant enough, yet still manageable.
For larger projects, it is standard practice to release payments only once a specific pillar is accepted. Therefore, there should be enough pillars, and they should be as independent from each other as possible.
Each pillar should then be broken down into smaller activities with estimated completion time in hours/days. The contractor should also specify what they need from you to deliver each task successfully. The documentation may also include customer needs analysis, feasibility analysis, and other supporting sections.
The project documentation can then be referenced in the contract, where you can define:
- Billing by pillars/activities
- What happens if a particular pillar exceeds the estimated budget
- How to file complaints about deliverables and when additional compensation applies
- Contract termination terms
- Other provisions
Data Warehouse and Project Documentation Pillars
The pillars of a medium-sized project may look like this. Each pillar should ideally be divided into smaller tasks. You can also ask ChatGPT to help break down your DWH project.
Project Pillars for a Data Warehouse Project — Example
- Customer Needs Analysis (may be handled separately in the contract and invoicing) (10 MD)
- Investment and Setup of SQL Server (6,000 USD + 3.5 MD) Note: This refers to an on-premises architecture. Cloud solutions such as Snowflake eliminate upfront investment and use a “pay-as-you-go” model.
- License Purchase (6,000 USD)
- Installation and Configuration of VS (0.5 MD)
- Installation and Configuration of SQL Server (report server, emails, job agent, security, testing, etc.) (2 MD)
- Installation of SW 1…n (1 MD)
- Architecture Design of the Data Warehouse (24 MD)
- Incremental Area — 50 tables from 3 source systems (3 MD)
- Stage Area — 50 tables from 3 source systems (3 MD)
- Presentation Layer — final data storage; creation of snowflake schemas resulting in 23 tables across 3 schemas (15 MD)
- Logging — logging framework (2 MD)
- ETL Area — database for ETL objects (1 MD)
- Design and Implementation of ETL Processes + Testing (60 MD) — Note: With modern democratizing tools, the required effort can be much lower.
- Source System 1…n → Increment Area — 10× SSIS packages extracting and preparing data + metadata creation (10 MD)
- Increment Area → Stage Area — 50× SSIS packages historizing data (10 MD)
- Stage Area → Presentation Layer — procedures and SSIS packages transforming and loading data (20 MD)
- Testing (functional and performance) (20 MD)
- Design and Implementation of Reporting (16 MD)
- Reporting Services — 5 reports (10 MD)
- Analysis Services — 2 OLAP cubes (3 MD)
- Excel / Power BI — 2 Power BI reports (3 MD)
- Employee Training (10 MD)
Note: Pillar 4 is significantly larger than the others. It should ideally be divided further to reduce risk and improve ongoing oversight so it doesn’t become a “black box.”
Update (2024-02-08) — This article was written several years ago. A major part of the project cost comes from the traditional ETL (extract, transfer, load) approach. A more modern approach is ELT (extract, load, transform), where data is loaded first and transformed afterward. This aligns with data democratization and data-driven management. New tools support this approach, reduce ETL costs by tens of percent, and enable less technical users to manage the solution. Examples include Keboola (ETL/ELT), Fivetran (ELT), dbt (transformation), and others.
Total Price for the On-premises Data Warehouse — Example
When all planned tasks are summed up, we get an estimated budget that can either be accepted or refined by adjusting requirements. Setting expectations in this structured way helps avoid unpleasant surprises.
- TOTAL 123.5 MD
- MD Rate 320 USD
- Total Work 39,520 USD
- Investment in SQL Server 6,000 USD
- TOTAL excluding VAT 45,520 USD
Note: The figures above represent only investment costs. You must also consider internal project management and potential staffing needs. Someone will need to maintain the data warehouse, implement changes in reports, integrate new systems, etc.
It’s ideal for a new employee to join the project from the beginning. If you don’t plan to hire, the project should be designed to minimize future maintenance. Alternatively, data warehouse administration can be outsourced at a reasonable cost.
Cloud Price (Fabric, Azure, Snowflake) — no infrastructure worries
Current trends in data-warehouse development no longer rely on on-premises architecture or large teams of programmers. Starting a project on modern platforms such as those below can significantly reduce CAPEX and OPEX while improving scalability. In many cases, costs can be 30–50% lower compared to traditional architecture.
Using platforms like MS Fabric — a robust, all-in-one data platform — removes upfront CAPEX entirely, replacing it with a monthly subscription. For small to medium-sized solutions, Fabric typically costs around 300–600 USD, depending on the chosen capacity plan and reserved capacity (e.g., F4 reserved for a year is roughly 300 USD). See: Fabric – Pricing Structure of Fabric – Models, Tiers, Selection and Recommendations.
Conclusion
This example illustrates the budget for a well-designed data warehouse in a medium-sized company integrating three business systems. The warehouse automatically pulls data from source systems daily and stores the results.
The contractor also delivers around a dozen high-quality reports to support decision-making, which are regularly sent to managers by email. Data analysts can connect to OLAP cubes through Excel, giving them access to all key data in the data warehouse.
Managers can access essential reports on their mobile devices through Power BI. The contractor provides complete training for all employees who will work with the data warehouse. Manual reporting processes that previously took significant time are eliminated and replaced with automated reporting.
Of course, what looks great on paper can be different in practice. Deliverables must be monitored closely, and the project must be actively managed to ensure the contractor doesn’t rest on their laurels!!!