The Adaptive Project Mart

Several years ago there was an addition to the corporate information factory (CIF) that received little attention. The architectural entity that was added was the project mart, or the adaptive project mart. The adaptive project mart is technology that exists between a data mart and an exploration warehouse. The adaptive project mart had some of the characteristics of a data mart and some of the characteristics of an exploration warehouse.

In its simplest form, the adaptive project mart is like a giant spreadsheet created from the data warehouse. What are the appeals of a spreadsheet? Spreadsheets are flexible and they can be easily manipulated. Spreadsheets are able to be controlled by a single user, for a single focused purpose. Spreadsheets are created quickly and changed just as quickly. In many ways, spreadsheets are like structured scratch pad areas for analytical processing. The commercial success of Lotus and Excel is testimony to the usefulness of spreadsheets on the desktop.

Adaptive project marts are useful for many reasons. One reason adaptive project marts are useful is that occasionally the “special project” arises. Management decides that something needs to be studied. This is a reaction to a new law, new technology or a new competitor. Management needs a framework or prototyping environment to determine its response. A special project is created.

The subject of the special project is not one of the things that is regularly monitored by a data mart. Therefore, an analyst is assigned the task of creating an entirely new study. Often, this study will be of a finite nature, and the study will be discarded after some conclusions are reached. For this reason, because of its temporary nature, the special study is called a project.

In many cases, the data for the special study comes from the data warehouse, but must be tweaked. Some data is pre-calculated, some is deleted, some may be added, etc. The data is configured to meet the precise requirements of the study.

The adaptive data mart seems to be an exploration warehouse. Indeed, it has many of the characteristics of an exploration warehouse, except that the adaptive data mart is aimed at usage by the end user, not a statistician. An exploration warehouse is aimed primarily for use by statisticians.

Adaptive project marts offer the same kind of flexibility to the world of data warehousing that spreadsheets offer to the desktop end user. In the past, adaptive project mart technology was mostly limited to software. But now there is a new twist for adaptive project marts. That twist is that there is a hardware solution that is available for adaptive project marts as well. That hardware is Netezza.

While Netezza fits in other architectural roles within the CIF, it also happens to fit nicely with the need for hardware for an adaptive project mart.

Consider that an organization has a need for analytical processing for a special report or project. The data warehouse currently resides on a hardware box running multiple queries on several levels of data. What does the organization do? The first reaction is to simply carve out more space within the existing hardware environment. While this is certainly an option, there is the issue of capacity and cost. What if the existing environment is already cramped? What if the existing data warehouse machine is already close to capacity? What about cost? As a rule, the most expensive hardware in an organization is running the large-scale data warehouse.

When an organization has a “rush job” to do, taking that rush job and putting it in a corner, out of harm’s way on a separate machine, makes a lot of sense. Not only is it less expensive, but it keeps the data warehouse environment “pure.” There is no contamination—in terms of capacity, data or anything else—when the special project is run on an adaptive data mart residing outside of the machine that houses the data warehouse.

Given the capacity and cost of Netezza, it makes sense that the adaptive project mart is more effectively run outside of the mainstream data warehouse environment.