By Madan Sheina

Netezza has shown off a new compression engine which it plans to add to its data warehousing appliance early next year

The new Compress Engine introduces patent-pending compression algorithms that compile column-based data tables to disk efficiently to maximize access and query performance. Netezza claims the engine will double the query processing performance of even its newest Netezza Performance Server (NPS) appliance family.

Netezza said its compression approach differs from other techniques used by rival database providers in that it's specifically designed to improve performance and not just to reduce data size and footprint.

The Compress Engine does this quite cleverly; as data is loaded into the NPS appliance it's automatically compressed column-by-column into a complied format and associated with an "instruction set" for de-compilation. When the data is requested from disk by a query the4 engine reads the instruction set and reassembles the data on the fly as it streams from the disk – in effect doubling the stream rate.

The Compress Engine is the latest addition to Netezza's Fast Engines Framework which uses field programmable gate arrays (FPGAs) and other commodity components to push processing power as close to the data as possible. Other Fast Engines in the Framework manage access to data flows, breaks-down data streams into relevant rows and columns, ensure data integrity and efficiently filter row and column data into memory or to specific queries. These Fast Engines have been deployed to around 125 Netezza customers to date.

Compress Engine will be included as part of the NPS appliance family, which was released last summer, in May 2008.

It continues a theme of price-performance improvements by Netezza this year. Last summer the company release NPS 4.0 which also claimed to "double" performance through several algorithmic software enhancements that provide better support for mixed processing workloads and larger numbers of concurrent users.

Pricing for NPS 4.0 with the new Compress Engine remains the same. Netezza also claims that customers are not forced to upgrade their hardware footprint or cooling requirements.