Paul Christidis writes:

> Existing IMAGE based application.  Other users want to access the same data
>  from a number of desktop utilities.  It has been suggested to periodically
>  extract the IMAGE data to flat files, transfer them to a 'data warehouse'
>  machine and let the users 'go at it'.
>
>  Any suggestions where, perhaps, the data stay in IMAGE and are still
>  available to the users or where the transfer occurs dynamically as changes
>  are made to the IMAGE bases system, would be welcomed.

Paul,

This subject represents one of my current irritants, so please excuse the
animated nature of my reply. In short, I feel that data warehouses are an idea
designed primarily to separate users from their money. Moreover, implementing
data warehouses is a process that tends to give the appearance of progress and
activity at the expense of careful thought.

I feel all the more strongly about this when the databases are IMAGE-based.

Data warehousing was an idea that was born almost wholly in the RDBMS
community, particularly among those people who had designed wildly over-
normalized databases. Query extractions against these databases simply took
forever -- and these people were faced with a choice: either use the database
for data entry or data reporting, but not both.

Nonetheless, the first great rule of data processing 25 years ago was: don't
duplicate your data. Keep a central, single point of control. If you duplicate
any part of your operation, data, code, whatever, you screw yourself up in a
thousand simple ways, all of which make life extraordinarily more complex.

I don't think that there's any reason to believe that the rule has any less
validity today.

Moreover, a well thought out, properly indexed IMAGE database can be designed
to allow both very high speed entry and very high speed retrievals, especially
now that CIUPDATE allows you to add new automatic masters where necessary and
these masters can be b-treed with such enormous ease.

Secondly, if the "data warehouse" is to appear on the same machine as the
original database, what have you gained? CPU utilization will be the same or
greater and database synchronizations are going to be a constant and pervasive
problem, particularly so if you care about the accuracy of the data in the
duplicated database.

Adding keys where necessary to existing IMAGE databases for effective, high-
efficiency queries is a virtually zero cost process, but it will get you
virtually everything that any data warehouse vendor will promise you, at
almost no increase in disc space utilization, CPU bother, or fiscal cost, but
with all of the advantages attendant to simplicity of operation -- and those
advantages can never be minimized.

Wirt Atmar