Chris Dunlop writes:

>We are considering developing an application which would involve handling
>large amounts of free-format text, as well as fixed format data.  We are
>entirely happy holding the fixed format data in Image, but much less happy
>about holding the free-format text there.

One downside of PostgreSQL 7.0.3, the current production release, is that
each tuple can be a maximum of 32KB and it's strongly recommended not to
exceed 8KB. PostgreSQL does have a nice integrated facility for handling
BLOBs, though, and the tuple size limit is scheduled to be removed
shortly in an upcoming release.

>XML would seem to be a good way of structuring this data.  I've read a
>little about XML but haven't used it yet.  But how could you store XML, in
>particular, 'PCDATA' (free format text in this case), on an HP 3000?  I
>understand that it is possible to do this in RDBMS, but I can't see how
>Image, with fixed data item lengths, could be used to do this.

Storing XML as a long string would work. This means that you'd need to
parse the XML in order to recover the structure. This may or may not be a
big burden: if you're guaranteed to actually want some part of the
document you parse -- that is, if conventional SQL queries retrieve
exactly the documents you want, so you never have to parse unwanted
documents -- the overhead is probably acceptable. If you actually want to
do the search based on the freeform document text, you need some better
way to approach this.

MySQL, another open-source RDBMS, has a facility for handling large text
fields and keywording them. But MySQL also has some limitations that
disqualify it for robust applications; specifically, the current versions
lack transaction support. MySQL is expected to acquire transaction
support in the near future. (The advantages of competition!)

>If anyone can suggest a way of handling structured text in Image like this,
>I'd be very pleased to hear.

Well, you can always store the text in big text fields in
sequentially-numbered detail records. If you encapsulate the reading and
writing in one place that simply accepts big text strings to store and
returns big text strings assembled from IMAGE records, that might be
acceptable. The performance could get pretty bad without frequent detail
reorganization, unless all the documents were pretty much the same length.

-- Bruce


--------------------------------------------------------------------------
Bruce Toback    Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc.            (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142      | But ah, my foes, and oh, my friends -
Phoenix AZ 85028                   | It gives a lovely light.
[log in to unmask]                   |     -- Edna St. Vincent Millay