I posted to HP3000-L (and Rene Woc commented via an internal Adager memo,
which I include below, after my original post):
>There are some other VERY subtle issues having to do with detail datasets,
>their HighWaterMark, their FreeEntryCount, and -- last but not least --
>their FreeEntry *LIST* (or delete chain). Further knowledge of these
>issues might be of interest to the followers of this thread (bits-and-bytes
>folks as well as management folks).
>
>I did a quick scan of my Adager Lab log book and, surrounded by electronic
>cobwebs, I found some notes from the late 1970s, when I was actively
>reverse-engineering the root file and the dataset structures to be able to
>write the low-level bits-and-bytes Adager procedures I needed.
>
>It turns out that if you delete ALL the entries of a detail dataset (via
>QUERY or via applications), the HighWaterMark is reset to zero, the
>FreeEntryCount is reset to the current capacity, and the FreeEntryList
>*HEAD* is reset to 0. So far, so good. What's not obvious is that the
>FreeEntry *LIST* itself remains unchanged ABOVE the HighWaterMark, with all
>of its pointers intact. If IMAGE actually took the time to go around
>zeroing out all of the FreeEntryList *POINTERS*, users would (very
>unhappily) discover that the LAST DBDELETE on a detail dataset would take a
>monumentally long time (particularly if you are a typical high-end Adager
>user with several hundred million entries).
>
>There are some nasty consequences of this. In particular, if the
>FreeEntryList gets corrupted (either the HEAD or any of the POINTERS), it
>may so happen that the old (by now CADAVER) FreeEntryList gets incorporated
>into the CURRENT FreeEntryList. I don't need to tell you that this may
>eventually lead to DBPUT placing NEW entries ABOVE the HighWaterMark, which
>would be totally out of reach for standard non-privileged programs (such as
>QUERY and your applications) and, even worse, might get over-written by new
>entries as the HighWaterMark grows... Quite horrible, indeed!
>
>In the past, IMAGE did not check for this. I haven't played with the issue
>for a long time, so I don't know whether DBPUT now checks before attempting
>to place the entry above the HighWaterMark. Perhaps the IMAGE Lab can
>share some info on the current state of affairs. What I know is that, at
>least since the early 1980s, Adager's SetErase function zeroes the old
>FreeEntryList and Adager's DetPack function brings everything in line.
>
>
>Ah... It's been so long since I worked on these issues. I guess I am a
>nostalgia buff :-)
Rene Woc (Adager's General Manager) did some experiments with TurboIMAGE/XL
HP30391C.06.18 regarding my post. Here are his results:
Date: Sat, 22 Nov 1997 1:52:51 -0600
To: [log in to unmask]
From: Rene Woc <[log in to unmask]>
Subject: Bad HWM and bad FreeEntryChainHead Tests
Alfredo,
IMAGE does not allow me to overwrite an entry that is already in use as a
result of a bad HWM or a bad FreeEntryChainHead (but I get some nasty error
messages as a result).
-------------------
First an example of what happens with a bad HWM:
-------------------
Initial state of INVOICE according to Adager's Report Dataset Function:
58.67 % full: 176 entries 124 available
Capacity: 300
Detail Highwater mark: 177
FreeEntry head: 177
Block length (words): 485
Media-record length (words): 121
Entry length (words): 113
Blocking factor: 4
Fields per entry: 21
Number of paths: 2
Primary path is path number 1
-------------------
Using Adager's Therapy function, I made HWM = 176 (it was 177). Upon adding
the 2nd entry with query, I got:
FREADDIR FAILURE 0 0
==============| ENTRY CANNOT BE ADDED |==============
In the old days, this used to be a message that said:
MPE File error 0 returned by FREDDIR on root file or dataset nn.
---------------------
If I enable the database for dumping and perform the same operation, I get:
TURBOIMAGE/XL ABORT. DUMPING PROCESS ENVIRONMENT, PLEASE WAIT.
DEBUG/iX C.05.08
HPDEBUG Intrinsic at: 170.01379cc4 dump'process'environment_285+$64
File system error opening a new file. (error #1303)
END OF FILE (FSERR 0) [*DUMPIMAG]
No LIST file is currently opened. (error #1306)
$29: No LIST file is currently opened. (error #1306)
CANNOT OPEN TEMPORARY DUMP FILE. HPFOPEN STATUS=-28639089
TURBOIMAGE/XL DUMP COMPLETE.
ABORT: DBPUT ON DATABASE SMTEST.SUBSET.TURBO;
TURBOIMAGE/XL ABORTS AT PROCEDURE: $00000197; ADDRESS: $013afc40
TURBOIMAGE/XL ABORTS ON DBB CONTROL BLOCK
LOST FREE SPACE IN SET #5, ENTRY 0 ACTIVE.
+-F-I-L-E---I-N-F-O-R-M-A-T-I-O-N---D-I-S-P-L-A-Y+
! FILE NAME IS SMTEST05.SUBSET.TURBO !
! FOPTIONS: SYS,BINARY,FORMAL,F,NOCCTL,DEQ !
! NOLABEL !
! AOPTIONS: IN/OUT,NOMR,NOLOCK,SHR,NOBUF !
! NOMULTI,WAIT,NOCOPY !
! DEVICE TYPE: 3 DEVICE SUBTYPE: 8 !
! LDEV: 43 DRT: 8 UNIT: 0 !
! RECORD SIZE: 512 BLOCK SIZE: 512 (WORDS) !
! EXTENT SIZE: 38 MAX EXTENTS: 8 !
! RECPTR: 0 RECLIMIT: 75 !
! LOGCOUNT: 2 PHYSCOUNT: 0 !
! EOF AT: 75 LABEL ADDR: %00000000000 !
! FILE CODE: -401 ULABELS: 1 !
! FILE OWNER: MGR.TURBO !
! PHYSICAL STATUS: 0000000000000000 !
! ERROR NUMBER: 0 RESIDUE: 0 !
! BLOCK NUMBER: 0 NUMREC: 1 !
+------------------------------------------------+
PLEASE SEND :STORE TAPE OF PRIVILEGED FILES:
J3261121.SUBSET.TURBO
XL.PUB.SYS
TO HEWLETT-PACKARD. TO DELETE THESE FILES, LOG ON TO SUBSET.TURBO
AND RUN DBDRIVER.PUB.SYS,PURGE.
********************* NOTE **************************************
I hope they don't mean "to delete XL.PUB.SYS," one of the 2 files they
ask the user to send to HP.
**********************************************************************
FREADDIR FAILURE 0 0
TurboIMAGE Error Message:
DBU DISABLED; POTENTIAL DAMAGE; ONLY DBCLOSE ALLOWED
----------------------------------------
Next I try to add an entry into a location that alerady has an entry by
corrupting the FreeEntryChainHead.
I get (from Query):
FREADDIR FAILURE 0 0
==============| ENTRY CANNOT BE ADDED |==============
-------------------------
Finally I try to add an entry to a location above the dataset's capacity:
I get (with Query):
TurboIMAGE Error Message:
DATABASE CORRUPTION WAS DETECTED
==============| ENTRY CANNOT BE ADDED |==============
***********
Conclusions
***********
1. IMAGE does not allow me to overwrite exiting entries (entries that have
their presence bit on). Ken Paul believes this has been the case ever since
he got involved with IMAGE internals. You and I recall this was not
always the case. I could not come up with the message "lost free space"
that users tipically reported when using Query on a database that had
a bad FreeEntry list. This could be an internal Query message. I did not
get it in any of my 3 tests today.
2. Adager's consistency check detects anomalies in HWM, FreeEntryCount, or
FreeEntryChainHead. Adager's mappers detect anomalies. Adager's therapy
works as expected.
Rene.
|
