Ah Ha!!
This explains the great consternation caused on a 918 where the customer had
not deleted $stdlists for over a year. SPOOLF ;DELETE would take forever,
with me finally breaking out and trying again.
Since spoolfiles are now just special flat files, I would assume that the
same technique should be applied to them. Maybe SPOOLF could be smartened
up!!
Gary
-----Original Message-----
From: FAIRCHILD,CRAIG (HP-Cupertino,ex1) [mailto:[log in to unmask]]
Sent: Tuesday, September 25, 2001 9:43 AM
To: [log in to unmask]
Subject: Re: [HP3000-L] HELP! I've created a monster(PURGE maximum
files)
Hi Phil,
PLEASE DON'T TRY ANY OF THE EARLIER SUGGESTIONS ON THIS THREAD! They will
meet with the same end.
You must purge files in reverse alphabetical order. You can do this by
creating a command file (list the file names to a file, sort the names in
inverse order, and then add the word 'purge ' to the front of each record),
or by using the PURGEGROUP command (copy the files that you want to save to
a temporary group, PURGEGROUP the old group, rebuild that group and copy the
files back from the temporary group).
Your system abort (probably a 2200) stems from the fact that the system is
trying desperately to make sure that all the changes to your directory are
permanently recorded. To do this, MPE uses its Transaction Management (XM)
facility on all directory operations. To make sure that the directories are
not corrupted, XM takes a beginning image of the area of the directory being
changed, and after the directory operation is complete, it takes an after
image. In this way, should the system ever crash in the middle of a
directory operation, XM can always recover the directory to a consistent
state - either before or after the operation, but not in a corrupted
in-between state.
On MPE, directories are actually just special files with records for each
other file or directory that is contained in them. They stored in sorted
alphabetical order, with the disk address of the file label for that file.
Pictorially, this could look like:
rec 1: A1 -> disk address 12345
rec 2: BB -> disk address 54321
rec 3: M3 -> disk address 11221
rec 4: XY -> disk address 32132
rec 5: ZZ -> disk address 43215
Because we must keep this list of files in alphabetical order, if you add or
delete a file, the remaining contents of the file need to be "shifted" to
make room, or to compact the directory. So, if you purge file A1, XM must
record the entire contents of the directory file as the before image, and
the entire remaining file as the after image:
Before Image After Image
--------------- ---------------
rec 1: A1 rec 1: BB
rec 2: BB rec 2: M3
rec 3: M3 rec 3: XY
rec 4: XY rec 4: ZZ
rec 5: ZZ rec 5: <empty>
So purging from the top of the directory causes us to log data equal to
twice the size of the directory. If the entire group were to be purged from
the top down a total of 10+8+6+4+2 or 30 records would be logged.
Purging from the bottom of directory causes XM to log much less data, since
most of the records stay in the same place and their contents don't change.
So purging ZZ causes this amount of data to be logged:
Before Image After Image
--------------- ---------------
rec 5: ZZ rec 5: <empty>
Purging the whole group from the bottom up causes only 10 records to be
logged (5x2).
The system abort comes from the fact that more data is being logged to XM
than it can reliably record. When its logs fill completely and it can no
longer provide protection for the transactions that have been initiated, XM
will crash the system to ensure data integrity.
In general, you should try to keep your directory sizes below 1.5MB. This
will mean a different number of files for root, account, group, and HFS
directories since they all have different record sizes and structures. You
can find a directory's size by using the LISTFILE command:
: listfile /CRAIGF/PUB,2
PATH= /CRAIGF/
CODE ------------LOGICAL RECORD----------- ----SPACE---- FILENAME
SIZE TYP EOF LIMIT R/B SECTORS #X MX
0W GBD 0 0 0 32 1 * *PUB/
or my personal favorite (available on release 6.5 and beyond):
listfile /CRAIGF/PUB,11
PATH= /CRAIGF/
Access FCode RecSiz Type EOF File Limit Disk Usage Exts Name
ERWS ----- ------ ----- ------------- ------------- --------KB -----
--------
R 0 GBD 0 0 8 1 PUB/
I hope this helps, and good luck!
Craig Fairchild
MPE/iX File System Architect
> Phil Conard writes...
>
> I've created a monster.
>
> ... HP reviewed our Memory dump
> and informed us that we've reached a maximum of 70,000 files within a
> single group.
>
> We've idenfified 40,000 files to purge. However the system crashes
> whenever we attempt to purge files from this.
>
> Unfortunately, this is a 24 x 7 production machine, with peek
> usage around
> 300+ users.
>
> Things that we have tried:
> * STORE with purge option(Crashed)
> * PURGE in a command file(Crashed)
> * PURGE in a JOB(Crashed)
>
> Any ideas? Can something be done within HFS?
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
|
