> -----Original Message-----
> From: HP-3000 Systems Discussion [mailto:[log in to unmask]]On
> Behalf Of Götz Neumann

Goetz,

Thanks for calling my attention to this posting. I did not take
notice of it originally as the title did not mention anything about
the "max file per disk" limit.

Let me just say upfront that we (in the lab) are aware of this problem
and are working trying to define a solution. No details are available
today with respect to the actual solution or the timeframe for availablilty.
One of the complicating factors is that some aspects of the current design
make it more difficult to address this problem with a patch.

Ironicaly, we had a meeting today (just before I saw this message) and
discussed this issue for quite a while. I personally believe that this
issue is either number 1 or very nearly number 1 on the list of "capacity"
type issues we need to address in the very near term.

> > Neil Harvey wrote:
> > And I must agree that the HP3000 with Samba/iX is an ideal image server
> > - except for one problem.
> >
> > I understand that there is a limitation on the number of files that can
> > be held on a single HP3000 disk.
Yes

> > I'm not sure what this exact limitation is,...
The exact limit is too complex to express succintly. Goetz was basically
correct, and I have commented some below.

> > but as an example of what we
> > would require, we have a client with a HP Netserver LX/Pro with 576GB of
> > Raid disk (an 32 x 18GB Seagate farm) and it has well over 9 million
> > 60KB (average) tiff files on line. That's about 281250 files per disk
> > drive.
You will not be able to put 281,250 files on one 9 GB disk (or for
that matter on any size disk) with todays Label Table design.

> > Since these are stored as "slashed" ten digit serial numbers
> > (\\IMAGESERVER\Raid\00\09\12\34\56.tif),
[stuff snipped...]
Every file that which is not in MPE Name Space automatically have
ACDs associated with them (because there is not security matrix
for files outside MPE Name Space so ACDs are used for file security).

NOTE: For a file to be in MPE Name Space the file has to reside in
an MPE GROUP within an MPE ACCOUNT, AND the filename has to conform
to MPE filename rules (begin with alpha, only alpha and nummeric
characters, etc).

Since all of the files are stored outside of MPE Name Space they
will all have ACDs. Each ACD will also require a label table "sub entry".
Effectively this will cause each of your files to require at least
two entries. There are only 256,000 entries in the label table.
If all the files are HFS files (non-MPE name space) then the effective
limit is <= 128K (256,000/2).

In theory, the limit could actually be less than that
if some other factors conspire to require a third lable table entry
(eg. if the files were large and required "enough" extents, had user
lables ??, were NM KSAM or NM SPOOL files). I don't believe this will
be the case for your files (unless they have user lables).

> (some details snipped)
> >
> > We haven't tried pushing this structure in the Posix filespace (although
> > we have stored many thousands of images like this on our HP3000), but I
> > understand that MPE wouldn't like it if the number of files on one
> > device got too big, whether they were MPE files or Posix files.
The maximum number of files per disk is limited by the label table. The
problem is worse, not better for "Posix" files because the each have an
ACD which uses a sub-entry (causing each file to use two entires minimum).
The maximum number of entires is 256,000 regardless of disk size, or
file type (today).

> > Maybe it would work. Maybe the field(s) in the table that holds the
> > number of files per disk could be made a double :)
It is not that simple, unfortunately.

[stuff deleted...]
[Goetz wrote:]
> 2) The bigger (and growing with proportional with todays disk sizes)
> issue is the size of the MPE file label table(s). MPE has one of these
> tables per disk, and it currently can hold up to 256,000 entries.
Correct.

> Normal MPE-named files use one file label table entry per file (I think
> there are a few exceptions like spoolfiles which use a second entry).
> True POSIX namespace files have always an associated ACD which uses up
> another file label table entry, and may actually also use another entry
> for some extension. I don't remember this right now, so it is either
> 128,000 Posix files 'per disk' or around 85,000.
This is basically correct. Without getting into it in too much detail,
the bottom line is that Posix files will require two entries where
normal files require on. Other files which require two entries are
NM KSAM files and Soolfiles.

> We increased this table during MPE/XL 3.0 (when the first FL disk arrays
> came out) from 32k entries to 256k entries. It looked sufficient at that
> time, who would have guessed todays 25 or 36 GB drives.
The big problem is that todays design is not particularly scalable
for the size disks we see comming (SOON!). We need to re-think this
to be able to handle the truely huge disks we will need to support
in the very near futre.

> One caveat here would be that if we enhance this (either in a future
> release or with an enhancement patch), users would probably only be able
> to get such large tables after a NEWVOL, NEWSET or INSTALL (for LDEV 1).
Probably true...the label table will need to be rebuilt. There are, as I
said above, issues which make this potentially difficult to patch. We are
presently considering design options, but no timeframe is available for
a fix (yet).

[stuff delteted...]
> Maybe Scott will comment.
You were right. I did comment. I wish I had a concrete answer today,
but you will have to "stay tuned".

Scott