At 05:23 PM 8/21/00 -0700, Ken Graham wrote:
>No Stan, I mean on two different machines running the same OS version, with
>the
>same licenses, the same amount of memory, the same software running, the
>same
>CPU Model, the same amount of disc free space(relatively speaking), will
>produce
>different results, in that one will work fine, and the other will produce
>errors,
>so that those sites that are developing 64 Bit code, cannot assume that
>their in house
>tests, showing that everything is working fine, should have any confidence
>in their
>test results.  Given a user with a problem, there is no guarantee that they
>will be
>able to reproduce it in house, even given the same model, memory, etc, etc,
>etc.
>This is not an opinion.  This is a fact.  I called in with a problem that HP
>could
>not reproduce on all of the machines at their disposal.  Here, it occured on
>only
>one machine, and for reasons that could not be ascertained.  HP dialed into
>our machine
>to confirm what was being seen.  The lab was not concerned that at the most
>basic
>and fundamental level, their implementation produces such inconsistent
>results,
>and knowingly.
>
>
> >> It produces inconsistent results, depending on [...] what the current
>load is

Ah yes, I've seen this as well, but it is not large-file specific.  It
is based upon how you obtain the mapped pointer used for write access
and the reason the results can vary is a timing issue.


> >huh?
>
>Because of how they implemented Large Files, normal memory mapped access
>does not
>mark a Large File page as "dirty", so writes using a Large Pointer obtained
>by any method
>other than HPFOPEN may not result in a "dirty" page being written to disk.
>Notice that
>the word is "may".  It is not guaranteed.  You can write your own routines
>to move data
>between pointers, including the use of Large Pointers, and it may appear to
>work.
>Move everything to a different machine, with a slightly different load, or
>number of CPU's,
>and the results may differ, or they may not.  It is inconsistent.  That you
>must get the
>pointer via HPFOPEN ___ONLY___ is not clearly documented.  That they provide
>a 64 Bit pointer
>via other methods allows a person of normal intelligence, to assume that
>they are in fact
>usable, when in fact they are not.


This issue is not limited to just large file pointers, as I've seen it
with long mapped pointers as well.  So this was not something that was
newly introduced as a part of this effort.  I too, received the same
feedback from an Expert Center engineer.  I argued with the engineer,
insisting that the issue be reported to the lab. However, several months
later I discovered that the call had been closed almost immediately by
the engineer and the reasons given were: a) user using privileged mode
(required to obtain pointer via other means); and b) not using HP's
documented mapped file access technique.  Using contacts I had within
CSY, I was successful to get this problem internally documented and
submitted to the file system team to consider/study.

In a separate conversation with a response center engineer I periodically
work with sometime later I learned of another person having the same
intermittent problem and pointed the RC engineer to the info on it.  The
engineer did indicate that the other person was also a vendor, so chances
are it was you.

It is unfortunately that we both experienced the same problems with a
support engineer blowing the issue aside.  On the other hand, once this
problem was highlighted to the lab team, they were extremely concerned
and several very experienced, key engineers looked into the issue.  It
is easy to condemn all of HP because of a decision made by a single
engineer.  However, I've found that having the issue revisited by
others sometimes results in a different understanding and sometimes
even a response.

A number of very intelligent engineers whom I have a great deal of respect
for worked hard at implementing the large file feature on basically a
32-bit OS. Prior to any work, they solicited feedback from a large number
of experienced technologists to craft an approach.  There were also a
number of presentations that were made at SIGSoftVend meetings were HP
engineers reviewed some of their proposed decisions/tradeoffs and solicited
feedback (did you attend these?)

There were a number of challenges involved in developing an approach
that would work on the existing architecture yet portable to the upcoming
64-bit based systems coming in the future.  In doing so, there were a
number of trade-offs that were made in order to progress the project.
Issues such as what file types to implement and how to address pointer
calculations to roll the space ID appropriately along with data movement
across space IDs.

The use of large files has not been wide-spread.  Only a handful of
vendors have implemented it and I personally, have not heard of any
end-users.  With anything new, there are bound to be some bugs that
are uncovered as more usage variation puts the design and coding to
test.  Considering that HP had to re-design a majority of virtual
storage management, secondary storage, the file system, type managers,
NMKSAM and a number of other components was a significant undertaking.
The fact that its backward compatible and that everyone I've talked
with that has moved to 6.5 has not seen any problems says heaps for
how well these core OS systems were designed.

Sure, there very well may be some other bugs that are uncovered as
usage expands.  As software developers, we all have had bugs in
our code at one time or another.  But knowing the experienced engineers
that worked so long and hard I have the utmost confidence that anything
that surfaces will be carefully examined.  To these folks, working on
MPE is not just a job, it is something they really want to do.