Hi Brian,

Part of the problem that you are seeing has to do with the way you are accessing the byte stream record files. The introduction of byte stream files on MPE/iX created an interesting challenge. The behavior of byte stream files is different than the behavior that "normal" MPE applications would expect when dealing with files. The MPE file system is record based - an application could issue an FREAD request of 100 bytes, but be assured that it would be returned only a record's worth of data (assuming the record size was less than 101 bytes, in this example.) If you requested a read of 100 bytes against a byte stream file, you'd get 100 bytes which may include one or many whole or partial records.

POSIX applications expect this behavior, but existing MPE applications would not be able to correctly handle byte stream files without being rewritten. The same thing is true for POSIX applications accessing the classic MPE file and record types. Padded records and structured files would cause POSIX applications all sorts of difficulties.

The solution that we chose was to introduce the concept of a user selectable data format view. At open time, you specify whether you want to treat the file data from a record based perspective, or a byte stream perspective. FOPEN only allows record based, HPFOPEN allows you to choose via option 77. This ability to view the same data differently allows MPE/iX API applications to work on byte stream files without change, and also allows POSIX-based API applications to work on fixed and variable length record files, also without change.

The magic of providing multiple views of the same data is accomplished through emulators that the file system provides. Although very useful, there are always limits to the transparency of an emulator. When representing a byte stream file to an application as a "record based file", the variable record format was chosen, since it most closely matched the semantics of byte stream files. In byte stream files there are no blocks or maximum record sizes, so values had to be arbitrarily chosen. In this case, 8KB (8092) bytes was chosen to be reported - this was the maximum record size that the emulator would return.

So, if you open a byte stream file and request a standard record based view of the file (which is the default), FFILEINFO will report back to you that the file is variable record, with a record size of 8,192 bytes. This doesn't represent the actual physical layout of the file, but it does represent the behavior that was requested at open time, and how the file system will treat data that is read or written to the file. It is the data that MPE based applications need in order to operate correctly on the byte stream files.

FLABELINFO is a different animal. FLABELINFO retrieves file information on a file by its name - there is no open file handle/number associated with the information request, and no indication of how you want to operate on or view the data in the file. Because of this, FLABELINFO always returns information about the physical characteristics of the file. For byte stream record files the record type will always be 9 (byte stream record), and the record size will always be 1 byte.

In order to correctly build your file copies, you'll need to call FLABELINFO to get the physical file characteristics. The values that you are using are coming from FFILEINFO, after you've opened the file for record based access. After you've built the file with the correct physical characteristics, you can continue to use the record based data view to read and write data to it.

Sorry about the long-windedness, but it seemed to me that some understanding of the nature of the data view associated with byte stream files was necessary to understand where the variable record, 8,192 byte values were coming from.

Take Care,
Craig

...
> My other problem, and even bigger problem with bytestream
> files is in writing records to the file.
>
> It is defined as a one byte rec size file but FLABELINFO says
> the rec size = 8192 bytes (item 30, a U32 field)
>
> I need to be able to build a temp file as an 8192 byte rec
> size file so that I can FWRITE records to the file which may
> be an 80 byte record, 256 byte record, "n" byte record, not
> one byte which will cause a serious truncation of data.
> However, if I build the temp "bytestream" file with an 8192
> byte rec len then it is not truly a bytestream file.
>
> Any other suggestions ?
>
> As I said, I am seriously tempted to stop users from
> processing bytestream files which would be the easiest
> solution to this problem.
>
>
> Brian.
...

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *