Hi Brian,

Byte stream files are the most basic of all file types. They are simply a collection of bytes of data without any structure placed on them by the file system. This is the standard file model that is used in every Unix, Linux and even Windows systems. MPE's file system has always been a structured file system, which means that the file system maintains a certain organization to the data stored in a file. The MPE file system understands things like logical records, and depending on the file type, performs interesting actions on the data (for example, Circular files, Message files, KSAM files and so on). 

To bridge the gap between standard byte stream file behavior (only the application knows the organization of data) and traditional MPE file type behavior (the file system knows what data belongs to what records), emulation type managers were created. To an MPE application, a byte stream file looks and behaves like a variable record file, even though the data is stored in a way that would allow any POSIX application to also read the same data. (POSIX applications also have emulator type managers that allow them to read fixed, variable and spool files in addition to plain byte stream files.) The way that the byte stream emulator detects record boundaries is through the use of the newline (\n) character, which is used, by convention, to separate data in Ascii text files on Unix based systems.

The underlying properties of a byte stream file is that each byte is considered its own record. In MPE file system terms, a record is the smallest unit of I/O that can be performed on a file. (You can write a partial record fixed length record, but the file system will pad it to a full record.) Since the smallest unit of I/O that can be performed on a byte stream file is a single byte, that becomes its MPE record size. In the MPE file system, the EOF tracks the number of records that are in a file. Since the record size of a byte stream file is one byte, the EOF of a byte stream file is also equal to the number of bytes in the file. This is why one 4-byte variable sized record is equal to 5 byte stream records (4 bytes of data + 1 \n character).

It's also worth noting that any file can be in any directory location and will behave the same way. (Well, almost. CM KSAM files are restricted to the MPE namespace. And of course the special files (that you don't normally see) that make up the file system root, accounts and groups are also restricted... 1 root, accounts as children of the root, groups as children of accounts. And lockwords aren't allowed outside the MPE namespace. But other than that the first sentence is true.) 

The general model that we had in architecting the whole POSIX addition was that behavior of a file does change regardless of where it is located. This was summed up in the saying, "A file is a file." So there are no such things as "MPE files" and "POSIX files". There's just files.

What does change is the way you name that file. Files in the MPE namespace can be named either through the MPE syntax (FILE.GROUP.ACCOUNT), or through the HFS syntax (/ACCOUNT/GROUP/FILE). You can also use symbolic links to create alternate names to the same file. This was summed up as a corrallary to the first saying, "But a name is not a name."

Take Care,
Craig

> -----Original Message-----
Brian asks,
> If I fcopy into a file that is not a bytestream file the eof 
> on the listf shows the correct eof (1 record). However, when 
> I do the fcopy into a bytestream file the eof isn't showing 
> the correct eof (5).
> 
> Why is this and what is the purpose of these bytestream files?
> 
> All of these files are in the MPE directory.
> 
> Brian.
...
* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *