Eben Yong writes


An interesting thing occurred to us this month, wherein we were trying
to convert a very large MPE file:

record count: 102370
record length: 25158
bytes:  2575424460, or approx. 2.5 GB

to a bytestream file.  Unbeknownst to us, the tobyte program has a 2 GB
limit--or so it seems.  Using the POSIX command 'tobyte <source> <dest>'
where <source> was the original file, the <dest> file was always
produced with a bytecount of 2147483647 (which is one byte less than 2
GB).  Moreover, it did not produce a warning or an error, so we didn't
initially realize the tobyte-ed file was incomplete.  Anyone know why
this is the case and if there's a workaround (other than splitting up
the original file and converting the split files separately)?
-----------------------------------------------------
The problem is not with tobyte.  The answer is the way bytestream files are
implemented.  The file system is record oriented for all file types
including bytestream files.  Bytestream files are simulated by making 1 byte
records.  However the maximum number of records that a any file can have, a
32 bit integer, has never changed.  So the maximum number of bytes a 1
byte/record file can have is 2147483647.  This number cannot be made larger,
say to 64 bits unless all file intrinsics that reference a record pointer
are changed to allow the larger value.  Now the fact that tobyte didn't give
an error when converting too large of a file should be considered a bug.

Mike Berkowitz
Guess? Inc.

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *