Gavin, Mike:

 

I would assume that HP/UX has no such limitation.  Oh well.  Thanks for the info.



	-----Original Message----- 

	From: Michael Berkowitz [mailto:[log in to unmask]] 

	Sent: Fri 11/15/2002 11:47 PM 

	To: Eben Yong; [log in to unmask] ' 

	Cc: 

	Subject: RE: POSIX 'tobyte' limitation

	

	



	Eben Yong writes

	

	

	An interesting thing occurred to us this month, wherein we were trying

	to convert a very large MPE file:

	

	record count: 102370

	record length: 25158

	bytes:  2575424460, or approx. 2.5 GB

	

	to a bytestream file.  Unbeknownst to us, the tobyte program has a 2 GB

	limit--or so it seems.  Using the POSIX command 'tobyte <source> <dest>'

	where <source> was the original file, the <dest> file was always

	produced with a bytecount of 2147483647 (which is one byte less than 2

	GB).  Moreover, it did not produce a warning or an error, so we didn't

	initially realize the tobyte-ed file was incomplete.  Anyone know why

	this is the case and if there's a workaround (other than splitting up

	the original file and converting the split files separately)?

	-----------------------------------------------------

	The problem is not with tobyte.  The answer is the way bytestream files are

	implemented.  The file system is record oriented for all file types

	including bytestream files.  Bytestream files are simulated by making 1 byte

	records.  However the maximum number of records that a any file can have, a

	32 bit integer, has never changed.  So the maximum number of bytes a 1

	byte/record file can have is 2147483647.  This number cannot be made larger,

	say to 64 bits unless all file intrinsics that reference a record pointer

	are changed to allow the larger value.  Now the fact that tobyte didn't give

	an error when converting too large of a file should be considered a bug.

	

	Mike Berkowitz

	Guess? Inc.