HP3000-L Archives

November 1997, Week 2

HP3000-L@RAVEN.UTC.EDU

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Meloy_Sue <[log in to unmask]>
Reply To:
Meloy_Sue <[log in to unmask]>
Date:
Mon, 10 Nov 1997 22:36:29 GMT
Content-Type:
text/plain
Parts/Attachments:
text/plain (40 lines)
F. Alfredo Rego ([log in to unmask]) wrote:
: ...
: I think the problem here is of understanding what 'significance' means.
: It doesn't mean significant places of decimals. It means the number of
: digits, before or after an arbitrarily long string of zeros relative to
: the decimal point, that accuracy will be preserved to.

: The EMU requirement is that conversions be accurate to six significant
: digits. So if I am converting =A32,345,678,901.23 to EMUs at (say) 1.4
: EMUs per =A3, I am allowed 1,675,480,000 as a sufficient answer. Note, six
: *digits*, not six *places*.

: ...

: I have thought about having a sliding decimal point, defined in a
: separate field, thus using COBOL to fake reals (base and mantissa) in a
: rather ugly way. But even then, I'm not sure we could reliably do the
: math.....

: ...

Business BASIC/iX DECIMAL and SHORT DECIMAL data types provide floating
point decimal arithmetic.  These types are represented internally
with an exponent and mantissa.  The ranges of values supported are:

SHORT DECIMAL: [-9.99999E63, -9.99999E-63], 0, [9.99999E-63, 9.99999E63]
               Precision is exact to 6 digits.

DECIMAL: [-9.99999999999E511, -1.00000000000E-511], 0,
         [1.00000000000E-511, 9.99999999999E511]
               Precision is exact to 12 digits.

If you are using a language that does not support decimal arithmetic
and you need exact results, you may be able to use integers (~9 digits)
or floating point reals (~15 digits for double, I forget how many for
long double) to do the arithmetic.  You would have to take care of
adjusting the decimal points yourself, though.

Sue Meloy

ATOM RSS1 RSS2