Subject: | |
From: | |
Reply To: | |
Date: | Sat, 16 Dec 1995 11:26:36 -0700 |
Content-Type: | text/plain |
Parts/Attachments: |
|
|
Tom Emerson writes:
>>> : :SETVAR total 216340.17+61300.00+39800.00
>>> : :SHOWVAR total
>>> : TOTAL = 317440.1562500
>Ok folks, since I helped write this, I think I should shed some light...
>
>In MPEX (yes, it is the VESOFT product that is the "culprit"),...
[snip]
> * REAL NUMBERS. Real numbers (32-bit single-precision, "HPREAL")
>are also
> supported -- saying
>
> %CALC 1.2 * 3.4
>
> in MPEX will give you the right value (4.08). The range of the
>magnitude
> of non-zero real values is 9.9999*(10^-75) through
>1.000004*(10^76); real
> numbers are accurate to 6.9 decimal places. Real constants can be
^^^^^^^^^^^^^^^
The problem is that you're formatting the number as if it were
double-precision. Since you're only doing single-precision calculations,
people get wrong answers. Change your floating point output routine to
display only single-precision and the problem should go away.
-- Bruce
---------------------------------------------------------------------------
Bruce Toback Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc. (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142 | But ah, my foes, and oh, my friends -
Phoenix AZ 85028 | It gives a lovely light.
[log in to unmask] | -- Edna St. Vincent Millay
|
|
|