HP3000-L Archives

December 1995, Week 3

HP3000-L@RAVEN.UTC.EDU

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Bruce Toback <[log in to unmask]>
Reply To:
Bruce Toback <[log in to unmask]>
Date:
Sat, 16 Dec 1995 11:26:36 -0700
Content-Type:
text/plain
Parts/Attachments:
text/plain (35 lines)
Tom Emerson writes:
>>> :    :SETVAR total 216340.17+61300.00+39800.00
>>> :    :SHOWVAR total
>>> :     TOTAL =     317440.1562500
>Ok folks, since I helped write this, I think I should shed some light...
>
>In MPEX (yes, it is the VESOFT product that is the "culprit"),...
[snip]
>   * REAL NUMBERS.  Real numbers (32-bit single-precision, "HPREAL")
>are also
>     supported -- saying
>
>        %CALC 1.2 * 3.4
>
>     in MPEX will give you the right value (4.08).  The range of the
>magnitude
>     of non-zero real values is 9.9999*(10^-75) through
>1.000004*(10^76); real
>     numbers are accurate to 6.9 decimal places.  Real constants can be
                  ^^^^^^^^^^^^^^^
 
The problem is that you're formatting the number as if it were
double-precision. Since you're only doing single-precision calculations,
people get wrong answers. Change your floating point output routine to
display only single-precision and the problem should go away.
 
-- Bruce
 
---------------------------------------------------------------------------
Bruce Toback    Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc.            (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142      | But ah, my foes, and oh, my friends -
Phoenix AZ 85028                   | It gives a lovely light.
[log in to unmask]                 |     -- Edna St. Vincent Millay

ATOM RSS1 RSS2