Subject: | |
From: | |
Reply To: | |
Date: | Thu, 7 Dec 1995 07:31:43 -0700 |
Content-Type: | text/plain |
Parts/Attachments: |
|
|
Jerry Fochtman commands:
>Esteem Colleagues --
and then writes:
>I ran across an interesting situation the other day when using the CI to
>help perform some simple calculations. Consider the following:
>
> :SETVAR total 216340.17+61300.00+39800.00
> :SHOWVAR total
> TOTAL = 317440.1562500
>
>I can appreciate that CI wouldn't necessarily know what precision to
>display the data. However, if this is rounded to 2 places, it would be
>317440.16 .... when it should be 317440.17...
You need to do :SETVAR HPFPPENTIUM FALSE. The default has been TRUE since
5.0 Pull, the first release after the HP/Intel joint venture was announced;
this is part of the migration strategy.
When I do what you did (cutting and pasting from your email), I get:
M5(7):SETVAR total 216340.17+61300.00+39800.00
^
Unexpected character found in number. (CIERR 9810)
I get this on 5.0 Push. I don't know what OS you're running, but it's
different from mine! The symptom you're seeing can be explained if the CI
is doing fixed-point math with fractions, using five or six bits for the
fractional part.
-- Bruce
---------------------------------------------------------------------------
Bruce Toback Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc. (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142 | But ah, my foes, and oh, my friends -
Phoenix AZ 85028 | It gives a lovely light.
[log in to unmask] | -- Edna St. Vincent Millay
|
|
|