HP3000-L Archives

December 1995, Week 2

HP3000-L@RAVEN.UTC.EDU

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Bruce Toback <[log in to unmask]>
Reply To:
Bruce Toback <[log in to unmask]>
Date:
Thu, 7 Dec 1995 07:31:43 -0700
Content-Type:
text/plain
Parts/Attachments:
text/plain (41 lines)
Jerry Fochtman commands:
 
>Esteem Colleagues --
 
and then writes:
>I ran across an interesting situation the other day when using the CI to
>help perform some simple calculations.  Consider the following:
>
>   :SETVAR total 216340.17+61300.00+39800.00
>   :SHOWVAR total
>    TOTAL =     317440.1562500
>
>I can appreciate that CI wouldn't necessarily know what precision to
>display the data.  However, if this is rounded to 2 places, it would be
>317440.16 .... when it should be 317440.17...
 
You need to do :SETVAR HPFPPENTIUM FALSE. The default has been TRUE since
5.0 Pull, the first release after the HP/Intel joint venture was announced;
this is part of the migration strategy.
 
When I do what you did (cutting and pasting from your email), I get:
 
   M5(7):SETVAR total 216340.17+61300.00+39800.00
                      ^
   Unexpected character found in number. (CIERR 9810)
 
I get this on 5.0 Push. I don't know what OS you're running, but it's
different from mine! The symptom you're seeing can be explained if the CI
is doing fixed-point math with fractions, using five or six bits for the
fractional part.
 
-- Bruce
 
 
---------------------------------------------------------------------------
Bruce Toback    Tel: (602) 996-8601| My candle burns at both ends;
OPT, Inc.            (800) 858-4507| It will not last the night;
11801 N. Tatum Blvd. Ste. 142      | But ah, my foes, and oh, my friends -
Phoenix AZ 85028                   | It gives a lovely light.
[log in to unmask]                 |     -- Edna St. Vincent Millay

ATOM RSS1 RSS2