Subject: | |
From: | |
Reply To: | COLE,GLENN (Non-HP-SantaClara,ex2) |
Date: | Fri, 9 Mar 2001 15:56:17 -0700 |
Content-Type: | text/plain |
Parts/Attachments: |
|
|
Anthony Ballo writes:
> Tried these statements seperately before with same results:
>
> _GC2 = 89.31
> _SLSGROSS = 4029.90
>
> MPEX/3000 28N90803 (c) VESOFT Inc, 1980 6.0 03:04740
> For help type 'HELP'
> %setvar _slsnet ROUND(100*( 4029.90 - 89.31 ))//100
>
> END OF PROGRAM
>
> _SLSGROSS = 4029.90
> _SLSNET = 3940.590088
Rats. Well, I *did* try it with a separate example
and it was fine:
/%setvar i ROUND(100*(3.14159 - 1.0))//100
/%showvar i
I = 2.140000
/
But, alas and alak, I ended up with the same results
as did Anthony.
Actually, this makes sense. Since MPEX is using single-precision
real numbers (HPREAL, not IEEE, at least in the older version of
MPEX I'm using), accurate to just under 7 decimal places,
it successfully rounded my example to two places.
However, while 3940.590 is the limit of the accuracy for this value,
MPEX always displays six digits after the decimal (unless told to
do otherwise, say with strwrite()), the last '088' is garbage.
("You want digits? I'll give you digits!")
Thus, _SLSNET will display the same value as above even
if you set it using
%setvar _SLSNET 3940.59
If you're dead set on sending only two digits after the decimal
to setvar, use something like this:
%setvar _slsnet ![strwrite( 4029.90 - 89.31 :9:2 )]
You can verify with %calc that the strwrite() expression results
in only two digits, but a %showvar after the above still will show
the trailing 088.
It appears MPEX does not support double-precision real numbers.
--Glenn
|
|
|