HP3000-L Archives

October 2008, Week 2

HP3000-L@RAVEN.UTC.EDU
Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
Re: Cobol: Puzzled by results of COBOL Compute with and w/o Rounded option
From:
Paul Raulerson <[log in to unmask]>
Reply To:
Paul Raulerson <[log in to unmask]>
Date:
Tue, 14 Oct 2008 14:55:00 -0700
Content-Type:
text/plain
Parts/Attachments:
text/plain (71 lines) 
 
HI Tony - 
  I believe you have a precision problem, and it is made a little worse by rounding. Try making all the variable the same length 
(i.e. S9(5)v9(3)).   You might want to add a couple extra places for better precision as well.  Just remember to move them to an 
appropriate display variable when your do. 

Yours,
-Paul


On Tuesday, October 14, 2008, at 01:49PM, "Tony Girgenti" <[log in to unmask]> wrote:
>Hello.
>
>Using HP3000, 928rx, MPEiX 6.0, HP COBOL II 78, Minisoft WS92 emulation.
>
>If this is true:
>
>LINE-ITEM-UNIT-PRICE		= 0	 (PIC S9(5)v99 COMP.)
>LINE-ITEM-UNIT-COST		= 0	 (PIC S9(5)v999 COMP.)
>STOP-TARIFF-RATE (STOP-NO)	= 142.79 (PIC S9(5)v99 COMP.)
>STOP-COST-RATE (STOP-NO)	= 142.79 (PIC S9(5)v999 COMP.)
>ORDER-TOTAL-WEIGHT		= 110.23 (PIC S9(5)v99 COMP.)	
>OHP				= 100.000(PIC S9(3)v999 COMP.)
>
>I don't understand why the LINE-ITEM-UNIT-PRICE and LINE-ITEM-UNIT-COST 
>yields the following results for this COMPUTE statement using ROUNDED
>
>COMPUTE LINE-ITEM-UNIT-PRICE ROUNDED = STOP-TARIFF-RATE (STOP-
>NO) / ORDER-TOTAL-WEIGHT * OHP
>ON SIZE ERROR MOVE ZEROS TO LINE-ITEM-UNIT-PRICE.
>
>Results:
>LINE-ITEM-UNIT-PRICE	= 129.50
>
>COMPUTE LINE-ITEM-UNIT-COST ROUNDED = STOP-COST-RATE (STOP-NO) / 
>ORDER-TOTAL-WEIGHT * OHP
>ON SIZE ERROR MOVE ZEROS TO LINE-ITEM-UNIT-COST.
>
>Results:
>LINE-ITEM-UNIT-COST	= 129.530
>
>Yet, if i take the ROUNDED option out of the compute, the results are:
>LINE-ITEM-UNIT-PRICE	= 129.50
>LINE-ITEM-UNIT-COST	= 129.500
>
>If i multiply this out on a calculator:
>142.79 / 110.23 * 100.000 = 129.5382
>
>My puzzlement lies in the mystery of why a ROUNDED option causes less 
>rounding than a COMPUTE without the ROUNDED option.
>
>I'm more concerned about finding out a way to get the answer of:
>LINE-ITEM-UNIT-PRICE	= 129.54
>LINE-ITEM-UNIT-COST	= 129.538
>
>I'm not so concerned about why it is happening as much as how to make it 
>give the results I want.
>
>Any help would be gratefully appreciated.
>
>Thanks,
>Tony
>
>* To join/leave the list, search archives, change list settings, *
>* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *
>
>

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *

ATOM RSS1 RSS2