ILE COBOL Programmer's Guide

Conversions and Precision

Conversion between fixed-point data formats (external decimal, packed decimal, and binary) are completed without loss of precision, as long as the target fields can contain all of the digits of the source operand.

Conversions Where Loss of Data is Possible

A loss of precision is possible in conversions between fixed-point data formats and floating-point data formats (short floating-point, long floating-point, and external floating-point). These conversions happen during arithmetic evaluations that have a mixture of both fixed-point and floating-point operands. (Because fixed-point and external floating-point items both have decimal characteristics, reference to fixed-point items in the following examples includes external floating-point items as well, unless stated otherwise.)

When converting from fixed-point to internal floating-point format, fixed-point numbers in base 10 are converted to the numbering system used internally, base 16.

Although the compiler converts short form to long form for comparisons, zeros are used for padding the short number.

When a USAGE COMP-1 data item is moved to a fixed-point data item with more than 6 digits, the fixed-point data item will receive only 6 significant digits, and the remaining digits will be zero.

Conversions that Preserve Precision

If a fixed-point data item with 6 or fewer digits is moved to a USAGE COMP-1 data item and then returned to the fixed-point data item, the original value is recovered.

If a USAGE COMP-1 data item is moved to a fixed-point data item of 6 or more digits and then returned to the USAGE COMP-1 data item, the original value is recovered.

If a fixed-point data item with 15 or fewer digits is moved to a USAGE COMP-2 data item and then returned to the fixed-point data item, the original value is recovered.

If a USAGE COMP-2 data item is moved to a fixed-point (not external floating-point) data item of 18 digits and then returned to the USAGE COMP-2 data item, the original value is recovered.

Conversions that Result In Rounding

If a USAGE COMP-1 data item, a USAGE COMP-2 data item, an external floating-point data item, or a floating-point literal is moved to a fixed-point data item, rounding occurs in the low-order position of the target data item.

If a USAGE COMP-2 data item is moved to a USAGE COMP-1 data item, rounding occurs in the low-order position of the target data item.

If a fixed-point data item is moved to an external floating-point data item where the PICTURE of the fixed-point data item contains more digit positions than the PICTURE of the external floating-point data item, rounding occurs in the low-order position of the target data item.

It is possible that when external floating-point data is DISPLAYed or ACCEPTed, or when an external floating-point literal is MOVEed to an external floating-point data item, the external floating-point data item displayed, accepted, or received can be an inaccurate value. This is because the floating-point data type is an approximation. When an external floating-point literal is accepted, displayed, or moved, it is first converted to a true floating-point value (IEEE), which can also affect its accuracy. For example, consider the following MOVE:

77 external-float-1 PIC +9(3).9(13)E+9(3).
   MOVE +123455779012.34523E+297 to external-float-1.
   DISPLAY "EXTERNAL-FLOAT-1=" external-float-1.

The displayed result of the MOVE is:

EXTERNAL-FLOAT-1=+123.4557790123452E+306


[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]