Table of Contents

Fixed Point Number - Precision

About

Precision is the maximum number of significant digits (on either side of the radix point) in the fixed point notation

If a value exceeds the precision, then an application such as a database would return an error.

Example

Single number

Decimal and precision

Precision Rounded to
significant figures
Rounded to
decimal places
6 12.3450 12.345000
5 12.345 12.34500
4 12.35 12.3450
3 12.3 12.345
2 12 12.35
1 10 12.3
0 N/A 12