Skip to main content
Renesas Electronics Europe - Knowledgebase

What are Resolution and precision of A/D converter?

Latest Updated:05/01/2006


Resolution and precision of A/D converter


Resolution and precision are the indicators used to evaluate the conversion results of an A/D converter. Both are used similarly, but they should be differentiated.

[A/D resolution]
The A/D resolution indicates how many bits of data are obtained as a result when the A/D converter converts the analog input voltage. Most A/D converters incorporated in ordinary microcontrollers have a resolution of 8 or 10 bits. Generally, the higher the resolution, the more accurate the result obtained. Note that a result with a better resolution than the resolution of the A/D converter cannot be obtained. As the resolution increases, the effect of the quantization error, which is unavoidable in an A/D converter, becomes less. However, simply making the resolution higher does not solve all problems. Greater conversion time becomes necessary as the resolution becomes higher. Furthermore, because the microcontroller's internal high-speed digital signal processing for conversion may affect analog circuits, the greater time may make it impossible to obtain results with a precision that matches the resolution. (See "Crosstalk".)

Coffee break
In the digital audio field, data is recorded on most CDs using PCM with a resolution of 16 bits, and in the case of DVD audio, data is recorded using PCM with a maximum resolution of 24 bits.
In the digital image field, on the other hand, the resolution cannot be directly compared because the image itself is compressed. It is currently common for the image output of a DAC to use 6 or 8 bits for each of R, G, and B; however, "high-end" devices may be found that use around 14 bits.

[A/D precision]
The A/D precision indicates the accuracy of the conversion result obtained by the A/D converter. A/D converters with the same resolution may not always obtain conversion results with the same precision. For example, a microcontroller's on-chip 10-bit A/D converter can produce conversion results with an error of 0.4% to 0.6%, or even as high as 1.2% in the case of low power supply voltage. This error reduces the precision accordingly.

Suitable Products