Why resolution counts
A digital indicator can only show values in discrete steps. The value actually present can be anywhere within half a division on either side of the displayed value, and rounding is a random error that is not corrected. For this reason it is included as an uncertainty component.
How it is calculated
If the resolution is δ, the value read differs from the real value by at most δ/2. With a rectangular distribution, the standard uncertainty is δ divided by twice the square root of three. In an analog instrument, resolution depends on the observer’s ability to estimate fractions of a division and must be judged using a documented criterion.
Two different situations in a calibration
- Resolution of the instrument under calibration: it is included because it limits how well the indicated value can be read and, therefore, the error being determined.
- Resolution of the standard or reference meter: it is also included, although it is normally much smaller.
Avoid counting it twice
If repeated readings vary by more than one digit, repeatability already largely includes the effect of resolution, and adding both components overestimates the uncertainty. A common practice is to take the larger of the two when they overlap. If repeated readings are identical, the observed spread is zero, but that does not mean zero uncertainty: resolution still limits what is known, and it is the dominant component.
An intermediate case
With a digital thermometer whose last digit flickers between two values, the effective resolution must be evaluated from the display noise and not only from the nominal step. Recording the mean reading and the spread makes it possible to decide which component dominates.
What an engineer sees in practice
The budget shows a resolution row with a root-three divisor. When the instrument has low resolution relative to the tolerance being verified, that row dominates and improving it (another instrument or more digits) is the most effective action. If the resolution is very fine compared with other effects, its contribution is negligible and does not change the combined uncertainty.