24/7 emergencies 814 777 6921812 230 6079WhatsApp 811 786 0768
Electrónica MonterreyElectrónica MonterreyLaboratorios Creativeit Talk to an engineer
Menu

How does instrument resolution enter into the uncertainty?

Instrument resolution enters the uncertainty as a Type B component with a rectangular distribution: its half-width is half the resolution, and the standard uncertainty is that half divided by the square root of three.

Why resolution counts

A digital indicator can only show values in discrete steps. The value actually present can be anywhere within half a division on either side of the displayed value, and rounding is a random error that is not corrected. For this reason it is included as an uncertainty component.

How it is calculated

If the resolution is δ, the value read differs from the real value by at most δ/2. With a rectangular distribution, the standard uncertainty is δ divided by twice the square root of three. In an analog instrument, resolution depends on the observer’s ability to estimate fractions of a division and must be judged using a documented criterion.

Two different situations in a calibration

  • Resolution of the instrument under calibration: it is included because it limits how well the indicated value can be read and, therefore, the error being determined.
  • Resolution of the standard or reference meter: it is also included, although it is normally much smaller.

Avoid counting it twice

If repeated readings vary by more than one digit, repeatability already largely includes the effect of resolution, and adding both components overestimates the uncertainty. A common practice is to take the larger of the two when they overlap. If repeated readings are identical, the observed spread is zero, but that does not mean zero uncertainty: resolution still limits what is known, and it is the dominant component.

An intermediate case

With a digital thermometer whose last digit flickers between two values, the effective resolution must be evaluated from the display noise and not only from the nominal step. Recording the mean reading and the spread makes it possible to decide which component dominates.

What an engineer sees in practice

The budget shows a resolution row with a root-three divisor. When the instrument has low resolution relative to the tolerance being verified, that row dominates and improving it (another instrument or more digits) is the most effective action. If the resolution is very fine compared with other effects, its contribution is negligible and does not change the combined uncertainty.

We do real tests, on video! Watch them on our YouTube channel →

Do you have a problem nobody has been able to measure?

Talk directly to the engineer in charge, with no middlemen.

Talk to an engineer