Publication details

Estimating the standard uncertainty contribution of the straight-line fit algorithm used to determine the position and width of a graduation line



Year of publication 2012
Type Article in Periodical
Magazine / Source Metrologia
MU Faculty or unit

Faculty of Science

Web Full Text
Field Applied statistics, operation research
Keywords straight-line fit; standard uncertainty; graduation line
Description We propose an implementation of the straight-line fit algorithm to evaluate data obtained as part of measurements of line structures, e. g. a grating line, which allows us to determine, at a conventional threshold value, the coordinates of the edges, the position and the width of its photometric profile as well as the related standard uncertainty contribution. These are derived for the (1 - alpha)100% confidence intervals with alpha is an element of [0, 1]. The implementation is demonstrated using data obtained by the PTB Nanometer Comparator (Fluegge and Koning 2001 Proc. SPIE 4401 275-83). The investigation revealed that large parts of the uncertainty of the position and the width of the line are due to the choice of the fit interval and the uncertainty of the determination of the minimum and maximum level of the photoelectric signal of the line.
Related projects: