This is the eighth in a series of articles about MSA. The focus of this article will be on assessing the measurement reproducibility between two measurement systems that must measure the same characteristic.
Does the following scenario seem familiar? An operator performs an online test on a product. The test shows a failure to meet specifications. This is an expensive product, so the line supervisor takes the part to an online tester located on an adjacent line and retests it. This time it passes. The supervisor instructs the operator to go ahead and use the part. Both of them lose confidence in the first measurement device. But which one provided the correct result?
There are several approaches that may be used to assess the reproducibility of two measurement devices.
One approach that is commonly used when the measurement devices are fully automated is to perform a standard R&R study and replacing the Operators with the Measurement Devices. Reproducibility is now tester to tester reproducibility. The operator*part interaction becomes the tester*part interaction. While this approach does quantify the reproducibility of the two devices, it has the drawback of providing little additional information if a significant difference between the two is noted. It is also limited to fully automated gages. You do have the option of adding gage as a third factor (Parts, Operators, Gages) and analyzing the resulting designed experiment using Analysis of Variance (ANOVA).
A second approach is known as the Youden or Iso-plot. This is a very simple graphical approach. An x-y plot is constructed with equal scales, and a 45º line is drawn through the origin. Parts are selected throughout the expected measurement range. Each part is measured using both gages. Each part is then plotted on the graph using the value measured by one gage on the x-axis and the value measured by the other gage on the y-axis. Ideally, the coordinates of all parts will lie on the 45º line. If the points fall consistently above or below the line, one gage is biased in respect to the other. The Youden plot provides an excellent visual assessment of the reproducibility of the two gages, but does not provide quantifiable results. A linear regression analysis of the data used to create the Youden plot will provide quantifiable results. Ideally, the regression constant should equal zero and the slope should equal one. The regression output will provide the relationship between the two gages as well as confidence and prediction limits.
A third approach is the Bland Altman plot. Parts are selected throughout the expected measurement range. Each part is measured using both gages. Calculate the differences between the gages (Gage1 – Gage2), the mean of each pair of measurements and the mean of the differences. Plot the differences on an x-y plot with the differences on the y-axis and the mean of the paired measurements on the x-axis. Draw a horizontal line for the mean of the differences. Draw two more horizontal lines at + / - 1.96 standard deviations of the differences. Approximately 95% of the differences should fall within the limits. The mean line indicates the bias between the two gages. A trend indicates that the bias changes with the size (a linearity difference between the gages). An increase (or decrease) in the width of the pattern with size indicates that the variation of at least one of the gages is dependent on size. If the variation within the limits is of no practical importance, the two gages may be used interchangeably.
Do not overlook the differences between gages when evaluating your measurement systems. Your operators will be quick to discover gages that inconsistently accept and reject product between the gages. They will then lose confidence in them and the test itself.