One-Way Analysis of Variance - CASE

Blood Glucose Monitoring Systems (actual data)

This case consists of comparing four brands of hand-held blood glucose monitoring devices. The goal is to determine if there are significant differences in measurement accuracy between four monitoring devices, called here Brand A, Brand B, Brand C, and Brand D. It is also of interest to attempt to identify using simple statistical analysis if a device can be considered to be significantly better than other devices under consideration, or if all devices are about the same in terms of their measurement accuracy when compared to the other brands.

The Original Data

The original data consists of four files (one file for each Brand A, B, C, and D) with four columns of data. The data include real measurements of glucose level taken from blood samples of 45 persons diagnosed with diabetes. Each subject provided two blood samples. Five measurements were taken from each blood sample: one measurement using each of the four hand-held glucose measurement devices (A,B,C,D), and one comparative measurement using a laboratory analyzer to provide a benchmark measurement. The total number of measurements for each device was hence 90. Actual brand names of the blood glucose monitoring systems are omitted at the request of the company providing the data.

The following data are available for each blood sample:

The Variables

As mentioned above, the original data consists of four files (one file for each Brand A, B, C, and D) with four columns of data. Each column is associated with a variable as follows:

Data Considered in This Case

It is clear that if the hand-held devices are accurate then the errors would be equal to zero. Hence, it was considered sufficient to concentrate the analysis on the Error % variable only.

As can be seen from the error data below, the errors (or differences in percent) are not zero, but rather significant variability appears to exist in the data:

What can we do!!

It was decided to attempt to test the hypothesis that the mean error or mean difference in measurements for each brand was the same, and that it was equal to zero, against the hypothesis that the mean error for at least two devices was not the same, and not zero. The following hypotheses were formulated:

It was also decided to find 95% confidence intervals for the error means of the hand-held devices to determine if zero error was included in the confidence interval for at least one device.

The above hypothesis test can be conducted as a One-way ANOVA. The results can then be supplemented by the error confidence intervals.

One-Way ANOVA and Confidence Intervals

The below ANOVA output and confidence intervals were obtained using Microsoft Excel Analysis Tools and statistical function add-ins.

It can be seen from the ANOVA summary that the data contain 90 error measurements for device brand A (see row labeled A-diff column labeled Count). Similarly, for devices brands B, C and D the number of error measurements is 89, 87, and 89 respectively. This suggests that there were some missing measurements for devices B, C, and D.

Note: In practical situations it is common to have missing observations and errors in data. You should always inspect your data carefully to make sure that measurements or observations are accurately recorded.

The column labeled Average gives a point estimate for the error measurement for each device. Recall again, that ideally this error should be zero. Based on the averages brand A shows the smallest error -3.10 (closest to zero), followed by brand D. Brand C shows the highest error 25.96.

The column labeled Variance gives a point estimate for the error variance for each device. Here again brand A has the smallest error variance 37.36, followed by brand D, with brand C having the highest error variance 306.89.

The ANOVA section of the table summarizes the one-way Analysis of Variance test. The Fcalc=83.82 > Fcrit=2.63 indicates that the amount of variability between the Groups (error measurement of devices) is significant. We conclude that the devices are not equally accurate, and that, based on the data, there appears to be a significant difference in measurement accuracy between the devices. Obviously, this result is very important for someone who has diabetes, and who, using a hand-held device, monitors the blood glucose content daily for optimum insulin dosage. There is no question that one would want to have the most accurate (smallest error) and reliable (smallest variance) measurements.

The lower part of the table shows the error confidence intervals both graphically and numerically for all devices. As you can see, no confidence interval includes zero at the 95% level of confidence. Device brand A has the narrowest confidence interval suggesting smallest variability of measurements (highest reliability).

These results also suggest that all devices should be calibrated. The simplest calibration, from the user perspective, means adjusting the device reading by the average error. For example, brand A readings are on an average 3.10% below the laboratory measurements. Therefore, adding 3.10% to the device measurement adjusts the average error to zero. As you can see, we have to know the amount of error before making such adjustments. If you calibrate all four devices similarly, then average errors for all devices are adjusted to zero. However, there is still error within the measurements of each device. This within treatment error is the smallest for brand A.