Relationship Between Standard Deviation and Root Mean Square Error (RMSE)

Facts about RMSE:

• Includes random and systematic errors
• More useful to use as it reveals biases (systematic error)
• It tells us how accurate the data is

Facts about Standard Deviation:

• Includes only random error
• Reflects only how precise the data is
• It does not tell us how accurate the data is in the presence of biases. It only tells us how precise the data is.

$$\text{RMSE}=\text{ Standard deviation }+\text{bias}$$

Table 1 illustrates the difference between standard deviation and the RMSE in revealing the presence of biases in measurements. The table represents a vertical accuracy evaluation for points cloud derived from UAS imagery by comparing it to a higher accuracy elevation model derived from a mobile lidar mapping system. The UAS-derived elevation model needed to meet 5 cm (0.164 ft) accuracy. If we used standard deviation alone, the data would meet the specifications with a value of 0.076 ft. However, looking at the high value of 0.246 ft. (7.5 cm) of the mean, it is obvious this data set contains a bias, and the only way to catch it is by either evaluating the value of the mean or using the RMSE as the accuracy measure. The high value of the RMSE = 0.257 ft. (7.83 cm) will flag the data as not meeting specifications. The far right column contains the error values after removing the bias of 0.246 ft. (7.5 cm) from the measurements. Once we remove the bias, the values for the RMSE and the standard deviation are equal and they both meet the project accuracy specifications. Removing a bias from elevation data could be as simple as shifting the entire dataset up or down by the magnitude of the bias itself, such practice is called z-pump.

Table 1 Vertical Accuracy Tabulation of Geospatial Product

Click for a text description of Table 1.