KANDA DATA

For those of you who are learning statistics, you’ve probably come across theories explaining the concepts of residual and error. At first glance, they seem almost identical, and many people even think they mean the same thing. However, in statistics, residual and error actually have different meanings.

So, what’s the fundamental difference between a residual and an error? In this article, I’m going to break it down for you.

What Are Residual and Error?

By definition, residual and error are quite similar. Both can be described as the difference between the actual observed value and the predicted value from a given model.

That being said, in statistics, residual and error are not exactly the same. We’ll get into their differences in the next section, but first, let’s build a basic understanding of the two.

Where exactly can we find the residual or error value? To make it easier to grasp, I’ll use an example from multiple linear regression.

When we conduct research and analyze data using multiple linear regression, we typically write the general equation as follows:

Y = bo + b1X1 + b2X2 + … + bnXn + e

In the equation above, Y is the dependent variable, X represents the independent variables, and bo, b1, b2, …bn are the estimated coefficients.

But here’s something you should pay attention to, at the very end of the equation, you’ll see the notation e. This is what we refer to as the residual or error, which we will discuss in detail later. This term in the regression equation represents factors that may affect the dependent variable but are not included in the regression model.

Hopefully, by now you have a clearer idea of where residual or error appears in the regression equation we’re using as an example in this article.

The Difference Between Residual and Error

Now, let’s move on to the actual difference between residual and error.

As I mentioned earlier, both residual and error are differences between the actual observation and the model’s predicted value. But the key distinction is: (a) Error refers to the measure in the population; and (b) Residual refers to the measure in the sample.

Let’s define error first. An error is the difference between the actual observed value and the value predicted by the model, calculated using all data from the population. Typically, we cannot directly observe the error because it involves the true values from the entire population, which we often don’t know.

Now, residual. A residual is the difference between the actual observed value from the sample we collect and the value predicted by the model. Unlike error, residuals can be calculated directly from the sample data in our study.

Residual as an Estimate of Error

From the definitions above, we can emphasize that residuals are for sample data, while errors are for population data.

As we all know, it’s often impractical to observe the entire population in research. Statistically, we can take a sample that represents the population, as long as we follow proper scientific sampling procedures.

In the context of the regression equation example earlier, because the sample comes from the population, the result we obtain is called a residual. Therefore, the residual we get from the sample can be considered an estimate of the error.

This aligns with the principle that a sample represents the population, and residuals represent errors.

Conclusion

After reading this article, I hope you now understand that residual and error are not exactly the same in statistics.

Although many people define them similarly, the difference is quite significant: (a) Residual:  The difference between the actual observed value and the predicted value from the model for sample data; (b) Error: the difference between the actual observed value and the predicted value from the model for population data.

That’s it for this article. I hope it’s useful and adds some new insights for those who need it. Thank you for being a loyal reader of Kanda Data. Stay tuned for more articles in the future!