When expressing a linear regression equation, the terms residual or error often appear at the end of the equation. But what exactly do residual and error mean? And what is the fundamental difference between the two?
In this discussion, let’s delve into the essential difference between residual and error, which is crucial to understand within the context of regression analysis. The determination of whether to calculate residual or error values depends on the research method employed.
Before we delve further into the comparison between residual and error, it’s important to have a strong understanding of the process of estimating a linear regression equation. This is because to calculate residual or error values, we need the intercept and estimation coefficients from the regression equation.
Coefficient Estimation in Linear Regression Analysis
Linear regression has become one of the commonly used analytical tools in research to study how one variable influences another. Within the framework of regression analysis, there’s an understanding that some variables act as influencers while others receive influence.
Variables that act as influencers are called independent variables, while those that receive influence are called dependent variables. In the notation of regression equations, the dependent variable is often represented as Y, while the independent variable is represented as X.
If the regression equation involves one dependent variable and one independent variable, it is known as simple linear regression. However, if there are two or more independent variables, then we have multiple linear regression.
Testing Assumptions in Linear Regression
In many research studies, linear regression is a frequently chosen method by researchers, employing the least squares method known as Ordinary Least Squares (OLS).
To ensure accurate and consistent estimation, it is important to conduct a series of tests known as classical assumption tests. Through these tests, we can verify whether all the assumptions required for linear regression using the least squares method have been adequately met.
Calculating Predicted Values of the Dependent Variable
As previously explained, calculating the residual value requires estimating the regression equation as the initial step. Today, we will focus on calculating the predicted values of the dependent variable.
These predicted values of the dependent variable will be used to determine the residual value. However, the question is: how can we calculate the predicted values of the dependent variable?
Before we can compute the predicted values of the dependent variable, the estimation results will provide us with the intercept and estimation coefficients of the independent variables.
To provide an illustration, let’s take a research example. Suppose a researcher wants to investigate how advertising costs and the number of marketing staff affect product sales.
In this case, multiple linear regression can be used, where advertising costs and the number of marketing staff become independent variables, while product sales become the dependent variable.
After obtaining the estimation results, we can formulate the appropriate regression equation as follows:
Y = 20.567 + 4.3X1 + 2.3X2
From this equation, we obtain an intercept value of 20.567, with estimation coefficients for variable X1 at 4.3 and for variable X2 at 2.3.
The next step is to calculate the predicted values of the dependent variable based on the regression equation of the estimation results. This involves using the actual values of X1 and X2 in the equation.
The predicted values for the dependent variable are calculated for each existing observation. For example, if we have 150 samples, we will apply this estimation equation to each observation, 150 times. The predicted results of the dependent variable will then be used to calculate the residual value.
Understanding Residuals
After exploring several introductions that have been presented earlier, it’s time to better understand the concept of residuals. Residuals are the differences between the actual values of the dependent variable and the predicted values of the dependent variable.
In simpler terms, residuals are the discrepancies between the observed Y value and the predicted Y value. The observed Y value is the actual data that has been observed or obtained through research. Meanwhile, the predicted Y value is the estimation result of the dependent variable based on the regression equation.
It’s important to note that residuals are used in the context of sample data. This means we are taking samples from the observed population. If we use sample data, then the difference between the actual value and the predicted value of the dependent variable is referred to as a residual. Now, what is the difference between residuals and errors?
Concept of Error in Regression Analysis
Conceptually, error refers to the difference between the actual values of the dependent variable and the predicted values of the dependent variable generated from regression estimation. If we look at the definition, there seems to be similarity between error and residual.
However, there is a fundamental difference between the two that needs to be noted. The main difference lies in whether we are using data from the population or just a sample. If using population data, the correct term is error.
Therefore, it is important for us to understand the difference between the actual values of the dependent variable and the predicted values, especially when applied to sample data and population data.
Conclusion
Based on the reviews I have presented, it can be concluded that there is a significant difference between residual and error in regression analysis. The difference primarily relates to the use of sample data or population data.
If we use sample data, the difference between the observed Y value and the predicted Y value is called a residual. Meanwhile, if we use population data, the difference between the observed Y value and the predicted Y value is called an error.
That concludes the article I have written on this occasion. Hopefully, this article can provide benefits and additional knowledge to the readers. Stay tuned for updates from Kanda Data in the next opportunity. Thank you for your attention.
