Regression Model Accuracy (MAE, MSE, RMSE, R-squared) Check in R


  Evaluating the model accuracy is an essential part of the process in creating machine learning models to describe how well the model is performing in its predictions. Evaluation metrics change according to the problem type. In this post, we'll briefly learn how to check the accuracy of the regression model in R.
   Linear model (regression) can be a typical example of this type of problems, and the main characteristic of the regression problem is that the targets of a dataset contain the real numbers only. Once, the model is created, we can evaluate it by checking the error rates in prediction. The errors represent how much the model is making mistakes in prediction. The basic concept of accuracy evaluation is that comparing the original target with the predicted one.

Regression model evaluation metrics

The MSE, MAE, RMSE, and R-Squared metrics are mainly used to evaluate the prediction error rates and model performance in regression analysis.
  • MAE (Mean absolute error) represents the difference between the original and predicted values extracted by averaged the absolute difference over the data set.
  • MSE (Mean Squared Error) represents the difference between the original and predicted values extracted by squared the average difference over the data set.
  • RMSE (Root Mean Squared Error) is the error rate by the square root of MSE.
  • R-squared (Coefficient of determination) represents the coefficient of how well the values fit compared to the original values. The value from 0 to 1 interpreted as percentages. The higher the value is, the better the model is.
The above metrics can be expressed,
Calculating in R

Next, we'll calculate the above metrics in R. First, we'll create sample original y - 'original' and the predicted y - 'predicted' vector data.

original = c( -2,  1, -3, 2, 3, 5, 4, 6, 5, 6, 7)
predicted = c(-1, -1, -2, 2, 3, 4, 4, 5, 5, 7, 7)

We can visualize it in a plot to check the difference visually.

x=1:length(original)
plot(x, original,pch=19, col="blue")
lines(x, predicted, col="red")
legend("topleft", legend = c("y-original", "y-predicted"),
       col = c("blue", "red"), pch = c(19,NA), lty = c(NA,1),  cex = 0.7)



Then, we'll calculate the MAE, MSE, RMSE, and R-squared with applying the above formula.

d = original-predicted
mse = mean((d)^2)
mae = mean(abs(d))
rmse = sqrt(mse)
R2 = 1-(sum((d)^2)/sum((original-mean(original))^2))

cat(" MAE:", mae, "\n", "MSE:", mse, "\n", 
    "RMSE:", rmse, "\n", "R-squared:", R2)
 MAE: 0.6363636 
 MSE: 0.8181818 
 RMSE: 0.904534 
 R-squared: 0.9173623

Next, we use R native function and the 'caret' package to calculate.

library(caret)
 
# R native funcitons
MAE(predicted, original)
MSE(predicted, original)

 [1] 0.6363636
 [1] 0.8181818
 
# caret package functions 
RMSE(predicted, original)
R2(predicted, original, form = "traditional")
[1] 0.904534
[1] 0.9173623

As you have noticed, the results are the same in both methods.
   In this post, we've learned how to calculate regression accuracy measurements MAE, MSE, RMSE, and R-squared in R. Thank you for reading!

The source code is listed below.

library(caret) original = c( -2, 1, -3, 2, 3, 5, 4, 6, 5, 6, 7) predicted = c(-1, -1, -2, 2, 3, 4, 4, 5, 5, 7, 7) x = 1:length(original) plot(x, original,pch=19, col="blue") lines(x, predicted,col="red") legend("topleft", legend = c("y-original", "y-predicted"), col=c("blue", "red"), pch=c(19,NA), lty=c(NA,1), cex = 0.7) d = original-predicted mse = mean((d)^2) mae = mean(abs(d)) rmse = sqrt(mse) R2 = 1-(sum((d)^2)/sum((original-mean(original))^2)) cat(" MAE:", mae, "\n", "MSE:", mse, "\n", "RMSE:", rmse, "\n", "R-squared:", R2) # R native funcitons fmse = MSE(predicted, original) fmae = MAE(predicted, original) # caret package functions frmse = RMSE(predicted, original) fr2 = R2(predicted, original, form = "traditional") cat(" MAE:", fmae, "\n", "MSE:", fmse, "\n", "RMSE:", frmse, "\n", "R-squared:", fr2)

No comments:
Post a Comment