Boosting Model Accuracy: Techniques to Improve R-Squared in Regression Models

Regression modeling is a popular statistical approach for understanding relationships between variables. The R-squared metric indicates how well the model fits the observed data, with higher values meaning more explanatory power.

Recently, I developed a regression model aiming to predict diabetes prevalence rates based on factors like physical inactivity and obesity levels. Despite iteratively tweaking the initial model, the best R-squared value achieved was 0.43. While decent, there was clear room for improvement in predictive accuracy.

In this post, I’ll share techniques I tried and am currently working on that helped significantly boost the R-squared for this project:

• Incorporating additional relevant predictors correlated with the outcome variable, such as socioeconomic status, food access, education, and healthcare availability. This allowed capturing more variance in the target variable.
• Applying transformations like logarithmic and Box-Cox to linearize non-linear relationships and correct residual patterns, improving overall model fit. Polynomial terms were also added to account for nonlinear effects.
• Using regularization techniques like ridge and lasso regression to reduce overfitting by penalizing model complexity. Lasso helped remove redundant variables.
• Carefully selecting the most predictive subset of explanatory variables through feature selection, eliminating noise-contributing variables.
• Employing ensemble modeling techniques like random forests to combine multiple models and reduce variance.