Ridge Regression

Bias and Variance

What exactly is overfitting? 2 factors play a role in overfitting. Bias and Variance. Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. Bias can determine if we got the relationship correct, so we fit the curve correctly. Variance is the variability of model prediction for a given data point. If a model has no variance then it will fit badly to unseen data.

There are a few ways to combat overfitting. One way is to use cross validation which will allow the model to generalize better to unseen data. Another way is to use regularization. Regularization is a technique that penalizes complexity. There is then also boosting and bagging. Bagging is used in random forests.