Day 78 in MIT Sloan Fellows Class 2023, Advanced Data Analytics and Machine Learning in Finance 7, Early stopping and learning rate optimization

What is overfitting?

Overfitting occurs when the model has a high variance, i.e., the model performs well on the training data but does not perform accurately in the evaluation set.

What is Overfitting in Deep Learning [+10 Ways to Avoid It]

How to avoid overfitting?

In deep learning, there are several ways to avoid overfitting.

• Regularization
  • L1, L2
• Dropout
• Cross Validation

Basically, these techniques are either forgetting what ML learned or penalizing learning itself. 

But it requires too much time to complete ML and is inefficient to amend after completion. 

Early stopping and learning rate optmization

They are more direct way to stop overfitting. During iteration(epochs), if your performance on test data underperforms to the previous result, just stop learning.

The other one is just decreasing learning rate to moderate the overfitting.