What is overfitting and how do you prevent it?

Overfitting happens when a model learns the noise and quirks of the training data instead of the general pattern. It scores very well on training data and poorly on new data. It is the high-variance side of the bias-variance trade-off. Underfitting is the opposite: the model is too simple to capture the pattern.

How to detect it. Compare training and validation scores. A large and growing gap is the signal. Learning curves make it visible.

How to prevent it.

• Get more, or more varied, training data, or use data augmentation.
• Use a simpler model, fewer features or feature selection.
• Apply regularization (L1, L2, dropout, weight decay).
• Use early stopping when validation loss stops improving.
• Use cross-validation to get a reliable estimate, and keep a separate test set untouched.

Never tune against the test set, or you overfit to it as well.