Overfitting

What is it?

Overfitting is when a model memorizes its training data — including noise and quirks — instead of learning the general pattern, so it does great in training but poorly on new data.

Explain like I'm 5

Why was it created?

It's not a feature but a fundamental failure mode. Recognizing overfitting is central to building models that actually work on unseen data.

Where is it used?

• Diagnosing poor generalization
• Model selection
• Regularization decisions
• Train/validation monitoring

Why should developers care?

It's the most common reason a model looks great in testing yet fails in production, so spotting and preventing it is a core ML skill.

How does it work?

A model flexible enough to fit any data can latch onto random noise in the training set. You detect it when training accuracy keeps rising while validation accuracy stalls or drops. Fixes reduce flexibility or add data so the model must learn general structure.

Real-world example

A fraud model hits 99% on training data but flags many legit transactions live — it memorized past cases instead of learning fraud patterns.

Common use cases

• Interpreting train vs validation gaps
• Choosing model complexity
• Deciding to add data or regularize
• Early stopping

Advantages

• (As a concept) guides good generalization
• Motivates validation and test discipline
• Drives regularization techniques
• Explains train/prod gaps

Disadvantages

• Hurts real-world performance
• Can be subtle to detect
• Tempting to ignore when training looks great
• Small datasets make it worse

When should you use it?

Always watch for it whenever you train or fine-tune a model.

When should you avoid it?

You don't want overfitting — you want to prevent it; the opposite failure (underfitting) means the model is too simple.

Alternatives

Related terms

Interview questions

Beginner

• What is overfitting in one sentence?
• How can you tell a model is overfitting?

Intermediate

• How do train and validation curves reveal overfitting?
• Name three ways to reduce it.

Senior

• How do bias-variance trade-offs relate to overfitting?
• Why does more data help, and when doesn't it?

Common misconceptions

• "High training accuracy means a good model" — it can signal memorization, not learning.
• "Overfitting only affects big models" — small datasets overfit even simple models.

Fun facts

• Dropout randomly disables neurons during training to combat overfitting.
• The opposite problem — too simple to learn the pattern — is called underfitting.

Timeline

• 1990s — Regularization and cross-validation formalized
• 2012+ — Dropout and data augmentation standard in deep learning

Learning resources

• Deep Learning (book)

Quick summary

Overfitting is when a model memorizes training data instead of the general pattern, doing well in training but poorly on new data.

Cheat sheet

• Memorizes noise, not pattern
• Train up, validation flat/down
• Fix: more data, regularize, early stop
• Opposite = underfitting

If you remember only one thing

Further reading

• Deep Learning (book)