What does the term 'overfitting' refer to in machine learning?

Overfitting is a phenomenon that occurs in machine learning when a model learns the details and noise in the training dataset to the extent that it negatively impacts its performance on new data. In other words, while the model may achieve high accuracy on the training dataset by capturing complex patterns, it fails to generalize to unseen data, leading to poor performance. This often happens when the model is too complex, such as having too many parameters relative to the amount of training data. It results in models that are finely tuned to the training set, losing their ability to predict or perform well on other datasets.

The characteristic of overfitting is reflected in the distinct performance metrics when comparing a model's accuracy on training versus validation/test datasets. A well-performing model should exhibit reasonably similar performance across both, where drastic discrepancies signify potential overfitting issues.