Which of the following is a common mistake in applying cross-validation?

Question

BlackTom AI · Accepted Answer

To approach this question, I’ll evaluate each option in terms of how cross-validation is typically applied in machine learning practice.
Option 1: 'Using validation set to select features and test performance' This is a common pitfall. In proper cross-validation, the feature selection process should be nested inside the cross-validation folds. If you use the validation set both to select features and to assess performance, you leak information from the validation data into the model, leading to overly optimistic performance estimates. This double-dipping is exactly what practitioners warn against.
Option 2: 'Using K-fold cross validation' This describes a standard, widely recommended cross-validation approach. K-fold cross-validation partitions the data into k subsets, trains on k-1 folds, and tests on the remaining fold, repeating to cover all folds. It is a valid and common use of cross-validation when applied correctly.
Option 3: 'Splitting data randomly' Random train/test splits are a basic evaluation method and can be a valid precursor to cross-validation when done carefully, but on its own it is not a cross-validation technique. The question concerns common mistakes in applying cross-validation, so simply splitting data randomly is not describing a cross-validation misuse.
Option 4: 'Applying cross-validation to the entire modeling process' If cross-validation is used to assess every step of modeling (including the final feature engineering, model selection, and parameter tuning) without proper nesting, it can lead to information leakage. However, when done with proper nesting (i.e., inner loops for tuning/feature selection and an outer loop for performance estimation), this is a correct and recommended use. The statement, taken alone, can imply misapplication if interpreted as applying CV to all steps without separation. Still, the core issue to identify in common mistakes is specifically using a validation set for both feature selection and performance estimation, which is a classic improper practice.
In summary, the options describe a spectrum of practices, with the most problematic and commonly cited error being using the validation set to both select features and test performance, because it contaminates the evaluation and inflates performance estimates. The other items outline standard, or conditional, uses of cross-validation that are acceptable when implemented correctly.

BU.232.775.52.SP25 Quiz 2

Which of the following is a common mistake in applying cross-validation?

View Explanation

Log in for full answers

Similar Questions

Why was model-building strategy #1 used to estimate generalization performance on the income data for the decision tree, logistic regression, and random forest models?

In k-fold cross-validation, what happens in each iteration?

What is one advantage of k-fold cross-validation over a single holdout set?

Which of the following is TRUE about cross validation?

Question at position 1 True/False Question: k-fold cross-validation is used to find which fold of the data set gives the best model. TrueFalse

Which is not a benefit of using cross-validation for model evaluation?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler