A supervised machine learning task has x1 and x2 as inputs (features), and y as its output (label). In the training phase, we use an 80-20 train-test split in a 10-fold cross-validation process, with a total of 1,000 labelled data records available. Which one of the following statements is FALSE?

Question

BlackTom AI · Accepted Answer

The question asks which statement is FALSE about a supervised learning setup with x1, x2 as inputs, y as the label, using an 80-20 train-test split within a 10-fold cross-validation, with 1,000 labeled records.
Option: 'In each cross-validation iteration, we employ a different ML model and eventually select the model with the best accuracy among the 10 iterations.'
This claim implies that across the 10 folds, a different model is tried in each iteration and then the best-performing model is chosen based on accuracy. In standard cross-validation, the model type and its hyperparameters are fixed for all folds, and the cross-validated performance is used to estimate how well that fixed model would generalize. The process of selecting among multiple models or hyperparameters would require either comparing several models outside of cross-validation or using nested cross-validation to avoid bias from using the same data both to select and to evaluate models. Therefore, this statement describes a practice that is not inherent to conventional cross-validation and is typically considered incorrect in standard CV setups.

Additional nuance to consider: the phrasing about an 80-20 train-test split within a 10-fold CV is itself inconsistent with the standard CV protocol. In 10-fold cross-validation, the data are partitioned into 10 folds of roughly equal size, and in each iteration the model is trained on 9 folds and tested on the remaining 1 fold (i.e., a 90-10 split for each iteration). An explicit 80-20 split would not align with typical 10-fold CV and would suggest a misalignment between the stated CV scheme and how folds are used.

If we were to evaluate why other potential interpretations of the statement are problematic: (1) If one actually used different models per fold, that would confound the cross-validated estimate with model selection bias unless properly nested or separated. (2) Selecting the best model across folds could lead to overoptimistic performance estimates because the selection leverages the test data from the folds. (3) The intended purpose of 10-fold CV is to average performance across folds for a single model, not to pick the best of several models per fold.

In summary, the core issue is that cross-validation does not inherently entail trying a different model in each of the 10 iterations and then selecting the best one; and the stated 80-20 split per fold further conflicts with the standard 90-10 allocation used in 10-fold CV. This makes the option false in the described setup.

ENGG2112 (ND) Practice Quiz 1

A supervised machine learning task has x1 and x2 as inputs (features), and y as its output (label). In the training phase, we use an 80-20 train-test split in a 10-fold cross-validation process, with a total of 1,000 labelled data records available. Which one of the following statements is FALSE?

查看解析

登录即可查看完整答案

类似问题

Why was model-building strategy #1 used to estimate generalization performance on the income data for the decision tree, logistic regression, and random forest models?

In k-fold cross-validation, what happens in each iteration?

What is one advantage of k-fold cross-validation over a single holdout set?

Which of the following is TRUE about cross validation?

Question at position 1 True/False Question: k-fold cross-validation is used to find which fold of the data set gives the best model. TrueFalse

Which is not a benefit of using cross-validation for model evaluation?

更多留学生实用工具

考试浏览器助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单