Which of the following approaches is best when deciding what model to use, e.g. when deciding what polynomial degree to use in regression?

Question

BlackTom AI · Accepted Answer

The question asks: Which approach is best when deciding what model to use, such as selecting the polynomial degree in regression?
Option provided: 'First, split the data into training and validation set. Train the models on the train data, and pick the one with lowest error on the validation set.'
Consideration of this option: Splitting data into training and validation sets and choosing the model based on validation error is a standard, practical approach to model selection. It helps assess how well a model generalizes to unseen data, which is the core goal beyond simply fitting the training data.
Reason this helps: Training on one portion and evaluating on a separate hold-out validation set reduces the risk of overfitting to the training data and provides an unbiased estimate of generalization performance. The model with the lowest validation error is typically preferred because it is expected to perform better on new data.
Potential caveats and refinements: Relying on a single train/validation split can lead to high variance in the chosen model if the split is not representative. In practice, one might use cross-validation (e.g., k-fold CV) to obtain a more stable estimate of generalization error across multiple splits, or use a separate test set for final evaluation after model selection. Additionally, care should be taken with the evaluation metric to ensure it aligns with the task (e.g., RMSE for regression, MAE, or others as appropriate). Regularization, model complexity limits, or information criteria could also inform model choice beyond raw validation error. Avoid relying on training error alone, as this tends to favor more complex models that overfit. The statement implicitly assumes the availability of multiple candidate models (e.g., various polynomial degrees) to compare, which is necessary for this approach to work.
In summary, the proposed method is sound as a general strategy for model selection, with the common caveats that single-split evaluation can be improved with cross-validation and that evaluation metrics and regularization considerations should guide the final choice.

DD2380 HT24 (AIHT24_2) Q4: Machine learning

Which of the following approaches is best when deciding what model to use, e.g. when deciding what polynomial degree to use in regression?

View Explanation

Log in for full answers

Similar Questions

In a 75%/25% Cross-Validation

Please select the incorrect statements about k-fold cross-validation and Leave-One-Out Cross-Validation (LOOCV):

Assuming doing a 10-fold cross validation for linear regression (Y = AX + B). It is possible that 10 different As and Bs would be generated, one for each model learned.

Cross-validation is a special case of the validation set approach.

Suppose we are going to perform K-fold cross-validation. If we make 5 folds from the data set that has 100 observations. Which of the following are true statements?

In a consumer society, many adults channel creativity into buying things

Economic stress and unpredictable times have resulted in a booming industry for self-help products

People born without creativity never can develop it

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler