Question at position 22 Multiple Choice Question: Scenario: A deadly, but curable, medical condition afflicts .01% of the population. A machine learning model uses symptoms as features and predicts this disease with an accuracy of 99.99% (this performance metric has been estimated with the cross-validation technique). Would you agree that the machine learning model is doing a good job in terms of making predictions? Yes, I would agree.We can't tell.No, I would not agree.

Question

BlackTom AI · Accepted Answer

Question restatement: A deadly but curable condition affects 0.01% of the population. A machine learning model uses symptoms as features and reports an accuracy of 99.99% via cross-validation. The prompt asks whether you would agree that the model is doing a good job in terms of making predictions. Answer options: 
- Yes, I would agree.
- We can't tell.
- No, I would not agree.

Option 1: Yes, I would agree. This line of reasoning would claim that a 99.99% accuracy is strong performance. However, because the disease is extremely rare (0.01%), a model that always predicts the negative class would already achieve 99.99% accuracy simply by avoiding false positives on everyone who does not have the disease. Therefore, this option ignores the fundamental issue of class imbalance and provides an overly optimistic view of performance. It also neglects how the model handles the minority class (the actual patients), which is precisely what matters for a deadly disease. In short, the high accuracy can be a misleading baseline rather than evidence of real predictive skill on the important cases.

Option 2: We can't tell. This choice raises a fair point about relying on accuracy alone. Yet in this scenario, the extreme class imbalance is a known factor, and accuracy alone is notoriously insufficient for judging performance on the rare diseased class. While it’s true that other information (like calibration, data leakage during cross-validation, or the distribution of predictions) could affect trust, the given numbers already signal a warning: high accuracy on an imbalanced task does not guarantee good predictive utility for identifying diseased individuals. Saying we can’t tell may be seen as overly cautious, since the misalignment between accuracy and real-world utility is well established in imbalanced classification problems. Nonetheless, this option highlights an important caveat about what accuracy measures imply.

Option 3: No, I would not agree. This option aligns with a prudent interpretation of the situation. With disease prevalence at 0.01%, the dataset is highly skewed toward the healthy class. A naive model predicting all negatives would reach 99.99% accuracy, so the reported 99.99% accuracy could reflect a baseline rather than genuine skill in detecting the diseased cases. To judge real performance, one should examine metrics that emphasize the minority class and risk, such as sensitivity (recall), precision, F1 score, and ROC-AUC, as well as calibration. If the model’s ability to correctly identify actual patients, and its rate of false alarms, are not demonstrated beyond baseline accuracy, the claim of “doing a good job” is not well-supported. This option foregrounds the contrast between overall accuracy and clinical usefulness.

Summary of the contrasts: Option 1 overstresses accuracy as a universal measure and ignores class imbalance; Option 2 foregrounds the need for more information but could be argued as a cautious stance; Option 3 correctly flags the fundamental concern that a 99.99% accuracy on an extremely imbalanced task is likely not indicative of practical predictive value for detecting the rare disease. The reasoning above shows why relying on accuracy alone is inadequate in this setting, and why the third option captures the appropriate skepticism about model usefulness in real-world prediction.

View Explanation

Log in for full answers

Similar Questions

Which of the following are appropriate ways to evaluate the performance of a Support Vector Machine (SVM) used for classification? (Select all that apply)

Which indicators are used for model evaluation in classification? (Select all that apply)

Which indicators are used for model evaluation in classification? (Select all that apply)

Match the contents to their correct groups. 1: True positive rate (TPR) 2: True negative rate (TNR) 3: Precision 4: Area under the curve (AUC)

Question at position 7 True/False Question: The accuracy metric can distinguish between the various types of misclassification errors a classifier can make. TrueFalse

Suppose you are training a classifier with 2000 data points belonging to a positive class and 250 belonging to a negative class. 1. Which performance metric would be adequate to assess the performance of the model? 2. Which performance metric would be inadequate for this data set?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler