When the number of features is large, kNN should use _______.

Question

BlackTom AI · Accepted Answer

Question restatement: When the number of features is large, kNN should use _______.

Option 1: 'multiple kernel functions' — This option refers to kernel methods used in algorithms like SVM or kernelized learning. kNN is a instance-based method that relies on distance metrics between data points, not kernel functions. Using multiple kernels is not a standard or practical requirement for kNN, so this is not appropriate.

Option 2: 'the Gram matrix' — The Gram matrix captures pairwise inner products and is central to kernel methods or certain linear algorithms, not to the kNN approach itself. Since kNN computes distances directly between samples, forming a Gram matrix is not a typical step in kNN prediction, making this choice unsuitable.

Option 3: 'Manhattan distance' — This suggests using the L1 distance measure. In high-dimensional spaces (large number of features), some distances like Euclidean can become less informative due to the concentration phenomenon, where distances between points tend to become similar. The Manhattan distance can sometimes maintain discriminative power by summing absolute feature differences, and it is often preferred in high-dimensional, sparse, or noisy feature settings. This makes it a reasonable recommendation for kNN with many features.

Option 4: 'Euclidean distance' — The standard choice in many kNN applications, but in very high-dimensional spaces Euclidean distance can suffer from reduced discrimination because distances tend to concentrate; all features contribute, including noisy or irrelevant ones, which can degrade performance. Therefore, while common, it may be less robust than Manhattan distance in some high-feature scenarios.

Putting the pieces together, in the context of many features, the rationale favors a distance metric that can be more robust to high dimensionality and sparsity. Manhattan distance (L1) often performs better than Euclidean (L2) under those conditions, and it is not tied to kernel methods or Gram matrices, which are not intrinsic to kNN.

Summary of the reasoning for each option:
- Multiple kernel functions: not aligned with kNN's distance-based paradigm; kernels belong to other algorithms.
- The Gram matrix: irrelevant to basic kNN distance computations and not used for prediction in standard kNN.
- Manhattan distance: aligns with high-dimensional robustness considerations and is a sensible choice for kNN with many features.
- Euclidean distance: common but can be less robust in high-dimensional spaces due to distance concentration; may underperform compared to Manhattan in some high-feature cases.

Overall takeaway: In the setting of a large number of features, using Manhattan distance is favored for kNN because it can provide better discrimination when many dimensions are involved, whereas the other options do not fit the kNN distance-based framework or are less suitable under high dimensionality.

Machine Learning: Fundamentals and Applications Quiz 6 Kernel Method

When the number of features is large, kNN should use _______.

View Explanation

Log in for full answers

Similar Questions

Some of the formulae that we have come across in Chapters 6 & 7 are listed below. Match each of these to the name they are most commonly known by, assuming two instances a and b and an m-dimensional feature space.

Which of the following distance metrics is particularly effective for high-dimensional data but less interpretable compared to other metrics?

In a consumer society, many adults channel creativity into buying things

Economic stress and unpredictable times have resulted in a booming industry for self-help products

People born without creativity never can develop it

A product has a selling price of $20, a contribution margin ratio of 40% and fixed cost of $120,000. To make a profit of $30,000. The number of units that must be sold is: Type the number without $ and a comma. Eg: 20000

Which of the following statement regarding cost is correct:

Under the assumptions used in cost-volume-profit analysis, as the activity increases:

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler