Consider the following dataset with features X, Y and Z
[table]
X | Y | Z
Jack | 0 | A Jack | 1 | B Amy | 0 | A Amy | 1 | B Sam | 0 | B Sam | 1 | B
[/table] Let H denote the entropy function. Which of the following is true?

Question

BlackTom AI · Accepted Answer

Question restatement: We are given a dataset with features X, Y, and Z and are asked to determine which statement about entropy H is true for each feature. The dataset rows are: (Jack, 0, A), (Jack, 1, B), (Amy, 0, A), (Amy, 1, B), (Sam, 0, B), (Sam, 1, B).

Now evaluate each option option by option:

Option a: H(X) = 1.213, H(Y) = 1, H(Z) = 0.154
- This is likely incorrect because H(Y) = 1 is plausible since Y is balanced (0 and 1 appear equally often). However, H(X) is computed from the distribution of X values {Jack, Amy, Sam}, each appearing twice, which yields probabilities of 2/6 for each unique X, giving H(X) = -∑ p log2 p = -3*(2/6) log2(2/6) = log2(3) ≈ 1.585 bits, not 1.213. Therefore H(X) value here is wrong.
- H(Z) would depend on the distribution of Z values {A, B}. In the data, A occurs twice and B occurs four times, so p(A)=1/3 and p(B)=2/3, giving H(Z) = -(1/3)log2(1/3) - (2/3)log2(2/3) ≈ 0.918 bits, not 0.154. So this option is inconsistent overall.

Option b: H(X) = 1.785, H(Y) = 1, H(Z) = 0.656
- H(Y) = 1 is fine, but H(X) = 1.785 implies a distribution with more than three equally likely outcomes or uneven probabilities that yield higher entropy; here there are exactly 3 distinct X values (each occurs twice), so H(X) = log2(3) ≈ 1.585, not 1.785. This makes the option incorrect.
- H(Z) = 0.656 is close to the actual 0.918, but since H(X) is already incorrect, this option fails.

Option c: H(X) = 1.585, H(Y) = 0, H(Z) = 0.918
- H(X) = 1.585 is correct as shown above, since X has three equally likely values (Jack, Amy, Sam) each with probability 1/3.
- H(Y) = 0 would imply that Y has no variability (a degenerate distribution where all samples have the same Y value). But in the dataset Y takes both 0 and 1 equally often, giving H(Y) = 1. This makes this option incorrect.
- H(Z) = 0.918 matches the computed value for Z’s distribution (A with probability 1/3 and B with probability 2/3), so this portion is correct.

Option d: H(X) = 1.585, H(Y) = 1, H(Z) = 0.9183
- H(X) = 1.585 is correct, as explained.
- H(Y) = 1 is correct since Y has two equally likely values 0 and 1 (each occurs 3 times in 6 samples).
- H(Z) = 0.9183 is correct (exactly the binary entropy for p(A)=1/3, p(B)=2/3).
- All components align with the data, so this option is fully correct.

Summary: The key ideas are (1) X takes three equally likely values, giving H(X) = log2(3) ≈ 1.585; (2) Y is balanced with two outcomes, giving H(Y) = 1; (3) Z has probabilities 1/3 and 2/3, giving H(Z) ≈ 0.918. Among the options, only option d matches all three entropies exactly.

COMP9417-Machine Learning & Data Mining - T3 2025

Consider the following dataset with features X, Y and Z [table] X | Y | Z Jack | 0 | A Jack | 1 | B Amy | 0 | A Amy | 1 | B Sam | 0 | B Sam | 1 | B [/table] Let H denote the entropy function. Which of the following is true?

View Explanation

Log in for full answers

Similar Questions

Consider the following dataset with features X, Y and Z [table] X | Y | Z Jack | 0 | A Jack | 1 | B Amy | 0 | A Amy | 1 | B Sam | 0 | B Sam | 1 | B [/table] Let H denote the entropy function. Which of the following is true?

Consider the following dataset with features X, Y and Z [table] X | Y | Z Jack | 0 | A Jack | 1 | B Amy | 0 | A Amy | 1 | B Sam | 0 | B Sam | 1 | B [/table] Let [math: H] denote the entropy function. Which of the following is true?

Consider the following dataset with features X, Y and Z [table] X | Y | Z 0 | 0 | 0 0 | 1 | 1 1 | 0 | 0 1 | 1 | 1 2 | 0 | 1 2 | 1 | 1 [/table] Compute the entropy of [math: XYZ+1]\frac{XY}{Z+1}

Consider the following dataset with features X, Y and Z [table] X | Y | Z Jack | 0 | A Jack | 1 | B Amy | 0 | A Amy | 1 | B Sam | 0 | B Sam | 1 | B [/table] Let [math: H] denote the entropy function. Which of the following is true?

Consider the following dataset with features X, Y and Z [table] X | Y | Z 0 | 0 | 0 0 | 1 | 1 1 | 0 | 0 1 | 1 | 1 2 | 0 | 1 2 | 1 | 1 [/table] Compute the entropy of [math: XYZ+1]\frac{XY}{Z+1}

Question at position 33 True/False Question: The entropy of a homogeneous set (a set of instances with the same label) is not always zero. TrueFalse

Question at position 16 Multiple Choice Question: Entropy can be negativeis used to calculate information gainis a measure of information gainis a measure of correlation between numeric variables

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler