In solving multi-class classification problems with
𝑛
classes using a neural network the output of the network will be output of
𝑛
neurons (logits) which correspond to probabilities of the output to belong to the corresponding classes (one-hot encoding).

To compute the loss (cross-entropy loss) the output of neurons is evaluated by one of nonlinear activation functions tanh, sigmoid, ReLU, softmax before the loss is computed.

Question

In solving  multi-class classification problems with 
𝑛
 classes using a neural network the output of the network will be output of 
𝑛
 neurons (logits) which correspond to probabilities of the output to belong to the corresponding classes (one-hot encoding).

To compute the loss (cross-entropy loss)  the output of  neurons is evaluated by one of nonlinear activation functions tanh, sigmoid, ReLU, softmax before the loss is computed.

BlackTom AI · Accepted Answer

Begin by unpacking the statement and the standard approach to multiclass classification with neural networks.
Option analysis:
Option: The statement claims that for n-class classification the network outputs are n neurons (logits) that correspond to probabilities of belonging to each class (one-hot encoding).
- This is partially misleading: those n outputs are typically called logits, which are unnormalized scores, not probabilities. Probabilities are obtained by applying a softmax (or an equivalent) to the logits, and they sum to 1 across classes. Therefore, calling the raw outputs "probabilities" is incorrect.
Option: It then says the loss (cross-entropy) is computed after applying one of the nonlinear activations tanh, sigmoid, ReLU, or softmax to the outputs.
- In multiclass classification, the conventional pipeline uses softmax activation to convert logits into a probability distribution over classes before computing cross-entropy. ReLU, tanh, and sigmoid are not appropriate for producing a proper probability distribution across all classes in the typical cross-entropy setup. While some frameworks compute cross-entropy with logits directly (combining softmax and cross-entropy for numerical stability), the activation applied before loss is usually softmax (not the other listed activations). If you apply softmax, you obtain class probabilities rather than raw logits. If you apply any of the other activations (tanh, sigmoid, ReLU) before cross-entropy, you would not obtain a proper probability distribution across all classes, and the loss formulation would be incorrect.
- Practically, the sigmoid activation would treat each class independently (binary relevance), which is a different setup (one-vs-rest) and not the standard multi-class cross-entropy with a softmax over all classes. ReLU and tanh do not produce a valid multi-class probability distribution and can lead to incorrect loss values.
Thus, the statement contains inaccuracies:
- It conflates logits with probabilities.
- It suggests applying multiple possible activations before the loss, including ones that do not yield proper class probabilities for cross-entropy in multiclass setup, whereas softmax is the correct activation to use, if any activation is applied at all (and many implementations combine softmax with cross-entropy for numerical stability).
In summary, the claim is false because logits are not probabilities, and the typical and correct approach is to apply softmax (or a softmax-with-cross-entropy combination) rather than tanh, sigmoid, or ReLU before computing the multiclass cross-entropy loss.

AMATH 482 A Checkpoint 4 quiz

View Explanation

Log in for full answers

Similar Questions

What of the following are commonly used activation functions for neural networks? I Exponential function. II Rectified Linear Unit (ReLU) function. III Sigmoid function. IV Logarithm function.

哪个激活函数保持恒定的导数？ Which activation function maintains constant derivatives?

Which of these plots represents a ReLU activation function?

Question at position 22 What is the purpose of the activation function in an MLP?It determines the loss function used in the MLP. It controls the learning rate of the MLP algorithm.It provides the non-linearity of the MLP network. It determines the number of hidden layers in the MLP.

Question at position 7 What activation functions are commonly used in hidden layers of neural networks? (Select all possible answers)SigmoidReLUSoftmaxTanhLinearUnit Step

Which of the following is the advantage of Leaky ReLU compared to ReLU?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler