What of the following are commonly used activation functions for neural networks?

I Exponential function.
II Rectified Linear Unit (ReLU) function.
III Sigmoid function.
IV Logarithm function.

Question

What of the following are commonly used activation functions for neural networks?

I	Exponential function.
II	Rectified Linear Unit (ReLU) function.
III	Sigmoid function.
IV	Logarithm function.

BlackTom AI · Accepted Answer

Restating the question in my own words: Which activation functions are commonly used in neural networks?

Option I: Exponential function.
- While exponential functions can appear in certain contexts (for example, as part of specific composite functions or in the computation of certain probability distributions), they are not typically used as the standard activation function in hidden layers of neural networks due to issues like exploding/vanishing gradients and lack of bounded output, which can hinder training stability.

Option II: Rectified Linear Unit (ReLU) function.
- ReLU is one of the most widely adopted activation functions in modern neural networks because it introduces nonlinearity while maintaining a simple gradient, helping mitigate the vanishing gradient problem for deep networks. It outputs zero for negative inputs and passes through positive inputs, which often leads to faster convergence in practice.

Option III: Sigmoid function.
- The sigmoid function maps inputs to the (0, 1) range and was historically popular for hidden layers and especially for output neurons in binary classification. However, it is less favored in deep networks today due to gradient saturation (gradients shrink as inputs move away from zero), which can slow learning. It remains a standard option in certain architectures or output-layer configurations.

Option IV: Logarithm function.
- The logarithm function is not used as an activation in neural networks. It is nonlinear, but its typical domain and behavior do not lend themselves well to producing useful, trainable activations across the broad range of inputs encountered during network training. Using it would also lead to undefined values for non-positive inputs and complicate gradient-based optimization.

Additional context on why some activations are preferred: Activation functions are chosen to introduce nonlinearity while maintaining favorable gradient properties for backpropagation. ReLU offers simplicity and effective gradient flow for positive values, while sigmoid provides a smooth, bounded output useful in certain probabilistic interpretations or binary classification schemes. Exponential and logarithmic activations tend to create training difficulties or instability in many standard architectures, which is why they are not common choices as general-purpose activations.

FINTECH 540.01.Fa25 Final Exam

What of the following are commonly used activation functions for neural networks? I Exponential function. II Rectified Linear Unit (ReLU) function. III Sigmoid function. IV Logarithm function.

View Explanation

Log in for full answers

Similar Questions

哪个激活函数保持恒定的导数？ Which activation function maintains constant derivatives?

Which of these plots represents a ReLU activation function?

Question at position 22 What is the purpose of the activation function in an MLP?It determines the loss function used in the MLP. It controls the learning rate of the MLP algorithm.It provides the non-linearity of the MLP network. It determines the number of hidden layers in the MLP.

Question at position 7 What activation functions are commonly used in hidden layers of neural networks? (Select all possible answers)SigmoidReLUSoftmaxTanhLinearUnit Step

Which of the following is the advantage of Leaky ReLU compared to ReLU?

What does the ReLU activation function do?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler