Which of the following is an example of a commonly used activation function in deep learning?

Question

BlackTom AI · Accepted Answer

Question restatement: Which of the following is an example of a commonly used activation function in deep learning?

Option 1: Linear function. While a linear activation preserves linearity, stacking linear activations across layers still yields a linear model and cannot capture nonlinear patterns, limiting the network's expressive power. This makes linear activations unsuitable as the activation function in hidden layers of deep networks.

Option 2: Sigmoid function. The sigmoid squashes inputs to the range (0, 1), and has historically been a standard activation used in neural networks. It introduces nonlinearity, allowing modeling of complex relationships. However, it can suffer from vanishing gradient issues for large magnitude inputs, which is a known drawback in deep architectures.

Option 3: Polynomial function. Although a polynomial form can be nonlinear, using a polynomial activation in deep networks is impractical due to issues like exploding/vanishing gradients, kinks, and increased computational load; it is not a standard activation used in practice for deep learning models.

Reasoning across options: The key criterion for an activation function in deep learning is to introduce nonlinearity while enabling effective gradient-based optimization. Linear activations fail to provide nonlinearity, so they are not suitable for deep stacks. Polynomial activations are not standard due to training and stability concerns, making them uncommon in practice. The sigmoid function, despite certain drawbacks, remains a classic and commonly referenced activation that fulfills nonlinearity and has been widely used across many architectures, particularly in earlier networks and in output layers for probabilistic interpretations.

In summary, the sigmoid function stands as the historically common activation among the options, while the other two choices do not meet the practical criteria for activation in deep learning hidden layers.

Learning AI Through Visualization 4 Module 4 Quiz

Which of the following is an example of a commonly used activation function in deep learning?

View Explanation

Log in for full answers

Similar Questions

What of the following are commonly used activation functions for neural networks? I Exponential function. II Rectified Linear Unit (ReLU) function. III Sigmoid function. IV Logarithm function.

哪个激活函数保持恒定的导数？ Which activation function maintains constant derivatives?

Which of these plots represents a ReLU activation function?

Question at position 22 What is the purpose of the activation function in an MLP?It determines the loss function used in the MLP. It controls the learning rate of the MLP algorithm.It provides the non-linearity of the MLP network. It determines the number of hidden layers in the MLP.

Question at position 7 What activation functions are commonly used in hidden layers of neural networks? (Select all possible answers)SigmoidReLUSoftmaxTanhLinearUnit Step

Which of the following is the advantage of Leaky ReLU compared to ReLU?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler