In implementing CNNs, we actually implement the forwards pass as a cross-correlation. This means that for the backward pass, the gradient of the loss with respect to the input is

Question

BlackTom AI · Accepted Answer

Question restatement: In implementing CNNs, the forward pass uses a cross-correlation operation. For the backward pass, which of the following describes the gradient of the loss with respect to the input?

Option 1: cross-correlation between upstream gradient and kernel
- This would imply the input gradient is computed via a cross-correlation with the kernel. However, in backpropagation for a convolution layer, the gradient with respect to the input is not computed by a cross-correlation with the kernel; the operation is typically a convolution, not a cross-correlation, when considering how the upstream gradient interacts with the kernel.

Option 2: convolution between upstream gradient and kernel
- This aligns with the standard backpropagation result for the input: you convolve the upstream gradient (the gradient flowing from the next layer) with the kernel (often with the kernel flipped, depending on conventions). The key point is that the operation is convolution, not cross-correlation, when computing the input gradient. This option correctly identifies the general form of the calculation as a convolutional operation with the kernel involved.

Option 3: convolution between upstream gradient and input
- This would imply using the input itself as part of the operation, which is circular for computing the gradient with respect to the input. The gradient should be derived using the kernel and the upstream gradient, not the input activations themselves in this way. Therefore this is incorrect.

Option 4: cross-correlation between upstream gradient and input
- This would mix the roles of the upstream gradient and the input in a cross-correlation. The backpropagation rule for the input does not involve correlating the upstream gradient with the input directly; the standard formulation uses the kernel (often rotated) in a convolution-like operation. Hence this is not the correct interpretation.

Summary: The correct characterization is that the gradient with respect to the input is obtained by a convolution between the upstream gradient and the kernel (subject to the usual kernel flipping convention). The other options either misstate the mathematical operation (cross-correlation vs convolution) or misidentify the operands involved (kernel vs input).

CS-7643-O01, OAN, OSZ Quiz #3: Convolutional Neural Network Architectures (Lesson 6), Visualization (Lesson 7), Advanced Computer Vision Architectures (Lesson 8)

In implementing CNNs, we actually implement the forwards pass as a cross-correlation. This means that for the backward pass, the gradient of the loss with respect to the input is

查看解析

登录即可查看完整答案

类似问题

Which of the following Neural Networks is used in Machine Vision system?

哪一项陈述正确描述了卷积神经网络 (CNN) 在识别中的特征？ Which statement describes a characteristic of the Convolutional Neural Network (CNN) in recognition correctly?

This is the CNN Architecture diagram that was explained in lectures. Match A, B, C, D, and E to the correct labels below. Note: Full marks will only be awarded for getting all matches correct here. You must make selections in all drop down lists in order for this question to be counted.

Convolutional layers are

Max pooling layers are:

What is the most typical use for a convolutional neural network?

更多留学生实用工具

考试浏览器助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单