In implementing CNNs, we actually implement the forwards pass as a cross-correlation. This means that for the backward pass, the gradient of the loss with respect to the input is

Question

BlackTom AI · Accepted Answer

Question restatement: In implementing CNNs, the forward pass uses a cross-correlation operation. For the backward pass, which of the following describes the gradient of the loss with respect to the input?

Option 1: cross-correlation between upstream gradient and kernel
- This would imply the input gradient is computed via a cross-correlation with the kernel. However, in backpropagation for a convolution layer, the gradient with respect to the input is not computed by a cross-correlation with the kernel; the operation is typically a convolution, not a cross-correlation, when considering how the upstream gradient interacts with the kernel.

Option 2: convolution between upstream gradient and kernel
- This aligns with the standard backpropagation result for the input: you convolve the upstream gradient (the gradient flowing from the next layer) with the kernel (often with the kernel flipped, depending on conventions). The key point is that the operation is convolution, not cross-correlation, when computing the input gradient. This option correctly identifies the general form of the calculation as a convolutional operation with the kernel involved.

Option 3: convolution between upstream gradient and input
- This would imply using the input itself as part of the operation, which is circular for computing the gradient with respect to the input. The gradient should be derived using the kernel and the upstream gradient, not the input activations themselves in this way. Therefore this is incorrect.

Option 4: cross-correlation between upstream gradient and input
- This would mix the roles of the upstream gradient and the input in a cross-correlation. The backpropagation rule for the input does not involve correlating the upstream gradient with the input directly; the standard formulation uses the kernel (often rotated) in a convolution-like operation. Hence this is not the correct interpretation.

Summary: The correct characterization is that the gradient with respect to the input is obtained by a convolution between the upstream gradient and the kernel (subject to the usual kernel flipping convention). The other options either misstate the mathematical operation (cross-correlation vs convolution) or misidentify the operands involved (kernel vs input).

CS-7643-O01, OAN, OSZ Quiz #3: Convolutional Neural Network Architectures (Lesson 6), Visualization (Lesson 7), Advanced Computer Vision Architectures (Lesson 8)

In implementing CNNs, we actually implement the forwards pass as a cross-correlation. This means that for the backward pass, the gradient of the loss with respect to the input is

View Explanation

Log in for full answers

Similar Questions

Which of the following Neural Networks is used in Machine Vision system?

哪一项陈述正确描述了卷积神经网络 (CNN) 在识别中的特征？ Which statement describes a characteristic of the Convolutional Neural Network (CNN) in recognition correctly?

This is the CNN Architecture diagram that was explained in lectures. Match A, B, C, D, and E to the correct labels below. Note: Full marks will only be awarded for getting all matches correct here. You must make selections in all drop down lists in order for this question to be counted.

Convolutional layers are

Max pooling layers are:

What is the most typical use for a convolutional neural network?

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler