What operations are part of a standard Transformer block? (Select all that apply.)

Question

BlackTom AI · Accepted Answer

The question asks which operations are part of a standard Transformer block and specifies that it is a select-all-that-apply type.
Option 1: Layer normalization. In a typical Transformer block, after the multi-head self-attention and feed-forward sublayers, the outputs are usually passed through layer normalization (often implemented with a pre-norm or post-norm variant). This is a core component that stabilizes training and helps with gradient flow, so this option is correct.
Option 2: Convolutional layer. Standard Transformer blocks do not include a convolutional layer as a built-in component. The hallmark of the Transformer is self-attention, not convolution, so this choice is incorrect.
Option 3: Residual connection. Residual (or skip) connections are integral to Transformer blocks, allowing the sublayer input to be added to its output, which helps with optimization and information flow. This option is correct.
Option 4: Self-attention layer. The self-attention mechanism is the defining feature of the Transformer, enabling tokens to attend to other tokens within the same sequence. This is certainly part of a standard Transformer block, so this option is correct.
In summary, options 1, 3, and 4 correspond to standard components of a Transformer block, while option 2 does not. Each correct choice reflects a widely recognized aspect of the Transformer's architectural design, whereas the incorrect choice represents an element not used in the canonical block structure.

What operations are part of a standard Transformer block? (Select all that apply.)

查看解析

登录即可查看完整答案

类似问题

What is the main architectural innovation in ChatGPT that allows it to handle complex language tasks?

A team of data scientists is experiencing poor performance with their Transformer model. After inspecting their implementation, you find several suspicious design choices. Which one would you identify as the MOST problematic based on the architectural principles discussed in the lecture?

In a consumer society, many adults channel creativity into buying things

Economic stress and unpredictable times have resulted in a booming industry for self-help products

People born without creativity never can develop it

A product has a selling price of $20, a contribution margin ratio of 40% and fixed cost of $120,000. To make a profit of $30,000. The number of units that must be sold is: Type the number without $ and a comma. Eg: 20000

Which of the following statement regarding cost is correct:

Under the assumptions used in cost-volume-profit analysis, as the activity increases:

更多留学生实用工具

考试浏览器助手

风格化写作助手

论文查重助手

文献引用助手

课堂转译助手

课堂笔记助手

Quiz搜索助手

学校历年真题

智能刷题助手

智能匹配练习

希望你的学习变得更简单