Location
- File:
chapters/nlp-book-chapter8.pdf
- Page: 60
- Section: 8.3.5.3 Rotary Positional Embedding
Problem Description
I found a notation inconsistency in the description of Rotary Positional Embeddings (RoPE).
In the text, the position is defined as $t$ ("When reaching position $t$..."); however, the formula uses $i$ inside the rotation term ( $\text{Ro}(\mathbf{x}, i\theta)$ ).
Suggested Fix
To maintain consistency with the text context, the variable in the formula should be updated from $i$ to $t$.
-
Current: $\text{Ro}(\mathbf{x}, i\theta)$
-
Correction: $\text{Ro}(\mathbf{x}, t\theta)$
This clarifies that the rotation depends on the position $t$.
Thank you!
Location
chapters/nlp-book-chapter8.pdfProblem Description$t$ ("When reaching position $t$ ..."); however, the formula uses $i$ inside the rotation term ( $\text{Ro}(\mathbf{x}, i\theta)$ ).
I found a notation inconsistency in the description of Rotary Positional Embeddings (RoPE).
In the text, the position is defined as
Suggested Fix$i$ to $t$ .
To maintain consistency with the text context, the variable in the formula should be updated from
This clarifies that the rotation depends on the position$t$ .
Thank you!