tiiuae
/

falcon-40b-instruct

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

FalconLLM commited on May 30, 2023

Commit

357dc3f

•

1 Parent(s): 38377d0

Update modelling_RW.py

Files changed (1) hide show

modelling_RW.py +1 -1

modelling_RW.py CHANGED Viewed

@@ -89,7 +89,7 @@ class RotaryEmbedding(torch.nn.Module):
     def forward(self, q, k):
         batch, seq_len, head_dim = q.shape
-        cos, sin = self.cos_sin(seq_len, q.device)
         return (q * cos) + (rotate_half(q) * sin), (k * cos) + (rotate_half(k) * sin)

     def forward(self, q, k):
         batch, seq_len, head_dim = q.shape
+        cos, sin = self.cos_sin(seq_len, q.device, q.dtype)
         return (q * cos) + (rotate_half(q) * sin), (k * cos) + (rotate_half(k) * sin)