hf-internal-testing
/

tiny-random-HeliumForCausalLM

Text Generation

Inference Endpoints

Model card Files Files and versions Community

Xenova HF staff commited on 16 days ago

Commit

bae0c51

·

verified ·

1 Parent(s): 42b128f

Update README.md

Files changed (1) hide show

README.md +2 -0

README.md CHANGED Viewed

@@ -11,9 +11,11 @@ tags: []
 import torch
 from transformers import HeliumConfig, HeliumForCausalLM, AutoTokenizer
 model_id = 'kyutai/helium-1-preview-2b'
 config = HeliumConfig.from_pretrained(
     model_id,
     hidden_size=32,
     intermediate_size=64,
     num_attention_heads=4,

 import torch
 from transformers import HeliumConfig, HeliumForCausalLM, AutoTokenizer
 model_id = 'kyutai/helium-1-preview-2b'
 config = HeliumConfig.from_pretrained(
     model_id,
+    head_dim=8,
     hidden_size=32,
     intermediate_size=64,
     num_attention_heads=4,