tiiuae
/

Falcon3-Mamba-7B-Base

Text Generation

Inference Endpoints

Model card Files Files and versions Community

JingweiZuo commited on 22 days ago

Commit

b88d1a4

·

verified ·

1 Parent(s): 013203b

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -30,7 +30,7 @@ Falcon3-Mamba-7B-Base supports a context length up to 32K and was mainly trained
   - state dimension: 16
   - 32k context length
   - 65k vocab size
-- Continue Pretrained from Falcon Mamba 7B, with another 1500 Gigatokens of data comprising of web, code, STEM and high quality data.
 - Postrained on 1.2 million samples of STEM, conversations, code, and safety.
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0

   - state dimension: 16
   - 32k context length
   - 65k vocab size
+- Continue Pretrained from [Falcon-Mamba-7b](https://arxiv.org/abs/2410.05355), with another 1500 Gigatokens of data consisting of web, code, STEM and high quality data.
 - Postrained on 1.2 million samples of STEM, conversations, code, and safety.
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0