Joseph717171/Hermes-3-Llama-3.1-8B-OQ8_0-F32.EF32.IQ4_K-Q8_0-GGUF
Updated
ā¢
758
ā¢
2
Joseph
Joseph717171
AI & ML interests
None yet
Recent Activity
updated
a model
about 13 hours ago
Joseph717171/Hermes-3-Llama-3.1-8B-OQ8_0-F32.EF32.IQ4_K-Q8_0-GGUF
updated
a model
1 day ago
Joseph717171/Models
new activity
2 days ago
Undi95/Phi4-abliterated:Awesome work, Undi95! This looks great!
Organizations
Joseph717171's activity
updated
a
model
about 13 hours ago
updated
a
model
1 day ago
Awesome work, Undi95! This looks great!
1
#1 opened 2 days ago
by
Joseph717171
liked
a
model
2 days ago
liked
a
model
3 days ago
upvoted
a
paper
3 days ago
liked
2
models
4 days ago
Great Model Base for ERP!
#1 opened 6 days ago
by
Joseph717171
liked
a
model
6 days ago
liked
a
model
10 days ago
liked
a
model
18 days ago
upvoted
a
paper
18 days ago
liked
2
models
20 days ago
reacted to
singhsidhukuldeep's
post with šš§
21 days ago
Post
3631
Exciting breakthrough in AI:
@Meta
's new Byte Latent Transformer (BLT) revolutionizes language models by eliminating tokenization!
The BLT architecture introduces a groundbreaking approach that processes raw bytes instead of tokens, achieving state-of-the-art performance while being more efficient and robust. Here's what makes it special:
>> Key Innovations
Dynamic Patching: BLT groups bytes into variable-sized patches based on entropy, allocating more compute power where the data is more complex. This results in up to 50% fewer FLOPs during inference compared to traditional token-based models.
Three-Component Architecture:
ā¢ Lightweight Local Encoder that converts bytes to patch representations
ā¢ Powerful Global Latent Transformer that processes patches
ā¢ Local Decoder that converts patches back to bytes
>> Technical Advantages
ā¢ Matches performance of Llama 3 at 8B parameters while being more efficient
ā¢ Superior handling of non-English languages and rare character sequences
ā¢ Remarkable 99.9% accuracy on spelling tasks
ā¢ Better scaling properties than token-based models
>> Under the Hood
The system uses an entropy model to determine patch boundaries, cross-attention mechanisms for information flow, and hash n-gram embeddings for improved representation. The architecture allows simultaneous scaling of both patch and model size while maintaining fixed inference costs.
This is a game-changer for multilingual AI and could reshape how we build future language models. Excited to see how this technology evolves!
The BLT architecture introduces a groundbreaking approach that processes raw bytes instead of tokens, achieving state-of-the-art performance while being more efficient and robust. Here's what makes it special:
>> Key Innovations
Dynamic Patching: BLT groups bytes into variable-sized patches based on entropy, allocating more compute power where the data is more complex. This results in up to 50% fewer FLOPs during inference compared to traditional token-based models.
Three-Component Architecture:
ā¢ Lightweight Local Encoder that converts bytes to patch representations
ā¢ Powerful Global Latent Transformer that processes patches
ā¢ Local Decoder that converts patches back to bytes
>> Technical Advantages
ā¢ Matches performance of Llama 3 at 8B parameters while being more efficient
ā¢ Superior handling of non-English languages and rare character sequences
ā¢ Remarkable 99.9% accuracy on spelling tasks
ā¢ Better scaling properties than token-based models
>> Under the Hood
The system uses an entropy model to determine patch boundaries, cross-attention mechanisms for information flow, and hash n-gram embeddings for improved representation. The architecture allows simultaneous scaling of both patch and model size while maintaining fixed inference costs.
This is a game-changer for multilingual AI and could reshape how we build future language models. Excited to see how this technology evolves!
- 2 replies
reacted to
davanstrien's
post with š„
22 days ago
Post
1761
Introducing FineWeb-C šš, a community-built dataset for improving language models in ALL languages.
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! š
data-is-better-together/fineweb-c
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! š
data-is-better-together/fineweb-c