meta-llama

/

Llama-3.2-11B-Vision

like 345

Follow

Meta Llama 7,965

This fix, fixes the issue with BOS not being prepended w/ AutoTokenizer

How can you test it:

 from transformers import AutoTokenizer
AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-11B-Vision").encode("hello")
[15339]
AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-11B-Vision", revision="pr/24").encode("hello")
[128000, 15339]