Running
332
๐
Danielus
danielus
AI & ML interests
None yet
Recent Activity
liked
a Space
6 days ago
reach-vb/2024-ai-timeline
liked
a Space
10 days ago
fffiloni/AniDoc
liked
a Space
10 days ago
lllyasviel/lumina_brush_uniform_lit
Organizations
None yet
danielus's activity
liked
a
Space
6 days ago
liked
2
Spaces
10 days ago
liked
2
models
17 days ago
reacted to
YerbaPage's
post with ๐
26 days ago
Post
1423
Curated list of **Repository-level Code Generation** papers & benchmarks! ๐ฅ
Stay ahead with the latest in:
โ Repo-level Issue Resolution
โ Repo-level Code Completion
โ Datasets & Benchmarks
๐ Check it out: https://github.com/YerbaPage/Awesome-Repo-Level-Code-Generation ๐ฅ
Stay ahead with the latest in:
โ Repo-level Issue Resolution
โ Repo-level Code Completion
โ Datasets & Benchmarks
๐ Check it out: https://github.com/YerbaPage/Awesome-Repo-Level-Code-Generation ๐ฅ
reacted to
m-ric's
post with ๐ฅ
29 days ago
Post
2202
๐ฃ๐ผ๐๐ฒ๐ป๐๐ถ๐ฎ๐น ๐ฝ๐ฎ๐ฟ๐ฎ๐ฑ๐ถ๐ด๐บ ๐๐ต๐ถ๐ณ๐ ๐ถ๐ป ๐๐๐ ๐: ๐ป๐ฒ๐ ๐ฝ๐ฎ๐ฝ๐ฒ๐ฟ ๐ฏ๐ ๐ ๐ฒ๐๐ฎ ๐ฐ๐น๐ฎ๐ถ๐บ๐ ๐๐ต๐ฎ๐ ๐๐ฒ ๐ฐ๐ฎ๐ป ๐ด๐ฒ๐ ๐ฟ๐ถ๐ฑ ๐ผ๐ณ ๐๐ผ๐ธ๐ฒ๐ป๐ถ๐๐ฒ๐ฟ๐! ๐ฅณ
Current LLMs process text by first splitting it into tokens. They use a module named "tokenizer", that -spl-it-s- th-e- te-xt- in-to- arbitrary tokens depending on a fixed dictionnary.
On the Hub you can find this dictionary in a model's files under tokenizer.json.
โก๏ธ This process is called BPE tokenization. It is suboptimal, everyone says it. It breaks text into predefined chunks that often fail to capture the nuance of language. But it has been a necessary evil in language models since their inception.
๐ฅ In Byte Latent Transformer (BLT), Meta researchers propose an elegant solution by eliminating tokenization entirely, working directly with raw bytes while maintaining efficiency through dynamic "patches."
This had been tried before with different byte-level tokenizations, but it's the first time that an architecture of this type scales as well as BPE tokenization. And it could mean a real paradigm shift! ๐๐
๐๏ธ ๐๐ฟ๐ฐ๐ต๐ถ๐๐ฒ๐ฐ๐๐๐ฟ๐ฒ:
Instead of a lightweight tokenizer, BLT has a lightweight encoder that process raw bytes into patches. Then the patches are processed by the main heavy-duty transformers as we do normally (but for patches of bytes instead of tokens), before converting back to bytes.
๐งฉ ๐๐๐ป๐ฎ๐บ๐ถ๐ฐ ๐ฃ๐ฎ๐๐ฐ๐ต๐ถ๐ป๐ด:
Instead of fixed tokens, BLT groups bytes based on their predictability (measured by entropy) - using more compute for complex sequences and efficiently handling simple ones. This allows efficient processing while maintaining byte-level understanding.
I hope this breakthrough is confirmed and we can get rid of all the tokenizer stuff, it will make model handling easier!
Read their paper here ๐ https://dl.fbaipublicfiles.com/blt/BLT__Patches_Scale_Better_Than_Tokens.pdf
Current LLMs process text by first splitting it into tokens. They use a module named "tokenizer", that -spl-it-s- th-e- te-xt- in-to- arbitrary tokens depending on a fixed dictionnary.
On the Hub you can find this dictionary in a model's files under tokenizer.json.
โก๏ธ This process is called BPE tokenization. It is suboptimal, everyone says it. It breaks text into predefined chunks that often fail to capture the nuance of language. But it has been a necessary evil in language models since their inception.
๐ฅ In Byte Latent Transformer (BLT), Meta researchers propose an elegant solution by eliminating tokenization entirely, working directly with raw bytes while maintaining efficiency through dynamic "patches."
This had been tried before with different byte-level tokenizations, but it's the first time that an architecture of this type scales as well as BPE tokenization. And it could mean a real paradigm shift! ๐๐
๐๏ธ ๐๐ฟ๐ฐ๐ต๐ถ๐๐ฒ๐ฐ๐๐๐ฟ๐ฒ:
Instead of a lightweight tokenizer, BLT has a lightweight encoder that process raw bytes into patches. Then the patches are processed by the main heavy-duty transformers as we do normally (but for patches of bytes instead of tokens), before converting back to bytes.
๐งฉ ๐๐๐ป๐ฎ๐บ๐ถ๐ฐ ๐ฃ๐ฎ๐๐ฐ๐ต๐ถ๐ป๐ด:
Instead of fixed tokens, BLT groups bytes based on their predictability (measured by entropy) - using more compute for complex sequences and efficiently handling simple ones. This allows efficient processing while maintaining byte-level understanding.
I hope this breakthrough is confirmed and we can get rid of all the tokenizer stuff, it will make model handling easier!
Read their paper here ๐ https://dl.fbaipublicfiles.com/blt/BLT__Patches_Scale_Better_Than_Tokens.pdf
- 2 replies
reacted to
julien-c's
post with ๐ฅ
about 1 month ago
Post
8237
After some heated discussion ๐ฅ, we clarify our intent re. storage limits on the Hub
TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)
docs: https://huggingface.co./docs/hub/storage-limits
We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐ฅ
cc: @reach-vb @pierric @victor and the HF team
TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)
docs: https://huggingface.co./docs/hub/storage-limits
We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐ฅ
cc: @reach-vb @pierric @victor and the HF team
- 28 replies
liked
a
model
about 1 month ago
reacted to
clem's
post with ๐
about 1 month ago
Post
4518
Six predictions for AI in 2025 (and a review of how my 2024 predictions turned out):
- There will be the first major public protest related to AI
- A big company will see its market cap divided by two or more because of AI
- At least 100,000 personal AI robots will be pre-ordered
- China will start to lead the AI race (as a consequence of leading the open-source AI race).
- There will be big breakthroughs in AI for biology and chemistry.
- We will begin to see the economic and employment growth potential of AI, with 15M AI builders on Hugging Face.
How my predictions for 2024 turned out:
- A hyped AI company will go bankrupt or get acquired for a ridiculously low price
โ (Inflexion, AdeptAI,...)
- Open-source LLMs will reach the level of the best closed-source LLMs
โ with QwQ and dozens of others
- Big breakthroughs in AI for video, time-series, biology and chemistry
โ for video ๐ดfor time-series, biology and chemistry
- We will talk much more about the cost (monetary and environmental) of AI
โ Monetary ๐ดEnvironmental (๐ข)
- A popular media will be mostly AI-generated
โ with NotebookLM by Google
- 10 millions AI builders on Hugging Face leading to no increase of unemployment
๐currently 7M of AI builders on Hugging Face
- There will be the first major public protest related to AI
- A big company will see its market cap divided by two or more because of AI
- At least 100,000 personal AI robots will be pre-ordered
- China will start to lead the AI race (as a consequence of leading the open-source AI race).
- There will be big breakthroughs in AI for biology and chemistry.
- We will begin to see the economic and employment growth potential of AI, with 15M AI builders on Hugging Face.
How my predictions for 2024 turned out:
- A hyped AI company will go bankrupt or get acquired for a ridiculously low price
โ (Inflexion, AdeptAI,...)
- Open-source LLMs will reach the level of the best closed-source LLMs
โ with QwQ and dozens of others
- Big breakthroughs in AI for video, time-series, biology and chemistry
โ for video ๐ดfor time-series, biology and chemistry
- We will talk much more about the cost (monetary and environmental) of AI
โ Monetary ๐ดEnvironmental (๐ข)
- A popular media will be mostly AI-generated
โ with NotebookLM by Google
- 10 millions AI builders on Hugging Face leading to no increase of unemployment
๐currently 7M of AI builders on Hugging Face
- 4 replies
liked
a
Space
about 2 months ago
liked
a
model
about 2 months ago
liked
a
Space
about 2 months ago
liked
a
model
2 months ago
FIM task
4
#2 opened 2 months ago
by
danielus
reacted to
TuringsSolutions's
post with ๐
2 months ago
Post
3955
Are you familiar with the difference between discrete learning and predictive learning? This distinction is exactly why LLM models are not designed to perform and execute function calls, they are not the right shape for it. LLM models are prediction machines. Function calling requires discrete learning machines. Fortunately, you can easily couple an LLM model with a discrete learning algorithm. It is beyond easy to do, you simply need to know the math to do it. Want to dive deeper into this subject? Check out this video.
https://youtu.be/wBRem2p8iPM
https://youtu.be/wBRem2p8iPM
- 8 replies
liked
a
model
2 months ago
liked
2
models
3 months ago
