PyTorchModelHubMixin: Bridging the Gap for Custom AI Models on Hugging Face
โข
11
Hafedh Hichri
not-lain
AI & ML interests
custom AI models with HF integration, multimodal rag and open-source contributions && may or may not be a huggingface fellow
Recent Activity
updated
a Space
about 4 hours ago
not-lain/image-retriever
updated
a Space
about 5 hours ago
not-lain/RAG-Chatbot
updated
a dataset
1 day ago
not-lain/lofiHipHop
Articles
Organizations
not-lain's activity
reacted to
FranckAbgrall's
post with ๐ฅ
6 days ago
Post
1036
๐ It should now be easier to identify discussions or pull requests where repository owners are participating on HF, let us know it that helps ๐ฌ๐ค
- 1 reply
reacted to
FranckAbgrall's
post with ๐ฅ๐
13 days ago
Post
1971
Hey!
โจ If you're using HF access tokens, we just released an overview of the permissions for fine-grained tokens by hovering over the badge on token settings page (org and user)
It will show the highest permission you've set for each entity ๐
โจ If you're using HF access tokens, we just released an overview of the permissions for fine-grained tokens by hovering over the badge on token settings page (org and user)
It will show the highest permission you've set for each entity ๐
reacted to
julien-c's
post with ๐ฅ
14 days ago
Post
7570
After some heated discussion ๐ฅ, we clarify our intent re. storage limits on the Hub
TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)
docs: https://huggingface.co./docs/hub/storage-limits
We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐ฅ
cc: @reach-vb @pierric @victor and the HF team
TL;DR:
- public storage is free, and (unless blatant abuse) unlimited. We do ask that you consider upgrading to PRO and/or Enterprise Hub if possible
- private storage is paid above a significant free tier (1TB if you have a paid account, 100GB otherwise)
docs: https://huggingface.co./docs/hub/storage-limits
We optimize our infrastructure continuously to scale our storage for the coming years of growth in Machine learning, to the benefit of the community ๐ฅ
cc: @reach-vb @pierric @victor and the HF team
- 28 replies
reacted to
thomwolf's
post with ๐ฅ
19 days ago
Post
1355
Most liked and most downloaded open-source AI models from 2022 to 2024
Interactive viz: https://aiworld.eu/embed/model/model/treemap
Discussion: huggingface/open-source-ai-year-in-review-2024
Interactive viz: https://aiworld.eu/embed/model/model/treemap
Discussion: huggingface/open-source-ai-year-in-review-2024
reacted to
hexgrad's
post with ๐ฅ
23 days ago
Post
2894
self.brag(): Kokoro finally got 300 votes in
Pendrokar/TTS-Spaces-Arena after
@Pendrokar
was kind enough to add it 3 weeks ago.Discounting the small sample size of votes, I think it is safe to say that hexgrad/Kokoro-TTS is currently a top 3 model among the contenders in that Arena. This is notable because:
- At 82M params, Kokoro is one of the smaller models in the Arena
- MeloTTS has 52M params
- F5 TTS has 330M params
- XTTSv2 has 467M params
- 5 replies
reacted to
loubnabnl's
post with ๐ฅ
30 days ago
Post
1632
Making SmolLM2 reproducible: open-sourcing our training & evaluation toolkit ๐ ๏ธ https://github.com/huggingface/smollm/
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
reacted to
victor's
post with ๐ฅ
30 days ago
Post
2250
Perfect example of why
Qwen/Qwen2.5-Coder-32B-Instruct is insane?
Introducing: AI Video Composer ๐ฅ
huggingface-projects/ai-video-composer
Drag and drop your assets (images/videos/audios) to create any video you want using natural language!
It works by asking the model to output a valid FFMPEG and this can be quite complex but most of the time Qwen2.5-Coder-32B gets it right (that thing is a beast). It's an update of an old project made with GPT4 and it was almost impossible to make it work with open models back then (~1.5 years ago), but not anymore, let's go open weights ๐.
Introducing: AI Video Composer ๐ฅ
huggingface-projects/ai-video-composer
Drag and drop your assets (images/videos/audios) to create any video you want using natural language!
It works by asking the model to output a valid FFMPEG and this can be quite complex but most of the time Qwen2.5-Coder-32B gets it right (that thing is a beast). It's an update of an old project made with GPT4 and it was almost impossible to make it work with open models back then (~1.5 years ago), but not anymore, let's go open weights ๐.
reacted to
luigi12345's
post with ๐
30 days ago
Post
3711
MinimalScrap
Only Free Dependencies. Save it.It is quite useful uh.
Only Free Dependencies. Save it.It is quite useful uh.
!pip install googlesearch-python requests
from googlesearch import search
import requests
query = "Glaucoma"
for url in search(f"{query} site:nih.gov filetype:pdf", 20):
if url.endswith(".pdf"):
with open(url.split("/")[-1], "wb") as f: f.write(requests.get(url).content)
print("โ
" + url.split("/")[-1])
print("Done!")
reacted to
merve's
post with โค๏ธ
about 1 month ago
Post
3100
your hugging face profile now has your recent activities ๐ค
replied to
cfahlgren1's
post
about 1 month ago
@cfahlgren1 I see you are a linkin park fan as well
reacted to
davanstrien's
post with โค๏ธ
about 1 month ago
Post
1306
huggingface.co/DIBT is dead! Long live https://huggingface.co./data-is-better-together!
We're working on some very cool projects so we're doing a bit of tidying of the Data is Better Together Hub org ๐ค
reacted to
sayakpaul's
post with ๐โค๏ธ
about 1 month ago
Post
2598
It's been a while we shipped native quantization support in
We currently support
This post is just a reminder of what's possible:
1. Loading a model with a quantization config
2. Saving a model with quantization config
3. Loading a pre-quantized model
4.
5. Training and loading LoRAs into quantized checkpoints
Docs:
https://huggingface.co./docs/diffusers/main/en/quantization/bitsandbytes
diffusers ๐งจWe currently support
bistandbytes as the official backend but using others like torchao is already very simple. This post is just a reminder of what's possible:
1. Loading a model with a quantization config
2. Saving a model with quantization config
3. Loading a pre-quantized model
4.
enable_model_cpu_offload()5. Training and loading LoRAs into quantized checkpoints
Docs:
https://huggingface.co./docs/diffusers/main/en/quantization/bitsandbytes
- 1 reply
reacted to
merve's
post with ๐๐ฅ
about 1 month ago
Post
4995
OmniVision-968M: a new local VLM for edge devices, fast & small but performant
๐จ a new vision language model with 9x less image tokens, super efficient
๐ aligned with DPO for reducing hallucinations
โก๏ธ Apache 2.0 license ๐ฅ
Demo hf.co/spaces/NexaAIDev/omnivlm-dpo-demo
Model https://huggingface.co./NexaAIDev/omnivision-968M
๐จ a new vision language model with 9x less image tokens, super efficient
๐ aligned with DPO for reducing hallucinations
โก๏ธ Apache 2.0 license ๐ฅ
Demo hf.co/spaces/NexaAIDev/omnivlm-dpo-demo
Model https://huggingface.co./NexaAIDev/omnivision-968M
- 4 replies
reacted to
BlinkDL's
post with ๐ฅ
about 1 month ago
Post
3938
RWKV-6-world-v3 (+3.1T tokens) is our best multilingual 7B model as of now:
BlinkDL/rwkv-6-world
It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).
RWKV-7-world-v4 soon :)
It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).
RWKV-7-world-v4 soon :)
reacted to
m-ric's
post with โค๏ธโค๏ธ๐ฅ
about 1 month ago
Post
3777
๐ง๐ต๐ฒ ๐ป๐ฒ๐
๐ ๐ฏ๐ถ๐ด ๐๐ผ๐ฐ๐ถ๐ฎ๐น ๐ป๐ฒ๐๐๐ผ๐ฟ๐ธ ๐ถ๐ ๐ป๐ผ๐ ๐ฆ, ๐ถ๐'๐ ๐๐๐ฏ ๐ฃ๐ผ๐๐๐! [INSERT STONKS MEME WITH LASER EYES]
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
โ๏ธ Computed with the great dataset maxiw/hf-posts
โ๏ธ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
See below: I got 105k impressions since regularly posting Hub Posts, coming close to my 275k on Twitter!
โ๏ธ Computed with the great dataset maxiw/hf-posts
โ๏ธ Thanks to Qwen2.5-Coder-32B for showing me how to access dict attributes in a SQL request!
cc @merve who's far in front of me
- 9 replies