Hugging Face's logo
Ross Wightman's picture
233 46 136

Ross Wightman

rwightman
heitorrapela's profile picture yoshitomo-matsubara's profile picture BuiDoan's profile picture
·

AI & ML interests

Computer vision, transfer learning, semi/self supervised learning, robotics.

Recent Activity

liked a model 2 days ago
microsoft/phi-4
reacted to merve's post with 🔥 2 days ago
ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 https://huggingface.co./collections/ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093 > The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️ > The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint) > The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬 the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️ > Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
posted an update 3 days ago
New `timm` 1.0.13 and `OpenCLIP` 2.30.0 releases to start the year. Both modest but worthwhile updates. `timm` added a number of new model weights, supporting loading of: * PaliGemma2 encoders (ported from https://huggingface.co./collections/google/paligemma-2-release-67500e1e1dbfdd4dee27ba48) * AIMv-2 encoders (ported from https://huggingface.co./collections/apple/aimv2-6720fe1558d94c7805f7688c) A few higher resolution 384x384 ConvNeXt-Nano ImageNet-12k pretrain & finetunes. See other changes here: https://github.com/huggingface/pytorch-image-models/releases/tag/v1.0.13 And support added in both `OpenCLIP` and `timm` for two CLIP models that were missed. The DFN L/14 is 🔥 * DFN CLIP L/14 w/ 39B samples seen - https://huggingface.co./apple/DFN2B-CLIP-ViT-L-14-39B, https://huggingface.co./timm/vit_large_patch14_clip_224.dfn2b_s39b * MetaCLIP H/14 (altogether) - https://huggingface.co./timm/vit_huge_patch14_clip_224.metaclip_altogether And last, ~70-80 models that were relying on `timm` remapping from OpenCLIP got their own timm hub instances to allow use with the upcoming Transformers `TimmWrapperModel`
View all activity

Articles

Organizations

Hugging Face's profile picture PyTorch Image Models's profile picture Spaces-explorers's profile picture Flax Community's profile picture LAION eV's profile picture Pixel Parsing's profile picture

rwightman's activity

liked a model 2 days ago
reacted to merve's post with 🔥 2 days ago
view post
Post
1589
ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093

> The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️

> The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)

> The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬

the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️

> Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
  • 1 reply
·
posted an update 3 days ago
view post
Post
1008
New timm 1.0.13 and OpenCLIP 2.30.0 releases to start the year. Both modest but worthwhile updates.

timm added a number of new model weights, supporting loading of:
* PaliGemma2 encoders (ported from google/paligemma-2-release-67500e1e1dbfdd4dee27ba48)
* AIMv-2 encoders (ported from apple/aimv2-6720fe1558d94c7805f7688c)

A few higher resolution 384x384 ConvNeXt-Nano ImageNet-12k pretrain & finetunes. See other changes here: https://github.com/huggingface/pytorch-image-models/releases/tag/v1.0.13

And support added in both OpenCLIP and timm for two CLIP models that were missed. The DFN L/14 is 🔥
* DFN CLIP L/14 w/ 39B samples seen - apple/DFN2B-CLIP-ViT-L-14-39B, timm/vit_large_patch14_clip_224.dfn2b_s39b
* MetaCLIP H/14 (altogether) - timm/vit_huge_patch14_clip_224.metaclip_altogether

And last, ~70-80 models that were relying on timm remapping from OpenCLIP got their own timm hub instances to allow use with the upcoming Transformers TimmWrapperModel
liked a model 8 days ago
liked 2 models 9 days ago
New activity in timm/mobilenetv4_conv_small.e2400_r224_in1k 11 days ago

Can't get attribute 'UniversalInvertedResidual'

2
#7 opened 11 days ago by
sddwadsa
updated 13 models 12 days ago