nicolo's picture

nicolo

nicolollo

·

AI & ML interests

None yet

Recent Activity

liked a model 12 days ago

nomic-ai/nomic-embed-text-v2-moe

reacted to grimjim's post with ❤️ 13 days ago

This recent paper points to an explanation for the unreasonable effectiveness of Frankenmerges: https://huggingface.co./papers/2502.05171 Specifically, the duplication of layers in Frankenmerges serves a purpose similar to what occurs in their recurrent-depth architecture. Successful frankenmerges that operate without additional fine-tuning are able to recover or "heal" from any damage due to abrupt transitions between layer blocks. Operational replicated layer blocks can provide functional benefits grounded in latent reasoning. Frankenmerges can also result in hybrid reasoning, by splicing together the latent reasoning of different models. Back in April 2024, I was able to duplicate a few layers in the Llama 3 8B model, turning it into a 9B model, without harming benchmarks significantly, despite any transition damage. https://huggingface.co./grimjim/llama-3-experiment-v1-9B My informal experimentation suggested that latent reasoning circuits could occupy continguous stacks of 2-4 layers, though the result was highly sensitive to the choice of transition location between layers.

liked a model 14 days ago

tomg-group-umd/huginn-0125

View all activity

Organizations

nicolollo's activity

New activity in fancyfeast/llama-joycaption-alpha-two-hf-llava 5 months ago

missing file?

#1 opened 5 months ago by

New activity in huihui-ai/Qwen2-VL-7B-Instruct-abliterated 5 months ago

Request gguf

#1 opened 5 months ago by

New activity in Qwen/Qwen2.5-3B 5 months ago

License

#1 opened 5 months ago by

New activity in akjindal53244/Llama-3.1-Storm-8B 6 months ago

Languages report ?

#7 opened 6 months ago by

New activity in upstage/solar-pro-preview-instruct 6 months ago

Very nice

#7 opened 6 months ago by

New activity in Qwen/Qwen2-VL-7B-Instruct 6 months ago

LoRA Finetuning Tool for Qwen2-VL-7B in Web UI (DPO updated)

#2 opened 6 months ago by

New activity in qnguyen3/nanoLLaVA-1.5 6 months ago

This model is amazing!

#1 opened 8 months ago by

New activity in Wi-zz/joy-caption-pre-alpha 6 months ago

Bitsandbytes vs Unsloth ?

#1 opened 6 months ago by

New activity in NikshepShetty/Florence-2-DOCCI-FT 6 months ago

Target module question

#1 opened 6 months ago by

New activity in bczhou/TinyLLaVA-1.5B 6 months ago

Finetuning GPU Requirement

#1 opened 11 months ago by

New activity in cgato/Thespis-7b-v0.5-SFTTest-2Epoch-GGUF 7 months ago

random question ? >.<

#1 opened 7 months ago by

New activity in ljnlonoljpiljm/florence-2-large-interleaved-docci-keywords 7 months ago

any custom data ?

#1 opened 7 months ago by

New activity in Zhang199/TinyLLaVA-Qwen2-0.5B-SigLIP 7 months ago

Finally!

#1 opened 7 months ago by

New activity in llava-hf/llava-interleave-qwen-0.5b-hf 7 months ago

Example error or is the model DPO ?

#3 opened 7 months ago by

New activity in cognitivecomputations/dolphin-vision-7b 7 months ago

The checkpoint you are trying to load has model type `bunny-qwen` but Transformers does not recognize this architecture.

#3 opened 7 months ago by

New activity in cognitivecomputations/dolphin-vision-7b 8 months ago

Transformers doesn't support it yet?

#2 opened 8 months ago by

Missing configuration_llava_qwen2.py and configuration_llava_qwen2.py ??

#1 opened 8 months ago by

New activity in ljnlonoljpiljm/paligemma_docci_caption_finetune_v2 8 months ago

Getting worse ?

#1 opened 8 months ago by

New activity in ljnlonoljpiljm/paligemma_docci_caption_enriched_v1 8 months ago

WOW

#1 opened 8 months ago by

New activity in ljnlonoljpiljm/paligemma_docci_caption_enriched_v1-5 8 months ago

Even better

#1 opened 8 months ago by