111

Osd111's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

liked a model 7 days ago

deepseek-ai/DeepSeek-R1

Text Generation • Updated 2 days ago • 953k • • 6.21k

New activity in NousResearch/hermes-function-calling-v1 4 months ago

account hacked

1

#7 opened 5 months ago by

Georgiagirls

New activity in NousResearch/Llama-3.2-1B 4 months ago

Can you also share the meta-llama/Llama-3.2-3B-Instruct

1

#1 opened 4 months ago by

Osd111

New activity in dreamgen/opus-v1.2-llama-3-8b 8 months ago

Curious about Fine-tuning Methods

3

#4 opened 10 months ago by

ElliottDyson

liked a model 10 months ago

NousResearch/Meta-Llama-3-8B

Text Generation • Updated Apr 30, 2024 • 25.7k • 96

reacted to giux78's post with 🚀 10 months ago

view post

Post

1780

On evaluating fine tuned 7B Italian open source LLMs I have collected many data points and I created a super simple explorative analyses. My hypothesis based on data are:

- mmlu is hard to improve when fine tuning a base model on a different language
- fine tuning also on single GPUs can improve by 5% to 10% the base model on common tasks but a lot more on specific cases with the right training time and data
- fine tuning can specialize well but at cost of loosing some foundational knowledge.

Here the data https://docs.google.com/spreadsheets/d/1MBcxy1loK8eIycZG4DN84Q2ejZ0jSjxUBgoShHDR6IY/edit?usp=sharing
Here the colab https://colab.research.google.com/drive/1ra4_skG5QYWSYOzvagOoIoj4bibQD8Gw?usp=sharing
Here an article with some considerations https://medium.com/@giuxale/an-analyses-on-italian-llms-models-evaluations-51bffe1d44d1

🚀

5

5

+

Reply