Hugging Face's logo

meditsolutions
/
MSH-Lite-7B-v1-Bielik-v2.3-Instruct-Llama-Prune

Text Generation
Safetensors
Polish
English
mistral
medit-lite
model-pruning
conversational
Model card Files Community
MSH-Lite-7B-v1-Bielik-v2.3-Instruct-Llama-Prune / README.md
mkurman's picture
mkurman
Update README.md
8fbaa3b verified
preview code
|
raw
history blame contribute delete
1.98 kB
metadata
license: apache-2.0
base_model:
  - meditsolutions/MSH-v1-Bielik-v2.3-Instruct-MedIT-merge
  - speakleash/Bielik-11B-v2.3-Instruct
pipeline_tag: text-generation
tags:
  - medit-lite
  - model-pruning
  - text-generation
language:
  - pl
  - en
MSH-Lite

Marsh Harrier Lite (MSH-Lite)

Marsh Harrier Lite (MSH-Lite) is a compact, efficient version of the MedIT Solutions MSH-v1-Bielik-v2.3-Instruct-MedIT-merge model, reduced to 7 billion parameters using advanced pruning techniques. This pruning retains the core functionality and efficiency of the original model while optimizing for reduced computational resource usage.

Key Features:

  • Pruned Model: Reduced from 11B to 7B parameters using the pruning method based on the MedIT Solutions LLaMA pruning framework.
  • Optimized Performance: Despite its reduced size, MSH-Lite delivers competitive performance across a wide array of NLP tasks.
  • Bilingual Support: Designed to handle both Polish (pl) and English (en) with high fluency.

Technical Details:

  • Base Model: MSH-v1-Bielik-v2.3-Instruct-MedIT-merge
  • Parameter Count: 7 billion
  • Architecture: Derived from Bielik's core architecture, with parameter optimization.
  • Model Efficiency: Ideal for deployments where computational efficiency is paramount.

Performance Highlights:

To be done.

Acknowledgments:

Gratitude to the SpeakLeash project and ACK Cyfronet AGH for their contributions and collaboration.