MalayaLLM_7B_Base / README.md
VishnuPJ's picture
Update README.md
3de582c verified
metadata
license: apache-2.0
language:
  - en
  - ml
pipeline_tag: text-generation

MalayaLLM [മലയാളം/Malayalam]

Baby MalayaLLM

MalayaLLM_7B_Base

This is an attempt to construct a Language Model (LLM) focused on generative AI for Malayalam language. While several LLMs are proficient in supporting multiple languages, including Malayalam, enhancing their performance for specific tasks such as content generation and question answering specifically in Malayalam can be achieved through dedicated training on a Malayalam dataset. In pursuit of this, I've undertaken the continuous pre-training of the LLAMA2 model using a comprehensive Malayalam dataset.

The model is currently in its early stages, and ongoing training and fine-tuning with a more comprehensive dataset are necessary to enhance its performance. I will consistently provide updated revisions to the model.

Github Repo:

For comprehensive insights into model training, fine-tuning, and other advanced techniques, refer to the MalayaLLM GitHub repository at the following link: https://github.com/VishnuPJ/MalayaLLM

Introducing the Developer:

Discover the mind behind this model and stay updated on their contributions to the field https://www.linkedin.com/in/vishnu-prasad-j/

Model description

The MalayaLLM models have been improved and customized to incorporate a comprehensive Malayalam vocabulary comprising approximately 18,000 tokens, expanding upon the groundwork laid by the original LLaMA-2.

Available Models

Model Type Data Base Model # Params Download Links
MalayaLLM 7B Base #v0.1 Base model 12GB LLaMA 7B 7B HF Hub
MalayaLLM 7B Instruct #v0.1 Instruction following model 52k instructions MalayaLLM 7B Base 7B HF Hub
MalayaLLM 7B Instruct #v0.2 Instruction following model 52k instructions MalayaLLM 7B Base 7B HF Hub
** Note : MalayaLLM 7B Instruct v0.2 is the latest model.

Quantized Version of Available Models

Model Format Bits Download Links
MalayaLLM 7B Instruct #v0.1 GGUF Q8_0 HF Hub
MalayaLLM 7B Instruct #v0.2 GGUF Q8_0 HF Hub

🌟Happy coding💻🌟