dudleymax/ludwig-llama2python

Citation

@misc{Molino2019,
  author = {Piero Molino and Yaroslav Dudin and Sai Sumanth Miryala},
  title = {Ludwig: a type-based declarative deep learning toolbox},
  year = {2019},
  eprint = {arXiv:1909.07930},
}

Model Card for Model ID

Model Details

Model Description

This model focuses on fine-tuning the Llama-2 7B large language model for Python code generation. The project leverages Ludwig, an open-source toolkit, and a dataset of 500k Python code samples from Hugging Face. The model applies techniques such as prompt templating, zero-shot inference, and few-shot learning, enhancing the model's performance in generating Python code snippets efficiently.

• Developed by: Kevin Geejo, Aniket Yadav, Rishab Pandey

• Model type: Fine-tuned Llama-2 7B for Python code generation

• Language(s) (NLP): Python (for code generation tasks)

• License: Not explicitly mentioned, but Llama-2 models are typically governed by Meta AI’s open-source licensing

• Finetuned from model [optional]: Llama-2 7B (Meta AI, 2023)

Model Sources [optional]

• Repository: Hugging Face

Uses

Direct Use

• Python code generation for software development
• Automation of coding tasks
• Developer productivity enhancement

Downstream Use [optional]

• Code completion, bug fixing, and Python code translation

Out-of-Scope Use

• Non-Python programming tasks
• Generation of sensitive, legal, or medical content

Bias, Risks, and Limitations

• Limited to Python programming tasks
• Dataset biases from Hugging Face's Python Code Dataset
• Environmental impact from computational costs during fine-tuning

Recommendations

Users should be aware of computational efficiency trade-offs and potential limitations in generalizing to new Python tasks.

How to Get Started with the Model

Use the code below to get started with the model:

# Example setup (simplified)
import ludwig
from transformers import AutoModel

model = AutoModel.from_pretrained("llama-2-7b-python")

Training Details

Training Data

• 500k Python code samples sourced from Hugging Face

Training Procedure

• Preprocessing [optional]: Hugging Face Python Code Dataset
• Training regime: Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA)

Speeds, Sizes, Times [optional]

• Not explicitly mentioned in the document

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Derived from Python code datasets on Hugging Face

Factors

• Python code generation tasks

Metrics

• Code correctness and efficiency

Results

• Fine-tuning improved Python code generation performance

Summary

The fine-tuned model showed enhanced proficiency in generating Python code snippets, reflecting its adaptability to specific coding tasks.

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator.

Model Architecture and Objective

• Llama-2 7B model architecture fine-tuned for Python code generation

Compute Infrastructure

• Not explicitly mentioned

Hardware

• Not specified

Software

• Ludwig toolkit and Hugging Face integration

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

• Llama-2: Open-source large language model by Meta AI
• LoRA (Low-Rank Adaptation): Efficient fine-tuning method modifying fewer model parameters
• PEFT: Parameter-efficient fine-tuning technique

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

Kevin Geejo, Aniket Yadav, Rishab Pandey

Model Card Contact

[email protected], [email protected], [email protected]

Framework versions

• Llama-2 version: 7B
• Ludwig version: 0.8
• Hugging Face integration: Latest