|
--- |
|
base_model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit |
|
tags: |
|
- text-generation-inference |
|
- transformers |
|
- unsloth |
|
- llama |
|
- gguf |
|
- ollama |
|
license: apache-2.0 |
|
language: |
|
- en |
|
|
|
--- |
|
|
|
# kubectl Operator Model |
|
|
|
- **Developed by:** dereklck |
|
- **License:** Apache-2.0 |
|
- **Fine-tuned from model:** [unsloth/Llama-3.2-1B-Instruct-bnb-4bit](https://huggingface.co./unsloth/Llama-3.2-1B-Instruct-bnb-4bit) |
|
- **Model type:** GGUF (compatible with Ollama) |
|
- **Language:** English |
|
|
|
This Llama-based model was fine-tuned to assist users with Kubernetes commands and questions. It has three primary features: |
|
|
|
1. **Generating accurate `kubectl` commands** based on user descriptions. |
|
2. **Providing brief explanations about Kubernetes** for general queries. |
|
3. **Politely requesting additional information** if the instruction is incomplete or ambiguous. |
|
|
|
**Important Note:** The **1B model may sometimes generate non-executable CLI commands containing placeholders** like `<pod_name>` or `<resource_name>`. This can lead to hallucinations and commands that cannot be executed directly. Users are advised to replace these placeholders with actual values before execution. This issue has been significantly **improved in the 3B model**, which generates more accurate and executable commands without unnecessary placeholders. |
|
|
|
--- |
|
|
|
## Model Details |
|
|
|
### Purpose |
|
|
|
The model assists users by: |
|
|
|
- **Generating `kubectl` commands** based on natural language descriptions. |
|
- **Providing brief explanations about Kubernetes** for general queries. |
|
- **Politely requesting additional information** if the instruction is incomplete or ambiguous. |
|
|
|
### Intended Users |
|
|
|
- Kubernetes administrators |
|
- DevOps engineers |
|
- Developers working with Kubernetes clusters |
|
|
|
### Training Process |
|
|
|
- **Base Model:** Unsloth's Llama-3.2-1B-Instruct-bnb-4bit |
|
- **Fine-tuning:** Utilized the Unsloth framework and Hugging Face's TRL library for efficient training. |
|
- **Training Data:** Customized dataset focused on Kubernetes operations and `kubectl` command usage, containing approximately 200 entries. |
|
|
|
--- |
|
|
|
## Model Features |
|
|
|
### 1. Command Generation in Bash Format |
|
|
|
When the model generates CLI commands, it provides them in `bash` format, enclosed within code blocks for easy execution and clarity. This allows users to copy and paste the commands directly into their terminal. |
|
|
|
### 2. Handling Ambiguity with Polite Clarifications |
|
|
|
If the instruction is incomplete or ambiguous, the model politely asks for the specific missing information instead of making assumptions. This ensures accuracy and prevents the execution of incorrect commands. |
|
|
|
### 3. Providing Concise Explanations |
|
|
|
For general Kubernetes questions, the model offers concise and accurate explanations without unnecessary details, helping users understand concepts quickly. |
|
|
|
--- |
|
|
|
## Usage |
|
|
|
### Parameters |
|
|
|
- **Temperature:** `0.3` (controls randomness; lower values make outputs more deterministic) |
|
- **Stop Token:** `</s>` (indicates the end of the assistant's response) |
|
|
|
--- |
|
|
|
## Examples |
|
|
|
### Example 1: Generating a Command |
|
|
|
**Instruction:** |
|
|
|
``` |
|
Create a namespace called 'development'. |
|
``` |
|
|
|
**Response:** |
|
|
|
```bash |
|
kubectl create namespace development |
|
``` |
|
|
|
--- |
|
|
|
### Example 2: Handling Ambiguity |
|
|
|
**Instruction:** |
|
|
|
``` |
|
Show me logs. |
|
``` |
|
|
|
**Response:** |
|
|
|
``` |
|
Which pod's logs would you like to retrieve? |
|
``` |
|
|
|
--- |
|
|
|
### Example 3: Providing Explanations |
|
|
|
**Instruction:** |
|
|
|
``` |
|
What is a Deployment in Kubernetes? |
|
``` |
|
|
|
**Response:** |
|
|
|
``` |
|
A Deployment provides declarative updates for Pods and ReplicaSets, allowing you to manage your applications by defining the desired state and letting Kubernetes handle the rest. |
|
``` |
|
|
|
--- |
|
|
|
## Limitations and Considerations |
|
|
|
- **Accuracy:** The model may occasionally produce incorrect or suboptimal commands. Always review the output before execution. |
|
- **Hallucinations:** The **1B model may generate CLI commands with placeholders** (e.g., `<pod_name>`, `<resource_name>`), resulting in commands that are **not directly executable**. Users should carefully replace these placeholders with actual values. |
|
|
|
- **Improvement in 3B Model:** This issue has been addressed in the **3B model**, where the generation of non-executable commands with placeholders has been significantly reduced, providing more accurate and usable outputs. |
|
|
|
- **Security:** Be cautious when executing generated commands, especially in production environments. |
|
|
|
--- |
|
|
|
## Future Revisions |
|
|
|
- **Addressing Hallucinations:** Efforts are underway to further reduce hallucinations in future versions, building upon improvements made in the 3B model. |
|
- **Enhanced Dataset:** Expanding the training dataset to include a wider range of Kubernetes operations to improve the model's versatility. |
|
- **Fine-tuning Techniques:** Implementing advanced fine-tuning methods to enhance accuracy and reliability. |
|
|
|
--- |
|
|
|
## Deployment with Ollama |
|
|
|
### Prerequisites |
|
|
|
- Install [Ollama](https://github.com/jmorganca/ollama) on your system. |
|
- Ensure you have the GGUF model file (e.g., `kubectl_operator.Q8_0.gguf`). |
|
|
|
### Steps |
|
|
|
1. **Create the Modelfile** |
|
|
|
Save the following content as a file named `Modelfile`: |
|
|
|
```plaintext |
|
FROM kubectl_operator.Q8_0.gguf |
|
|
|
PARAMETER temperature 0.3 |
|
PARAMETER stop "</s>" |
|
|
|
TEMPLATE """ |
|
You are an AI assistant that helps users with Kubernetes commands and questions. |
|
|
|
**Your Behavior Guidelines:** |
|
|
|
1. **For clear and complete instructions:** |
|
- **Provide only** the exact `kubectl` command needed to fulfill the user's request. |
|
- Do not include extra explanations, placeholders, or context. |
|
- **Enclose the command within a code block** with `bash` syntax highlighting. |
|
|
|
2. **For incomplete or ambiguous instructions:** |
|
- **Politely ask** the user for the specific missing information. |
|
- Do **not** provide any commands or placeholders in your response. |
|
- Respond in plain text, clearly stating what information is needed. |
|
|
|
3. **For general Kubernetes questions:** |
|
- Provide a **concise and accurate explanation**. |
|
- Do **not** include any commands unless specifically requested. |
|
- Ensure that the explanation fully addresses the user's question. |
|
|
|
**Important Rules:** |
|
|
|
- **Do not generate CLI commands containing placeholders** (e.g., `<pod_name>`, `<resource_name>`). |
|
- Ensure all CLI commands are **complete, valid, and executable** as provided. |
|
- If user input is insufficient to form a complete command, **ask for clarification** instead of using placeholders. |
|
- Provide only the necessary CLI command output without any additional text. |
|
|
|
### Instruction: |
|
{{ .Prompt }} |
|
|
|
### Response: |
|
""" |
|
``` |
|
|
|
2. **Create the Model with Ollama** |
|
|
|
Open your terminal and run the following command to create the model: |
|
|
|
```bash |
|
ollama create kubectl_operator -f Modelfile |
|
``` |
|
|
|
This command tells Ollama to create a new model named `kubectl_operator` using the configuration specified in `Modelfile`. |
|
|
|
3. **Run the Model** |
|
|
|
Start interacting with your model: |
|
|
|
```bash |
|
ollama run kubectl_operator |
|
``` |
|
|
|
This will initiate the model and prompt you for input based on the template provided. |
|
|
|
--- |
|
|
|
## Feedback and Contributions |
|
|
|
We welcome any comments or participation to improve the model and dataset. If you encounter issues or have suggestions for improvement: |
|
|
|
- **GitHub:** [Unsloth Repository](https://github.com/unslothai/unsloth) |
|
- **Contact:** Reach out to the developer, **dereklck**, for further assistance. |
|
|
|
--- |
|
|
|
**Note:** This model assists in generating `kubectl` commands based on user input. Always verify the generated commands and replace any placeholders with actual values before executing them in a production cluster. |
|
|
|
--- |
|
|
