metadata
license: llama3
tags:
- gguf
- llama3
- ollama
- dolphin
base_model: cognitivecomputations/dolphin-2.9-llama3-70b
Dolphin 2.9 Llama 3 70b 🐬 (GGUF)
Quantized and converted to GGUF for Ollama.
Example Modelfile
FROM ./dolphin-2.9-llama3-70b-Q6_K.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""
SYSTEM """You are Dolphin, a helpful AI assistant.
"""
PARAMETER num_ctx 8192
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
Provided files
| Name | Quant method | Size |
|---|---|---|
| dolphin-2.9-llama3-70b-Q6_K.gguf-part00000 dolphin-2.9-llama3-70b-Q6_K.gguf-part00001 | Q6_K | 54G |
| dolphin-2.9-llama3-70b-Q8_0.gguf-part00000 dolphin-2.9-llama3-70b-Q8_0.gguf-part00001 | Q8_0 | 70G |
Combine multi part files into one GGUF
cat dolphin-2.9-llama3-70b-Q6_K.gguf-part00000 dolphin-2.9-llama3-70b-Q6_K.gguf-part00001 > dolphin-2.9-llama3-70b-Q6_K.gguf