Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,16 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q8_0/Llama-3_1-Nemotron-51B-Instruct-Q8_0-00001-of-00002.gguf filter=lfs diff=lfs merge=lfs -text
+Llama-3_1-Nemotron-51B-Instruct-Q8_0/Llama-3_1-Nemotron-51B-Instruct-Q8_0-00002-of-00002.gguf filter=lfs diff=lfs merge=lfs -text

Llama-3_1-Nemotron-51B-Instruct-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a345f2964d6f8d4e5df8491f42309d8d57d4bb28ed82b4bf055be00825fdcef
+size 19418642464

Llama-3_1-Nemotron-51B-Instruct-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1070e828494fc4526c9bdefb00e9886b19ce44be8decd25439bcb4d73bf2119
+size 27349751840

Llama-3_1-Nemotron-51B-Instruct-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:594fa2f9919f63dae49bd2df20f160f58c83da1196d3b6150e64be99cfc9c0e2
+size 25182345248

Llama-3_1-Nemotron-51B-Instruct-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee4a354133aa0363fc0ca0c55897dc4f966e6af722c0c1420e23f10a71bfac22
+size 22652393504

Llama-3_1-Nemotron-51B-Instruct-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1f598784cd2101e969803d8d3afcc077b82ce7bfb7a641adb3fcc0505caf2a2
+size 29252368416

Llama-3_1-Nemotron-51B-Instruct-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:529b290c8c5cb880d90e3924a8daebf8b88c40f9867b7a6cbc45aa90704ded62
+size 31037306912

Llama-3_1-Nemotron-51B-Instruct-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f6dbbaadf103013884ae2fcaab900c9631ed4411faaa5fa3668dc772d094ca5
+size 29484496928

Llama-3_1-Nemotron-51B-Instruct-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08f8a77e4d4e5fca16f4f1748b19073d9ee514ef0f4a1dede808c45a83c16169
+size 35558504480

Llama-3_1-Nemotron-51B-Instruct-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a9cba9918ff0f84425fdf15684753eee5877cad1ffc4c45968871f25acd69ed
+size 36465391648

Llama-3_1-Nemotron-51B-Instruct-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e72b7126ca10b0fb7deebab1e6f166ca3e050f3d284764f38900f276c37c8fd
+size 35558504480

Llama-3_1-Nemotron-51B-Instruct-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58c1b39f3c59fe3796c2c75206a31e013f2f5caf01dd3bd87ec69eca8ae63437
+size 42258774048

Llama-3_1-Nemotron-51B-Instruct-Q8_0/Llama-3_1-Nemotron-51B-Instruct-Q8_0-00001-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f5589c9e06f57089c32648c0231672a848e17bc75e3232a1c6bcd1665a8cfcc
+size 34942533024

Llama-3_1-Nemotron-51B-Instruct-Q8_0/Llama-3_1-Nemotron-51B-Instruct-Q8_0-00002-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:861aab327aa8e47fc1ca3783d2d705a472f49a0cc84f8114625eab4c94a1d1fd
+size 19788839744

README.md

@@ -0,0 +1,92 @@
+---
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+tags:
+- nvidia
+- llama-3
+- pytorch
+- TensorBlock
+- GGUF
+license: other
+license_name: nvidia-open-model-license
+license_link: https://developer.download.nvidia.com/licenses/nvidia-open-model-license-agreement-june-2024.pdf
+base_model: nvidia/Llama-3_1-Nemotron-51B-Instruct
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<div style="display: flex; justify-content: space-between; width: 100%;">
+    <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;">
+            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
+        </p>
+    </div>
+</div>
+
+## nvidia/Llama-3_1-Nemotron-51B-Instruct - GGUF
+
+This repo contains GGUF format model files for [nvidia/Llama-3_1-Nemotron-51B-Instruct](https://huggingface.co/nvidia/Llama-3_1-Nemotron-51B-Instruct).
+
+The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4391](https://github.com/ggerganov/llama.cpp/commit/9ba399dfa7f115effc63d48e6860a94c9faa31b2).
+
+<div style="text-align: left; margin: 20px 0;">
+    <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
+        Run them on the TensorBlock client using your local machine ↗
+    </a>
+</div>
+
+## Prompt template
+
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+
+Cutting Knowledge Date: December 2023
+Today Date: 26 Jul 2024
+
+{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
+
+{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+```
+
+## Model file specification
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [Llama-3_1-Nemotron-51B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q2_K.gguf) | Q2_K | 19.419 GB | smallest, significant quality loss - not recommended for most purposes |
+| [Llama-3_1-Nemotron-51B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q3_K_S.gguf) | Q3_K_S | 22.652 GB | very small, high quality loss |
+| [Llama-3_1-Nemotron-51B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q3_K_M.gguf) | Q3_K_M | 25.182 GB | very small, high quality loss |
+| [Llama-3_1-Nemotron-51B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q3_K_L.gguf) | Q3_K_L | 27.350 GB | small, substantial quality loss |
+| [Llama-3_1-Nemotron-51B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q4_0.gguf) | Q4_0 | 29.252 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
+| [Llama-3_1-Nemotron-51B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q4_K_S.gguf) | Q4_K_S | 29.484 GB | small, greater quality loss |
+| [Llama-3_1-Nemotron-51B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q4_K_M.gguf) | Q4_K_M | 31.037 GB | medium, balanced quality - recommended |
+| [Llama-3_1-Nemotron-51B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q5_0.gguf) | Q5_0 | 35.559 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
+| [Llama-3_1-Nemotron-51B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q5_K_S.gguf) | Q5_K_S | 35.559 GB | large, low quality loss - recommended |
+| [Llama-3_1-Nemotron-51B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q5_K_M.gguf) | Q5_K_M | 36.465 GB | large, very low quality loss - recommended |
+| [Llama-3_1-Nemotron-51B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q6_K.gguf) | Q6_K | 42.259 GB | very large, extremely low quality loss |
+| [Llama-3_1-Nemotron-51B-Instruct-Q8_0](https://huggingface.co/tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF/blob/main/Llama-3_1-Nemotron-51B-Instruct-Q8_0) | Q8_0 | 54.731 GB | very large, extremely low quality loss - not recommended |
+
+
+## Downloading instruction
+
+### Command line
+
+Firstly, install Huggingface Client
+
+```shell
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, downoad the individual model file the a local directory
+
+```shell
+huggingface-cli download tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF --include "Llama-3_1-Nemotron-51B-Instruct-Q2_K.gguf" --local-dir MY_LOCAL_DIR
+```
+
+If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
+
+```shell
+huggingface-cli download tensorblock/Llama-3_1-Nemotron-51B-Instruct-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```