Triangle104/2B_or_not_2B-Q5_K_M-GGUF

This model was converted to GGUF format from SicariusSicariiStuff/2B_or_not_2B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

The model's name is fully credited to invisietch and Shakespeare; without them, this model would not have existed.

Regarding the question, I am happy to announce that it is, in fact, 2B, as it is so stated on the original Google model card, which this model was finetuned on.

If there's one thing we can count on, it is Google to tell us what is true, and what is misinformation. You should always trust and listen to your elders, and especially to your big brother.

This model was finetuned on a whimsical whim, on my poor laptop. It's not really poor, the GPU is 4090 16GB, but... it is driver-locked to 80watts because nVidia probably does not have the resources to make better drivers for Linux. I hope nVidia will manage to recover, as I have seen poor Jensen with the same old black leather jacket for years upon years. The stock is down like 22% already in this month (August 11th, 2024).

Finetuning took about 4 hours, while the laptop was on my lap, and while I was talking about books and stuff on Discord. Luckily, the laptop wasn't too hot, as 80 watts is not the 175w I was promised, which would have surely been hot enough to make an Omelette. Always remain an optimist fellas!

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/2B_or_not_2B-Q5_K_M-GGUF --hf-file 2b_or_not_2b-q5_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/2B_or_not_2B-Q5_K_M-GGUF --hf-file 2b_or_not_2b-q5_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/2B_or_not_2B-Q5_K_M-GGUF --hf-file 2b_or_not_2b-q5_k_m.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/2B_or_not_2B-Q5_K_M-GGUF --hf-file 2b_or_not_2b-q5_k_m.gguf -c 2048