adding in disclaimer and cleaning up gpu start up

README.md

@@ -82,14 +82,9 @@ model = AutoModelForCausalLM.from_pretrained("sambanovasystems/BLOOMChat-176B-v1
 
 ### Tutorial on using the model for text generation
 
-[Transformers BLOOM Inference](https://github.com/huggingface/transformers-bloom-inference)
+[This tutorial](https://github.com/huggingface/transformers-bloom-inference) from Huggingface will be the base layer for running our model. The tutorial is intended for BLOOM; however, since our model is based off of BLOOM we can repurpose it.
 
-Specifically we tested BLOOM inference via command-line in this repository.
-
-Running command:
-```
-python -m inference_server.cli --model_name sambanovasystems/BLOOMChat-176B-v1 --model_class AutoModelForCausalLM --dtype int8 --deployment_framework hf_accelerate --generate_kwargs '{"do_sample": false, "temperature": 0.8, "repetition_penalty": 1.2, "top_p": 0.9, "max_new_tokens": 512}'
-```
+For setup instructions follow the Huggingface tutorial.
 
 NOTE: Things that we had to modify in order for BLOOMChat to work:
 - Install transformers version 4.27.0
@@ -113,7 +108,15 @@ class HFAccelerateModel(Model):
         kwargs["max_memory"] = reduce_max_memory_dict
 ```
 
-
+Running command for int8 (sub optimal performance, but fast inference time):
+```
+python -m inference_server.cli --model_name sambanovasystems/BLOOMChat-176B-v1 --model_class AutoModelForCausalLM --dtype int8 --deployment_framework hf_accelerate --generate_kwargs '{"do_sample": false, "temperature": 0.8, "repetition_penalty": 1.2, "top_p": 0.9, "max_new_tokens": 512}'
+```
+Running command for bf16
+```
+python -m inference_server.cli --model_name sambanovasystems/BLOOMChat-176B-v1 --model_class AutoModelForCausalLM --dtype bf16 --deployment_framework hf_accelerate --generate_kwargs '{"do_sample": false, "temperature": 0.8, "repetition_penalty": 1.2, "top_p": 0.9, "max_new_tokens": 512}'
+```
+**DISCLAIMER:** When using int8, the results will be subpar compared to bf16 as the model is being [quantized](https://huggingface.co/blog/hf-bitsandbytes-integration#introduction-to-model-quantization). 
 
 ### Suggested Inference Parameters
 - Temperature: 0.8