Create README.md

README.md

@@ -0,0 +1,16 @@
+---
+license: mit
+language:
+- en
+- zh
+base_model:
+- deepseek-ai/DeepSeek-V3
+pipeline_tag: text-generation
+library_name: transformers
+---
+# DeepSeek V3 AWQ
+AWQ of the DeepSeek V3 chat model.
+
+This quant modified some of the model code to fix the overflow issue when using float16.
+
+Tested on vLLM with 8x H100, inference speed 5 tokens/s with batch size 1 and short proompts.