Create README.md

README.md

@@ -0,0 +1,48 @@
+## About 
+
+This is the 8-bit quantized version of Facebook's mbart model.
+
+According to the abstract, MBART is a sequence-to-sequence denoising auto-encoder pretrained on large-scale monolingual corpora in many languages using the BART objective. mBART is one of the first methods for pretraining a complete sequence-to-sequence model by denoising full texts in multiple languages, while previous approaches have focused only on the encoder, decoder, or reconstructing parts of the text.
+
+This model was contributed by [valhalla](https://huggingface.co/valhalla). The Authors’ code can be found [here](https://github.com/facebookresearch/fairseq/tree/main/examples/mbart)
+
+## Usage info 
+
+Install requred packages
+
+```!pip install -U bitsandbytes sentencepiece```
+
+then import model from 🤗 transformers library
+
+```python
+from transformers import MBartTokenizer, AutoModelForSeq2SeqLM, pipeline
+
+tokenizer = AutoTokenizer.from_pretrained("Ransaka/mbart-large-cc25-8bit")
+model = AutoModelForSeq2SeqLM.from_pretrained("Ransaka/mbart-large-cc25-8bit", device_map='auto')
+
+# you'll get an output like this if import succeed
+# ===================================BUG REPORT===================================
+# Welcome to bitsandbytes. For bug reports, please run
+
+# python -m bitsandbytes
+
+#  and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
+# ================================================================================
+# bin /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cuda113_nocublaslt.so
+# CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
+# CUDA SETUP: Highest compute capability among GPUs detected: 6.0
+# CUDA SETUP: Detected CUDA version 113
+# CUDA SETUP: Loading binary /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cuda113_nocublaslt.so...
+
+#create summarization pipeline
+text = """Right now, major tech firms are clamouring to replicate the runaway success of ChatGPT,
+          the generative AI chatbot developed by OpenAI using its GPT-3 large language model.
+          Much like potential game-changers of the past, such as cloud-based Software as a Service
+          (SaaS) platforms or blockchain technology (emphasis on potential), established companies
+          and start-ups alike are going public with LLMs and ChatGPT alternatives in fear of being left behind.
+      """
+pipe = pipeline('text2text-generation', model=model, tokenizer=tokenizer)
+pipe(text)
+#[{'generated_text': 'theore, major tech are clamouring to replicate the generative AI chatbot developed by OpenAI using its AI'}]
+
+```