README.md

## About 

This is the 8-bit quantized version of Facebook's mbart model.

According to the abstract, MBART is a sequence-to-sequence denoising auto-encoder pretrained on large-scale monolingual corpora in many languages using the BART objective. mBART is one of the first methods for pretraining a complete sequence-to-sequence model by denoising full texts in multiple languages, while previous approaches have focused only on the encoder, decoder, or reconstructing parts of the text.

This model was contributed by [valhalla](https://huggingface.co./valhalla). The Authors’ code can be found [here](https://github.com/facebookresearch/fairseq/tree/main/examples/mbart)

## Usage info 

Install requred packages

```!pip install -U bitsandbytes sentencepiece```

then import model from 🤗 transformers library

```python
from transformers import MBartTokenizer, AutoModelForSeq2SeqLM, pipeline

tokenizer = AutoTokenizer.from_pretrained("Ransaka/mbart-large-cc25-8bit")
model = AutoModelForSeq2SeqLM.from_pretrained("Ransaka/mbart-large-cc25-8bit", device_map='auto')

# you'll get an output like this if import succeed
# ===================================BUG REPORT===================================
# Welcome to bitsandbytes. For bug reports, please run

# python -m bitsandbytes

#  and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
# ================================================================================
# bin /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cuda113_nocublaslt.so
# CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
# CUDA SETUP: Highest compute capability among GPUs detected: 6.0
# CUDA SETUP: Detected CUDA version 113
# CUDA SETUP: Loading binary /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cuda113_nocublaslt.so...

#create summarization pipeline
text = """Right now, major tech firms are clamouring to replicate the runaway success of ChatGPT,
          the generative AI chatbot developed by OpenAI using its GPT-3 large language model.
          Much like potential game-changers of the past, such as cloud-based Software as a Service
          (SaaS) platforms or blockchain technology (emphasis on potential), established companies
          and start-ups alike are going public with LLMs and ChatGPT alternatives in fear of being left behind.
      """
pipe = pipeline('text2text-generation', model=model, tokenizer=tokenizer)
pipe(text)
#[{'generated_text': 'theore, major tech are clamouring to replicate the generative AI chatbot developed by OpenAI using its AI'}]

```