README.md

---
library_name: transformers.js
base_model: google/paligemma2-3b-ft-docci-448
---

https://huggingface.co./google/paligemma2-3b-ft-docci-448 with ONNX weights to be compatible with Transformers.js.

## Usage (Transformers.js)

If you haven't already, you can install the [Transformers.js](https://huggingface.co./docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@huggingface/transformers) using:
```bash
npm i @huggingface/transformers
```

**Example:** Image captioning with `onnx-community/paligemma2-3b-ft-docci-448`.
```js
import { AutoProcessor, PaliGemmaForConditionalGeneration, load_image } from '@huggingface/transformers';

// Load processor and model
const model_id = 'onnx-community/paligemma2-3b-ft-docci-448';
const processor = await AutoProcessor.from_pretrained(model_id);
const model = await PaliGemmaForConditionalGeneration.from_pretrained(model_id, {
    dtype: {
        embed_tokens: 'fp16', // or 'q8'
        vision_encoder: 'fp16', // or 'q4', 'q8'
        decoder_model_merged: 'q4', // or 'q4f16'
    },
});

// Prepare inputs
const url = 'https://huggingface.co./datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg'
const raw_image = await load_image(url);
const prompt = '<image>caption en'; // Caption the image in English
const inputs = await processor(raw_image, prompt);

// Generate a response
const output = await model.generate({
    ...inputs,
    max_new_tokens: 100,
})

const generated_ids = output.slice(null, [inputs.input_ids.dims[1], null]);
const answer = processor.batch_decode(
    generated_ids,
    { skip_special_tokens: true },
);
console.log(answer[0]);
// A side view of a light blue 1970s Volkswagen Beetle parked on a gray cement road. It is facing to the right. It has a reflection on the side of it. Behind it is a yellow building with a brown double door on the right. It has a white frame around it. Part of a gray cement wall is visible on the far left.
```


---

Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co./docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).