Update README.md
Browse files
README.md
CHANGED
@@ -35,12 +35,13 @@ The abstract of the paper states that:
|
|
35 |
You should ask specific questions to the model in order to get consistent generations. Here we are asking the model whether the sum of values that are in a chart are greater than the largest value.
|
36 |
|
37 |
```python
|
38 |
-
from transformers import
|
39 |
import requests
|
40 |
from PIL import Image
|
41 |
|
42 |
-
|
43 |
-
|
|
|
44 |
url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/20294671002019.png"
|
45 |
image = Image.open(requests.get(url, stream=True).raw)
|
46 |
|
@@ -52,7 +53,7 @@ print(processor.decode(predictions[0], skip_special_tokens=True))
|
|
52 |
|
53 |
To run the predictions on GPU, simply add `.to(0)` when creating the model and when getting the inputs (`inputs = inputs.to(0)`)
|
54 |
|
55 |
-
|
56 |
|
57 |
You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
|
58 |
```bash
|
|
|
35 |
You should ask specific questions to the model in order to get consistent generations. Here we are asking the model whether the sum of values that are in a chart are greater than the largest value.
|
36 |
|
37 |
```python
|
38 |
+
from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
|
39 |
import requests
|
40 |
from PIL import Image
|
41 |
|
42 |
+
processor = Pix2StructProcessor.from_pretrained('google/matcha-chartqa')
|
43 |
+
model = Pix2StructForConditionalGeneration.from_pretrained('google/matcha-chartqa')
|
44 |
+
|
45 |
url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/20294671002019.png"
|
46 |
image = Image.open(requests.get(url, stream=True).raw)
|
47 |
|
|
|
53 |
|
54 |
To run the predictions on GPU, simply add `.to(0)` when creating the model and when getting the inputs (`inputs = inputs.to(0)`)
|
55 |
|
56 |
+
# Converting from T5x to huggingface
|
57 |
|
58 |
You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
|
59 |
```bash
|