nielsr HF staff commited on
Commit
d2c0788
1 Parent(s): f367350

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -23
README.md CHANGED
@@ -30,29 +30,6 @@ The abstract of the paper states that:
30
 
31
  # Using the model
32
 
33
- ## Converting from T5x to huggingface
34
-
35
- You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
36
- ```bash
37
- python convert_pix2struct_checkpoint_to_pytorch.py --t5x_checkpoint_path PATH_TO_T5X_CHECKPOINTS --pytorch_dump_path PATH_TO_SAVE --is_vqa
38
- ```
39
- if you are converting a large model, run:
40
- ```bash
41
- python convert_pix2struct_checkpoint_to_pytorch.py --t5x_checkpoint_path PATH_TO_T5X_CHECKPOINTS --pytorch_dump_path PATH_TO_SAVE --use-large --is_vqa
42
- ```
43
- Once saved, you can push your converted model with the following snippet:
44
- ```python
45
- from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
46
-
47
- model = Pix2StructForConditionalGeneration.from_pretrained(PATH_TO_SAVE)
48
- processor = Pix2StructProcessor.from_pretrained(PATH_TO_SAVE)
49
-
50
- model.push_to_hub("USERNAME/MODEL_NAME")
51
- processor.push_to_hub("USERNAME/MODEL_NAME")
52
- ```
53
-
54
- ## Get predictions from the model
55
-
56
  You should ask specific questions to the model in order to get consistent generations. Here we are asking the model whether the sum of values that are in a chart are greater than the largest value.
57
 
58
  ```python
@@ -73,6 +50,26 @@ print(processor.decode(predictions[0], skip_special_tokens=True))
73
 
74
  To run the predictions on GPU, simply add `.to(0)` when creating the model and when getting the inputs (`inputs = inputs.to(0)`)
75
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  # Contribution
77
 
78
  This model was originally contributed by Fangyu Liu, Francesco Piccinno et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
 
30
 
31
  # Using the model
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  You should ask specific questions to the model in order to get consistent generations. Here we are asking the model whether the sum of values that are in a chart are greater than the largest value.
34
 
35
  ```python
 
50
 
51
  To run the predictions on GPU, simply add `.to(0)` when creating the model and when getting the inputs (`inputs = inputs.to(0)`)
52
 
53
+ ## Converting from T5x to huggingface
54
+
55
+ You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
56
+ ```bash
57
+ python convert_pix2struct_checkpoint_to_pytorch.py --t5x_checkpoint_path PATH_TO_T5X_CHECKPOINTS --pytorch_dump_path PATH_TO_SAVE --is_vqa
58
+ ```
59
+ if you are converting a large model, run:
60
+ ```bash
61
+ python convert_pix2struct_checkpoint_to_pytorch.py --t5x_checkpoint_path PATH_TO_T5X_CHECKPOINTS --pytorch_dump_path PATH_TO_SAVE --use-large --is_vqa
62
+ ```
63
+ Once saved, you can push your converted model with the following snippet:
64
+ ```python
65
+ from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
66
+
67
+ model = Pix2StructForConditionalGeneration.from_pretrained(PATH_TO_SAVE)
68
+ processor = Pix2StructProcessor.from_pretrained(PATH_TO_SAVE)
69
+
70
+ model.push_to_hub("USERNAME/MODEL_NAME")
71
+ processor.push_to_hub("USERNAME/MODEL_NAME")
72
+
73
  # Contribution
74
 
75
  This model was originally contributed by Fangyu Liu, Francesco Piccinno et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).