Habana
jwieczorekhabana commited on
Commit
3efb9a3
1 Parent(s): 625264e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -7
README.md CHANGED
@@ -13,18 +13,16 @@ This model only contains the `GaudiConfig` file for running the [Swin Transforme
13
  **This model contains no model weights, only a GaudiConfig.**
14
 
15
  This enables to specify:
16
- - `use_habana_mixed_precision`: whether to use Habana Mixed Precision (HMP)
17
- - `hmp_opt_level`: optimization level for HMP, see [here](https://docs.habana.ai/en/latest/PyTorch/PyTorch_Mixed_Precision/PT_Mixed_Precision.html#configuration-options) for a detailed explanation
18
- - `hmp_bf16_ops`: list of operators that should run in bf16
19
- - `hmp_fp32_ops`: list of operators that should run in fp32
20
- - `hmp_is_verbose`: verbosity
21
  - `use_fused_adam`: whether to use Habana's custom AdamW implementation
22
  - `use_fused_clip_norm`: whether to use Habana's fused gradient norm clipping operator
23
-
 
 
24
  ## Usage
25
 
26
  The model is instantiated the same way as in the Transformers library.
27
  The only difference is that there are a few new training arguments specific to HPUs.
 
28
 
29
  [Here](https://github.com/huggingface/optimum-habana/blob/main/examples/image-classification/run_image_classification.py) is an image classification example script to fine-tune a model. You can run it with Swin with the following command:
30
  ```bash
@@ -48,7 +46,8 @@ python run_image_classification.py \
48
  --use_lazy_mode \
49
  --gaudi_config_name Habana/swin \
50
  --throughput_warmup_steps 2 \
51
- --ignore_mismatched_sizes
 
52
  ```
53
 
54
  Check the [documentation](https://huggingface.co/docs/optimum/habana/index) out for more advanced usage and examples.
 
13
  **This model contains no model weights, only a GaudiConfig.**
14
 
15
  This enables to specify:
 
 
 
 
 
16
  - `use_fused_adam`: whether to use Habana's custom AdamW implementation
17
  - `use_fused_clip_norm`: whether to use Habana's fused gradient norm clipping operator
18
+ - `disable_autocast`: whether to disable autocast; this parameter takes precedence over --bf16 flag and is temporary as some scripts produce nan values.
19
+ In those cases this parameter is already present in huggingface topology Habana gaudi_config.json.
20
+
21
  ## Usage
22
 
23
  The model is instantiated the same way as in the Transformers library.
24
  The only difference is that there are a few new training arguments specific to HPUs.
25
+ This model is supported only in mixed precision training with bf16 type.
26
 
27
  [Here](https://github.com/huggingface/optimum-habana/blob/main/examples/image-classification/run_image_classification.py) is an image classification example script to fine-tune a model. You can run it with Swin with the following command:
28
  ```bash
 
46
  --use_lazy_mode \
47
  --gaudi_config_name Habana/swin \
48
  --throughput_warmup_steps 2 \
49
+ --ignore_mismatched_sizes \
50
+ --bf16
51
  ```
52
 
53
  Check the [documentation](https://huggingface.co/docs/optimum/habana/index) out for more advanced usage and examples.