Philip May
commited on
Commit
·
823f41f
1
Parent(s):
03e7c09
Update README.md
Browse files
README.md
CHANGED
@@ -22,4 +22,5 @@ For example, it does not work on V100 GPUs. On A100, however, it does.
|
|
22 |
That is why we suggest to use [DeepSpeed](https://github.com/microsoft/DeepSpeed) for training.
|
23 |
In particular, we recommend the [ZeRO-3 Example](https://huggingface.co/docs/transformers/main_classes/deepspeed#zero3-example) `auto` configuration.
|
24 |
|
25 |
-
> ZeRO-Offload pushes the boundary of the maximum model size that can be trained efficiently using minimal GPU resources, by exploiting computational and memory resources on both GPUs and their host CPUs.
|
|
|
|
22 |
That is why we suggest to use [DeepSpeed](https://github.com/microsoft/DeepSpeed) for training.
|
23 |
In particular, we recommend the [ZeRO-3 Example](https://huggingface.co/docs/transformers/main_classes/deepspeed#zero3-example) `auto` configuration.
|
24 |
|
25 |
+
> ZeRO-Offload pushes the boundary of the maximum model size that can be trained efficiently using minimal GPU resources, by exploiting computational and memory resources on both GPUs and their host CPUs.
|
26 |
+
see [ZeRO-Offload](https://www.deepspeed.ai/features/#zero-offload)
|