Attempts to fill out the 1B3 model details that diverge from the main one.

by meg HF staff - opened May 27, 2022

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+18

-23

meg

BigScience Workshop org May 27, 2022

Here I am using: https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/smaller_models/tr11b-1B3-ml.slurm to help flesh it out.

Attempts to fill out the 1B3 model details that diverge from the main one.2e65771c

christopher

BigScience Workshop org May 27, 2022

@meg It all looks great but I think we will have to update the hardware information. If I'm not mistaken, V100s will either have 16GB or 32GB of memory; and I think it should be 64 of them (16 nodes, each with 4 GPUs).

@teven How much memory did the GPUs have?

Update hardware info006a5e8a

Update hardware info3be67070

meg changed pull request status to merged May 27, 2022

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment