Will it be possible to release huggingface checkpoints?
#1
by
xansar
- opened
Thank you for your excellent work. I noticed that other repositories of MAP-Neo have released checkpoints in Hugging Face format, but this hasn't been done for neo_2b_general. Transforming checkpoints between Megatron and Hugging Face Transformers can be challenging due to differences in CUDA versions and other environmental settings. Could it be possible to release checkpoints in Hugging Face format? This would greatly facilitate the community in following and building upon your impressive work.
Thanks for your efforts on Neo. In addition to the HuggingFace format model mentioned above, could you tell us which ckpt do you think performs best currently according to your evaluation?