mosaicml
/

mosaic-bert-base-seqlen-1024

Model card Files Files and versions Community

mosaic-bert-base-seqlen-1024 / config.json

Commit History

Change attention_probs_dropout_prob to 0.1 so that triton FlashAttention dependencies are avoided

3bb233a

jacobfulano commited on Jan 3

Update config so that alibi_starting_size is explicitly set to 1024

e4c6e65

jacobfulano commited on May 31, 2023

Upload BertForMaskedLM

a76db42

jacobfulano commited on Apr 28, 2023