a collection of pre-training corpora refined by ProX
GAIR-ProX
community
AI & ML interests
NLP Research
Organization Card
GAIR-ProX, a subsidiary of GAIR, spearheads the ProX Project. This initiative aims to enhance pre-training efficiency by refining corpus documents using language models at scale. Through meticulous operations (e.g., document-level filtering and chunk-level cleaning), implemented as scalable, executable programs, ProX seeks to improve pre-training data quality at scale, ultimately developing more robust and efficient language models.
Collections
1
models
None public yet