Spaces:
Runtime error
Runtime error
fix typo
Browse files
architectures/polycoder.txt
CHANGED
@@ -1,4 +1,4 @@
|
|
1 |
-
[PolyCoder](https://github.com/VHellendoorn/Code-LMs) uses GPT2 architecture, with BPE tokenizer trained on a random 5% subset of the data (all languages), and a context
|
2 |
|
3 |
<div align="center">
|
4 |
|
|
|
1 |
+
[PolyCoder](https://github.com/VHellendoorn/Code-LMs) uses GPT2 architecture, with BPE tokenizer trained on a random 5% subset of the data (all languages), and a context length of 2048. To study the effect of scaling of model size, the odel was trained in 3 different sizes.
|
2 |
|
3 |
<div align="center">
|
4 |
|