malteos commited on
Commit
750817b
1 Parent(s): 32bb80e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -1,3 +1,30 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - oscar
5
+ language:
6
+ - uk
7
+ library_name: transformers
8
+ pipeline_tag: text-generation
9
  ---
10
+
11
+ # GPT2 Ukrainian
12
+
13
+ A generative language model for the Ukrainian language follows the [GPT-2 architecture](https://huggingface.co/gpt2) (124M parameters).
14
+
15
+ - hidden size: 768
16
+ - number of heads: 12
17
+ - number of layers: 12
18
+ - seq length: 1024
19
+ - tokens: 11238113280 (3 epochs)
20
+ - steps: 57167
21
+
22
+ ## Training data
23
+
24
+ - OSCAR
25
+ - Wikimedia dumps
26
+
27
+ ## License
28
+
29
+ MIT
30
+