loubnabnl HF staff commited on
Commit
cf212fc
1 Parent(s): c0a7d69

update model

Browse files
Files changed (1) hide show
  1. architectures/incoder.txt +5 -2
architectures/incoder.txt CHANGED
@@ -2,10 +2,13 @@
2
 
3
  |Model | # parameters |
4
  | - | - |
 
5
  | Decoder |6.7B |
6
 
7
 
8
  [Causal Masking objective](https://arxiv.org/abs/2201.07520) is a hybrid approach of Causal and Masked language models, "it combines the benefit of per-token generation with optional bi-directionality specifically tailored to prompting".
9
- During the training of InCoder, spans of code were randomly masked and moved to the end of each file, which allows for bidirectional context. Figure 1 below illustrates the training process.
10
 
11
- So in addition to program synthesis (via left-to-right generation), InCoder can also perform editing (via infilling). The model gives promising results in some zero-shot code infilling tasks such as type prediction, variable re-naming and comment generation.
 
 
 
2
 
3
  |Model | # parameters |
4
  | - | - |
5
+ | Decoder |1.3B |
6
  | Decoder |6.7B |
7
 
8
 
9
  [Causal Masking objective](https://arxiv.org/abs/2201.07520) is a hybrid approach of Causal and Masked language models, "it combines the benefit of per-token generation with optional bi-directionality specifically tailored to prompting".
10
+ During the training of InCoder, spans of code were randomly masked and moved to the end of each file, which allows for bidirectional context. Figure 1 illustrates the training process.
11
 
12
+ So in addition to program synthesis (via left-to-right generation), InCoder can also perform editing (via infilling). The model gives promising results in some zero-shot code infilling tasks such as type prediction, variable re-naming and comment generation.
13
+
14
+ In the code generation demo we use InCoder 1.3B.