loubnabnl HF staff commited on
Commit
650da67
1 Parent(s): 264a5f8
Files changed (1) hide show
  1. architectures/intro.txt +2 -0
architectures/intro.txt ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ Various architectures are used in code generation models, but most of them use the auto-regressive left-to-right setting, such as GPT. However [InCoder](https://huggingface.co/facebook/incoder-6B) used a decoder-only Transformer with Causal Masking objective,
2
+ that combines both next token prediction and bidirectional context through masking. AlphaCode used an encoder-decoder architecture. For model-specific information about the architecture, please select a model below: