Edit model card

Introduction

A led-large-16384 model to summarize ArXiv papers. Inputs are the abstracts of papers and full documents, and outputs are the summaries of the papers.

Allenai's Longformer Encoder-Decoder (LED).

As described in Longformer: The Long-Document Transformer by Iz Beltagy, Matthew E. Peters, Arman Cohan, led-base-16384 was initialized from bart-base since both models share the exact same architecture. To be able to process 16K tokens, bart-base's position embedding matrix was simply copied 16 times.

Downloads last month
54
Safetensors
Model size
460M params
Tensor type
F32
Β·
Inference Examples
Inference API (serverless) is not available, repository is disabled.

Spaces using AlgorithmicResearchGroup/led_large_16384_arxiv_summarization 4

Evaluation results