Export small model at 850 epochs

Update model card

Use SmolTalk dataset for instruction-tuning

Add library metadata

Export small at 400 epochs

Fix training forward pass

Fix unsqueeze missing dimension argument

Optimize next-token prediction

Export LightGPT small at 150 epochs

Improve the README

Fix model card

Blanket optimizations

A little nicer

Configurable feed-forward ratio

Fix README

More scaling law stuff

Merge branch 'master'

Compensate for git issues

Initial commit

Broad improvements

Use Fineweb instead of Openwebtext

Add MFU estimation for Ampere GPUs

Compile before wrapping with FSDP

Add FSDP

Cleanup checkpointing history

A little nicer

Merge branch 'main' of https://huggingface.co./andrewdalpino/LightGPT

Remove old license info

Add metadata to README

Merge branch 'main' of https://huggingface.co./andrewdalpino/LightGPT

initial commit

Initial commit