Note Zephyr is by far the best aligned open-sourced LLM I've used. They recently have a -beta and a -gamma (fine-tuned out of Gemma) version too.

Note "We clean data"

Note Trade more computation with less memory. It's much like if you do not want to remember all the corollary from a math class, you'd then have to derive everything from the 3 axioms.

Note Perplexity score used to decide which abstract makes more sense, given all the previous works on the field of neuroscience, beating expert's annotation. Advice to expert: pay attention when you annotate, otherwise you might lose your job (!)

Note Fixed control vector gets added into each layer's output, convenient package here: https://github.com/vgel/repeng quite easy to use, and allow linear control along user-defined semantic dimension

Note Revolutionise next-token prediction based pre-training to enhance reasoning. Routing through multiple rationales for next-k-token prediction, combined with RL-based survival of the fittest rationales, achieve significant improvement in reasoning at the cost of huge increase in training cost. Likely lags behind Q* due to the missing of adaptive control of thought process.

Note Arsenal for training VLM-based Front-end designer.

Note Stanford's drop-in replacement model for automating front-end design. Image (or a sketch) of the target website in, front-end code out.

SALT-NLP/Design2Code-18B-v0

Updated Mar 16 • 34

Note This LLM does Front-End engineering for you at the cost of your electricity.

Note Apple's own VLM.

vikhyatk/moondream2

Image-Text-to-Text • Updated Aug 26 • 196k • 690

Note Very performant small VLM. It appears an extra vision encoder might just do the trick?

prometheus-eval/prometheus-13b-v1.0

Text2Text Generation • Updated Oct 14, 2023 • 3.91k • 126

Note Fine-tuned LLM for acting as LLM-as-a-Judge

Note How to train a GPT-4 level function calling LLM from UC Berkeley

gorilla-llm/gorilla-openfunctions-v2

Text Generation • Updated Apr 18 • 11.7k • 209

Note GPT-4 level FunctionCalling LLM from UC Berkeley

Note Soft Prompt Compression from Samsung

Note LLM OS with MemGPT from UC Berkeley