Is this Llama-2 or CodeLlama-2?
#1
by
Haoxiang-Wang
- opened
Is this Llama-2 or CodeLlama-2? I don't see any public release of Llama-2 34B.
This is CodeLlama-2 based. It was an experiment to see if the capabilities of Llama-2 34B could be recovered from CodeLlama by fine tuning on plain text data. It sort of worked, I guess, given that the benchmarks all did improve by a couple of points. My real takeaway from it was that it would need way, way more compute than I have access to to meaningfully pull it off.