Po-Hao, Hsu's picture

8

Po-Hao, Hsu

Ball16

·

AI & ML interests

None yet

Recent Activity

liked a Space 11 days ago

mteb/leaderboard

liked a Space 5 months ago

lmarena-ai/chatbot-arena-leaderboard

reacted to m-ric's post with 🚀 6 months ago

𝗢𝗽𝗲𝗻 𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝗼𝗻 𝗳𝗶𝗿𝗲 𝗿𝗶𝗴𝗵𝘁 𝗻𝗼𝘄! 🔥 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸-𝗩𝟮.𝟱 𝗮𝗻𝗱 𝗼𝘁𝗵𝗲𝗿 𝘁𝗼𝗽 𝗿𝗲𝗹𝗲𝗮𝘀𝗲𝘀 Mistral AI just released Pixtral-12B, a vision models that seems to perform extremely well! From Mistral’s own benchmark, it beats the great Qwen2-7B and Llava-OV. 🤔 But Mistral’s benchmarks evaluate in Chain-of-Thought, and even in CoT they show lower scores for other models than the scores already published in non-CoT, which is very strange… Evaluation is not a settled science! But it’s only the last of a flurry of great models. Here are the ones currently squatting the top of the Models Hub page: ❶ 🔊 𝐋𝐥𝐚𝐦𝐚-𝟑.𝟏-𝟖𝐁 𝐎𝐦𝐧𝐢, a model built upon Llama-3.1-8B-Instruct, that simultaneously generates text and speech response with an extremely low latency of 250ms (Moshi, Kyutai’s 8B, did 140ms) ❷ 🐟🗣️ 𝐅𝐢𝐬𝐡 𝐒𝐩𝐞𝐞𝐜𝐡 𝐯𝟏.𝟒, text-to-speech model that supports 8 languages 🇬🇧🇨🇳🇩🇪🇯🇵🇫🇷🇪🇸🇰🇷🇸🇦 with extremely good quality for a light size (~1GB weights) and low latency ❸ 🐳 𝐃𝐞𝐞𝐩𝐒𝐞𝐞𝐤-𝐕𝟐.𝟓, a 236B model with 128k context length that combines the best of DeepSeek-V2-Chat and the more recent DeepSeek-Coder-V2-Instruct. Depending on benchmarks, it ranks just below Llama-3.1-405B. Released with custom ‘deepseek’ license, quite commercially permissive. ❹ 𝐒𝐨𝐥𝐚𝐫 𝐏𝐫𝐨 published by Upstage: a 22B model (so inference fits on a single GPU) that comes just under Llama-3.1-70B performance : MMLU: 79, GPQA: 36, IFEval: 84 ❺ 𝐌𝐢𝐧𝐢𝐂𝐏𝐌𝟑-𝟒𝐁, a small model that claims very impressive scores, even beating much larger models like Llama-3.1-8B. Let's wait for more scores because these look too good! Let’s keep looking, more good stuff is coming our way 🔭

View all activity

Organizations

models

None public yet

datasets

None public yet