Not exactly replicatable since the absence of LAM

I would also use a https://huggingface.co./microsoft/phi-2 model, we need a smaller model for quick inference for easy queries

I would use ph-2 on device for daily conversation and detecting users intention, then pass to cloud hosted much larger LLM to do more complicated stuff (such as web browsing and more).

How would you aim for the cheapest latency using existing tooling?

Are you thinking of running it on a device or in the cloud?

or a https://huggingface.co./distil-whisper/distil-large-v2 for even faster speech-to-text

• facebook's Seamless M4T for the audio and voice to make multilingual , there's an ondevice model demo here: https://huggingface.co./spaces/Tonic/SeamlessOnDevice

Same models but with SOLAR 10.7B Instruct, as it has similar performance with less parameters

We are starting like those at pi5, we called 'piBrain'

whisper-tiny - tinyllama - vits

everyone is focusing on the model side, but what about the hardware, id be interested in seeing some Open source versions of this by the community.