Stream response of Falcon models
#39
by
kronus86
- opened
Is it possible to stream output of Falcon models (similar to ChatGPT stream option https://platform.openai.com/docs/guides/gpt).
Please use hugging face text generation inference, not sure falcon 7b is supported. if supported then use its generate_stream function to get the same behavour.
Thanks for you reply. I checked and it does support Flacon 7B and 40B models.
kronus86
changed discussion status to
closed