hi @hydroweaver !
You can do the following model = AutoModelForCausalLM.from_pretrained("jadechoghari/Ferret-UI-Llama8b", trust_remote_code=True), however you would need to setup the tokenizer (of ferret) + some other preprocessing things wich the code in the repo abstracts. Hope this helps 🤗!

Thanks for the response @jadechoghari ! Just to confirm one thing, I don't think we can run this on CPU right (running on windows), faced multiple errors as follows:

1. inference.py image_path is set to . instead of image_path
2. inference.py when run via main.py gets stuck in the following error:

eval.json file created successfully.
Error occurred during inference:
Command '['python', '-m', 'model_UI', '--model_path', 'jadechoghari/Ferret-UI-Gemma2b', '--data_path', 'eval.json', '--image_path', 'image.png', '--answers_file', 'eval_output.jsonl', '--num_beam', '1', '--max_new_tokens', '32', '--conv_mode', 'ferret_gemma_instruct']' returned non-zero exit status 1.
Subprocess output:

Inference Text: (None, None)

3. However, on running model_UI.py as follows, the model gets downloaded:

python -m model_UI --model_path "jadechoghari/Ferret-UI-Gemma2b" --data_path "eval.json" --image_path "image.png" --answers_file "eval_output.jsonl" --num_beam 1 --max_new_tokens 32 --conv_mode "ferret_gemma_instruct"

4.After model download, it gets stuck as follows: AssertionError: Torch not compiled with CUDA enabled