Lumisync - SpecLM

• Classification and Auto-properties selection
  • I was testing one shot with no system prompt for all the models on perplexity labs and was shocked at the smaller models ability when there is an internet search run and added to the prompt.
  • Previous experiences with these models was sub-par, but with internet access through an agent framework, this can be done with these models.

3/8/24 ##UPDATE: MODEL TESTING ###

• Mistral-7b-Instruct-2.0
• CodeLlama 34b GGUF
• mistral-medium

• This is using mistral-medium API calls, but will be running locally as soon as the weights are released.
• I still need to adjust parameters and tasks/prompts, but it seems to be working.
• The context window is limited to 32k, so i need to chunk the inputs for longer documents, working on that now.
• Also, need to adjust for JSON input...doing that as well.