doberst commited on
Commit
34f26ef
1 Parent(s): 21172b0

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +108 -0
README.md CHANGED
@@ -1,3 +1,111 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Model Card for Model ID
6
+
7
+ <!-- Provide a quick summary of what the model is/does. -->
8
+
9
+ **slim-sentiment** is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.
10
+
11
+ slim-sentiment has been fine-tuned for **sentiment analysis** function calls, generating output consisting of JSON dictionary corresponding to specified keys.
12
+
13
+ Each slim model has a corresponding 'tool' in a separate repository, e.g.,
14
+
15
+ [**'slim-sentiment-tool'**](https://huggingface.co/llmware/slim-sentiment-tool), which a 4-bit quantized gguf version of the model that is intended to be used for inference.
16
+
17
+ Inference speed and loading time is much faster with the 'tool' versions of the model.
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+ - **Developed by:** llmware
24
+ - **Model type:** Small, specialized LLM
25
+ - **Language(s) (NLP):** English
26
+ - **License:** Apache 2.0
27
+ - **Finetuned from model:** Tiny Llama 1B
28
+
29
+ ## Uses
30
+
31
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
32
+
33
+ The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.
34
+
35
+ Example:
36
+
37
+ text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."
38
+
39
+ model generation - {"sentiment": ["negative"]}
40
+
41
+ keys = "sentiment"
42
+
43
+ All of the SLIM models use a novel prompt instruction structured as follows:
44
+
45
+ "<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "
46
+
47
+
48
+ ## How to Get Started with the Model
49
+
50
+ The fastest way to get started with BLING is through direct import in transformers:
51
+
52
+ import ast
53
+ from transformers import AutoModelForCausalLM, AutoTokenizer
54
+
55
+ model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
56
+ tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
57
+
58
+ text = "The markets declined for a second straight days on news of disappointing earnings."
59
+
60
+ keys = "sentiment"
61
+
62
+ prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "
63
+
64
+ # huggingface standard generation script
65
+ inputs = tokenizer(prompt, return_tensors="pt")
66
+ start_of_output = len(inputs.input_ids[0])
67
+
68
+ outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
69
+ pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)
70
+
71
+ output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)
72
+
73
+ print("input text sample - ", text)
74
+ print("llm_response - ", output_only)
75
+
76
+ # where it gets interesting
77
+ try:
78
+ # convert llm response output from string to json
79
+ output_only = ast.literal_eval(output_only)
80
+ print("converted to json automatically")
81
+
82
+ # look for the key passed in the prompt as a dictionary entry
83
+ if keys in output_only:
84
+ if "negative" in output_only[keys]:
85
+ print("sentiment appears negative - need to handle ...")
86
+ else:
87
+ print("response does not appear to include the designated key - will need to try again.")
88
+
89
+ except:
90
+ print("could not convert to json automatically - ", output_only)
91
+
92
+
93
+ ## Using as Function Call in LLMWare
94
+
95
+ We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.
96
+
97
+ Check out llmware for one such implementation:
98
+
99
+ from llmware.models import ModelCatalog
100
+ slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
101
+ response = slim_model.function_call(text,params=["sentiment"], function="classify")
102
+
103
+ print("llmware - llm_response: ", response)
104
+
105
+
106
+ ## Model Card Contact
107
+
108
+ Darren Oberst & llmware team
109
+
110
+
111
+