nifleisch commited on
Commit
742ada5
·
verified ·
1 Parent(s): ecada08

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -10
README.md CHANGED
@@ -58,18 +58,16 @@ You can run the smashed model with these steps:
58
  pip install autoawq
59
  ```
60
  2. Load & run the model.
61
- ```python
62
- from transformers import AutoModelForCausalLM, AutoTokenizer
63
- from awq import AutoAWQForCausalLM
64
 
65
- model = AutoAWQForCausalLM.from_quantized("PrunaAI/OpenLLM-France-Lucie-7B-AWQ-4bit-smashed", trust_remote_code=True, device_map='auto')
66
- tokenizer = AutoTokenizer.from_pretrained("OpenLLM-France/Lucie-7B")
67
-
68
- input_ids = tokenizer("Quelle est la couleur des pruneaux?", return_tensors='pt').to(next(model.parameters()).device)["input_ids"]
69
-
70
- outputs = model.generate(input_ids, max_new_tokens=216)
71
- tokenizer.decode(outputs[0])
72
 
 
 
 
73
  ```
74
 
75
  ## Configurations
 
58
  pip install autoawq
59
  ```
60
  2. Load & run the model.
61
+ ```python
62
+ from transformers import AutoModelForCausalLM, AutoTokenizer
63
+ from awq import AutoAWQForCausalLM
64
 
65
+ model = AutoAWQForCausalLM.from_quantized("PrunaAI/OpenLLM-France-Lucie-7B-AWQ-4bit-smashed", trust_remote_code=True, device_map='auto')
66
+ tokenizer = AutoTokenizer.from_pretrained("OpenLLM-France/Lucie-7B")
 
 
 
 
 
67
 
68
+ input_ids = tokenizer("Quelle est la couleur des pruneaux?", return_tensors='pt').to(next(model.parameters()).device)["input_ids"]
69
+ outputs = model.generate(input_ids, max_new_tokens=216)
70
+ tokenizer.decode(outputs[0])
71
  ```
72
 
73
  ## Configurations