Update README.md

README.md

@@ -8,7 +8,43 @@ tags: []
 <!-- Provide a quick summary of what the model is/does. -->
 
 
-
+# How to Get Started with the Model
+To load and use this model with the Transformers library by Hugging Face, follow the steps outlined in the code snippet below. This code demonstrates how to configure the model, load it along with its tokenizer, and perform inference to generate text based on a given prompt.
+
+## Code Format:
+```python
+from peft import PeftModel, PeftConfig
+
+test_config = PeftConfig.from_pretrained("checkiejan/phi2-marking-test-full")
+model_base = AutoModelForCausalLM.from_pretrained(
+    test_config.base_model_name_or_path,
+    device_map="auto",
+    quantization_config=BitsAndBytesConfig(
+        load_in_4bit=True,
+        bnb_4bit_compute_dtype=torch.bfloat16,
+        bnb_4bit_quant_type="nf4",
+    ),
+    torch_dtype=torch.bfloat16,
+    trust_remote_code=True,
+)
+tokenizer = AutoTokenizer.from_pretrained(test_config.base_model_name_or_path)
+
+# Add/set tokens same tokens to base model before merging, like we did before starting training  
+tokenizer.add_tokens(["<|im_start|>", "<PAD>"])
+tokenizer.pad_token = "<PAD>"
+tokenizer.add_special_tokens(dict(eos_token="<|im_end|>"))
+model_base.resize_token_embeddings(
+    new_num_tokens=len(tokenizer),
+    pad_to_multiple_of=64)   # phi2 default is 64, see configuration_phi.py
+model_base.config.eos_token_id = tokenizer.eos_token_id
+lora_model = PeftModel.from_pretrained(model_base, "checkiejan/phi2-marking-test-full")
+
+inputs = tokenizer(prompt, return_tensors="pt", return_attention_mask=True)
+outputs = lora_model.generate(**inputs)
+text = tokenizer.batch_decode(outputs, skip_special_tokens=True)
+print(''.join(text))
+```
+This code snippet sets up the model and tokenizer, configures the necessary parameters, and demonstrates how to generate text based on a given prompt. Ensure to replace "Your prompt here" with your actual input text.
 ## Model Details
 
 ### Model Description