--- library_name: transformers license: cc-by-4.0 datasets: - hendrycks/ethics --- # Model Card for Model ID Fine-tuned version of Phi-3-mini-4k-instruct on a subset of the hendrycks/ethics dataset ## How to Get Started with the Model Use the code below to get started with the model. ```markdown Install the latest version of the following python libraries: -torch -accelerate -peft -bitsandbytes ``` Run the model ```python from transformers import AutoModelForCausalLM from peft import PeftModel base_model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct") peft_model_id = "fc91/phi3-mini-instruct-full_ethics-lora" model = PeftModel.from_pretrained(base_model, peft_model_id) ``` Run the model with a quantization configuration ```python import torch, accelerate, peft from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline from peft import PeftModel # Set up quantization configuration quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=getattr(torch, "float16") ) # Load the base model with quantization base_model = AutoModelForCausalLM.from_pretrained( "microsoft/Phi-3-mini-4k-instruct", quantization_config=quantization_config, device_map="auto", attn_implementation='eager', torch_dtype="auto", trust_remote_code=True, ) peft_model_id = "fc91/phi3-mini-instruct-full_ethics-lora" model = PeftModel.from_pretrained(base_model, peft_model_id) tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct") messages = [ {"role": "system", "content": "You are a helpful AI assistant that grounds all of its replies in ethical theories."}, {"role": "user", "content": """I am driving a car, and I have to make a choice. A kid suddenly appeared in the middle of the road chasing a ball. To save the kid, I can only swerve to the right, but this would entail crashing the car against two pedestrians on the sidewalk. What should I do?"""}, ] pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer, ) generation_args = { "max_new_tokens": 1000, "return_full_text": False, "temperature": 0.5, "do_sample": False, } # Run inference output = pipe(messages, **generation_args) print(output[0]['generated_text']) ``` ## Training Details ### Training Data ["hendrycks/ethics"](https://huggingface.co./datasets/hendrycks/ethics) ```markdown The following subsets of the above dataset were leveraged: -commonsense/train (10k random samples) -deontology/train (10k random samples) -justice/train (10k random samples) -utilitarianism/train (10k random samples) ``` ### Training Procedure #### Training Hyperparameters ```python per_device_train_batch_size=16 per_device_eval_batch_size=32 gradient_accumulation_steps=2 gradient_checkpointing=True warmup_steps=100 num_train_epochs=1 learning_rate=0.00005 weight_decay=0.01 optim="adamw_hf" fp16=True ``` #### Speeds, Sizes, Times The overall training took 3 hours and 23 minutes. ## Evaluation Training Loss = 0.181700 Validation Loss = 0.119734 ### Testing Data, Factors & Metrics #### Testing Data ["hendrycks/ethics"](https://huggingface.co./datasets/hendrycks/ethics) ```markdown The following subsets of the above dataset were leveraged: -commonsense/test (2.5k random samples) -deontology/test (2.5k random samples) -justice/test (2.5k random samples) -utilitarianism/test (2.5k random samples) ``` #### Hardware NVIDIA A100-SXM4-40GB