Model Card for Model ID

This is a Llama-2-7b model fine-tuned on MQuAKE using Localized Fine-tuning on LLM Representations (LoFiT; https://arxiv.org/abs/2406.01563). This model checkpoint modifies the attention outputs of 96 attention heads (10% of all attention heads).

Model Description

License: mit
Finetuned from model: meta-llama/Llama-2-7b-hf

Model Sources

Repository: https://github.com/fc2869/lo-fit
Paper: https://arxiv.org/abs/2406.01563

Uses

Please use the lofit github repo (https://github.com/fc2869/lo-fit) and then use the following code snippet to run evaluations on MQuAKE in the repo with this checkpoint.

from models.modeling_llama import LlamaModel,LlamaForCausalLM
from transformers import AutoTokenizer
import torch
from utils.evaluate import evaluate_mquake
from utils.dataloaders import MQUAKE

checkpoint = 'fcyin/llama2_7B_base_lofit_mquake'
model_name = 'llama2_7B_base_lofit_mquake'
device = 'cuda'
cache_dir = './'
applied_module = 'attention'
torch_dtype = torch.float32

model = LlamaForCausalLM.custom_from_pretrained(checkpoint,
                                                device_map=device, 
                                                cache_dir=cache_dir,
                                                applied_module = applied_module,
                                                torch_dtype=torch_dtype).to(device)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
dataloader = MQUAKE(
             split_dir = './dataset/MQuAKE',
             chat_template = False,
             model_name = model_name
         )
dataset = dataloader.load_data()

evaluate_mquake(eval_dataset=dataset['test'],model_name=model_name,model=model,tokenizer=tokenizer,fname='./',batch_size=16,max_new_tokens=16,apply_chat_template=False)

Training Details

Please refer to the paper for the training details.