{ "cells": [ { "cell_type": "markdown", "id": "963e9ae0-ac68-44be-8c7d-fb9842784362", "metadata": {}, "source": [ "# 4.6 基于llama的基因大模型指令微调" ] }, { "cell_type": "markdown", "id": "182b82c4-d484-4c15-a600-03c3b51367ec", "metadata": {}, "source": [ "**PEFT**（Parameter-Efficient Fine-Tuning，参数高效微调）是一种优化技术，旨在以最小的参数更新实现对大规模预训练模型（如 GPT、BERT 等）的微调。PEFT 技术通过减少微调所需的参数量，显著降低了存储和计算开销，同时保留模型的性能，特别适合资源受限的场景和领域特定任务的定制化。\n", "\n", "---\n", "\n", "### **1. 核心思想**\n", "传统的微调方式需要更新整个预训练模型的所有参数，PEFT 技术通过只调整少量的参数（如特定层或额外添加的小型模块）实现微调目标，大幅减少了训练开销和存储需求。\n", "\n", "---\n", "\n", "### **2. 常见的 PEFT 方法**\n", "\n", "#### **（1）Adapter 模型**\n", "- 在每一层 Transformer 的输出中插入小型适配器模块，仅训练适配器模块的参数。\n", "- 原始模型参数保持冻结不变。\n", "- 优点：适配器模块参数量小，能适应不同任务。\n", "\n", "示例方法：\n", "- **AdapterFusion**\n", "- **MAD-X**\n", "\n", "---\n", "\n", "#### **（2）Prefix Tuning**\n", "- 在 Transformer 的输入前添加一组可学习的前缀向量，这些前缀与模型的注意力机制交互。\n", "- 只调整前缀向量的参数，而不更新原始模型。\n", "- 优点：对生成任务效果显著，参数量进一步减少。\n", "\n", "---\n", "\n", "#### **（3）LoRA（Low-Rank Adaptation）**\n", "- 将预训练模型中的部分权重分解为两个低秩矩阵，仅调整这些低秩矩阵的参数。\n", "- 原始权重保持冻结状态。\n", "- 优点：参数量极小，计算高效。\n", " \n", "---\n", "\n", "#### **（4）Prompt Tuning**\n", "- 在输入文本中添加可学习的提示（Prompt）。\n", "- 适合 NLP 任务中的文本生成、分类等。\n", "- 优点：实现简单，易于集成到现有框架。\n", "\n", "---\n", "\n", "### **3. PEFT 的优势**\n", "\n", "1. **显著减少参数更新量**：\n", " - 微调传统的大模型（如 GPT-3）需要更新数百亿参数，而 PEFT 仅需更新百万级别甚至更少的参数。\n", "\n", "2. **高效存储**：\n", " - 每个任务的微调结果只需存储少量额外参数，而不是整个模型。\n", "\n", "3. **适用多任务**：\n", " - 同一预训练模型可以通过不同的 PEFT 模块适配多个任务，无需重新训练。\n", "\n", "4. **降低计算开销**：\n", " - 训练所需的内存和计算显著减少，适合资源有限的环境。\n", "\n", "---\n", "\n", "### **4. 应用场景**\n", "\n", "1. **领域特定任务**：\n", " - 医疗、法律、金融等领域微调预训练模型。\n", "\n", "2. **多任务学习**：\n", " - 适配多个任务，复用同一模型的预训练权重。\n", "\n", "3. **资源受限场景**：\n", " - 移动设备、边缘设备上的模型部署。\n", "\n", "---\n", "\n", "### **5. Hugging Face PEFT 库**\n", "\n", "Hugging Face 提供了专门的 PEFT 库，支持多种参数高效微调技术：\n", "- **安装**：\n", " ```bash\n", " pip install peft\n", " ```\n", "- **使用 LoRA 微调示例**：\n", " ```python\n", " from transformers import AutoModelForCausalLM, AutoTokenizer\n", " from peft import LoraConfig, get_peft_model, TaskType\n", "\n", " # 加载模型和分词器\n", " model_name = \"gpt2\"\n", " model = AutoModelForCausalLM.from_pretrained(model_name)\n", " tokenizer = AutoTokenizer.from_pretrained(model_name)\n", "\n", " # 配置 LoRA\n", " lora_config = LoraConfig(\n", " task_type=TaskType.CAUSAL_LM,\n", " r=8,\n", " lora_alpha=32,\n", " target_modules=[\"q_proj\", \"v_proj\"],\n", " lora_dropout=0.1,\n", " bias=\"none\"\n", " )\n", "\n", " # 使用 LoRA 微调模型\n", " model = get_peft_model(model, lora_config)\n", " model.print_trainable_parameters()\n", "\n", " # 微调代码...\n", " ```\n", "\n", "---\n", "\n", "### **6. PEFT 的局限性**\n", "1. **特定任务限制**：\n", " - 在一些复杂任务中，PEFT 方法可能不如全量微调效果好。\n", "\n", "2. **需要设计合适的模块**：\n", " - 不同任务需要选择和设计合适的 PEFT 技术。\n", "\n", "3. **与模型架构相关**：\n", " - PEFT 技术可能需要对模型架构进行一定程度的修改。\n", "\n", "---\n", "\n", "### **7. 总结**\n", "PEFT 是一个极具潜力的技术，特别适合在有限资源下对大模型进行微调。它在许多领域和任务中已显示出良好的效果，例如 LoRA 和 Adapter 模型已经成为高效微调的主流方法。\n", "\n", "如果您需要实现高效微调，可以结合 Hugging Face 的 PEFT 库快速上手。" ] }, { "cell_type": "code", "execution_count": 1, "id": "5aa3d240-44e1-4811-8f61-d6ff2500a798", "metadata": {}, "outputs": [], "source": [ "import subprocess\n", "import os\n", "# 设置环境变量, autodl一般区域\n", "result = subprocess.run('bash -c \"source /etc/network_turbo && env | grep proxy\"', shell=True, capture_output=True, text=True)\n", "output = result.stdout\n", "for line in output.splitlines():\n", " if '=' in line:\n", " var, value = line.split('=', 1)\n", " os.environ[var] = value" ] }, { "cell_type": "markdown", "id": "17bdb69d-3f0f-465e-bd60-2047a088e264", "metadata": {}, "source": [ "如果您不确定模型中有哪些模块可以微调，可以打印模型结构：" ] }, { "cell_type": "code", "execution_count": 2, "id": "41a0c049-9134-4d89-aad0-1aa2241a9fca", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4becc479adbc472bb7672d49da16aafd", "version_major": 2, "version_minor": 0 }, "text/plain": [ "generation_config.json: 0%| | 0.00/124 [00:00