{ "cells": [ { "cell_type": "markdown", "id": "c2e6b786-d18e-4a0d-aa59-0792dcb49c5f", "metadata": {}, "source": [ "# 4.2 基于GPT2的指令微调" ] }, { "cell_type": "markdown", "id": "02cd6e13-bbfb-413a-8236-ff092456fd1c", "metadata": {}, "source": [ "我还是用第二章中的分类的例子，使用指令微调的形式，来再次解决分类问题。\n", "\n", "使用 GPT-2 进行文本分类的两种方法：**使用 GPT-2 的分类头（Classification Header）** 和 **将分类任务转换为指令微调**，在思路、实现、优劣势和适用场景上存在明显差异。以下是详细对比：\n", "\n", "---\n", "\n", "### **1. 核心思路**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **基本概念** | 在 GPT-2 顶部添加一个分类头（通常是一个线性层），直接预测分类标签。 | 将分类任务转化为自然语言指令，模型通过微调理解并完成指令形式的任务。 |\n", "| **实现方式** | 修改 GPT-2 模型，添加 `num_labels` 分类头并定义分类损失函数。 | 构建任务指令数据（Instruction + Input + Output），然后微调模型。 |\n", "| **数据形式** | 文本与其分类标签的直接映射。 | 文本通过指令转化为生成任务。例如：
`Input`: 文章内容
`Output`: 分类结果。 |\n", "\n", "---\n", "\n", "### **2. 数据格式**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **数据格式** | - 输入：文本
- 标签：离散类别标签（如 0, 1, 2）。 | - 指令：自然语言描述任务（如 \"请分类以下文本\"）。
- 输入：分类文本。
- 输出：分类结果（文本形式）。 |\n", "| **示例** | 输入：`\"This is a happy day!\"`
标签：`1`（表示积极） | `Instruction`: \"请对以下文本进行情感分类\"
`Input`: `\"This is a happy day!\"`
`Output`: `\"积极\"` |\n", "\n", "---\n", "\n", "### **3. 模型结构**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **模型结构** | - GPT-2 + 分类头（线性层）。 | - GPT-2 原始结构，无需额外的分类头。 |\n", "| **损失函数** | - 使用交叉熵损失（Cross Entropy Loss）。 | - 使用自回归的语言建模损失（Language Modeling Loss）。 |\n", "\n", "---\n", "\n", "### **4. 训练过程**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **微调对象** | 主要微调分类头部分的参数（可选择冻结 GPT-2 的主干部分）。 | 微调整个 GPT-2 模型（或使用参数高效微调如 LoRA）。 |\n", "| **标签处理** | 离散化标签（如 0, 1, 2）。 | 标签转化为自然语言（如“积极”、“中立”、“消极”）。 |\n", "| **训练难度** | - 简单，标准分类任务流程。
- 数据需求较小，适合小规模微调。 | - 复杂，需要构造高质量的指令数据集。
- 数据需求较大，适合多任务场景。 |\n", "\n", "---\n", "\n", "### **5. 优缺点分析**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **优点** | - 训练速度快，计算资源需求较低。
- 实现简单，适合单一任务。 | - 泛化能力强，支持多任务扩展。
- 与多任务微调和开放式生成兼容。 |\n", "| **缺点** | - 只能处理分类任务，难以扩展为其他任务。
- 需要人工调整分类头和损失函数。 | - 数据构造复杂且对数据质量依赖较高。
- 训练资源需求较大，训练时间较长。 |\n", "\n", "---\n", "\n", "### **6. 适用场景**\n", "\n", "| **方法** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-----------------------------|-------------------------------------------------------------|-------------------------------------------------------|\n", "| **适用场景** | - 单任务文本分类，如情感分析、垃圾邮件检测等。 | - 多任务场景，支持分类、翻译、摘要等任务的统一处理。 |\n", "| **数据规模** | 适合小数据集，数千到数万条数据即可训练效果良好。 | 适合大数据集，特别是多任务、多领域的数据集。 |\n", "| **需求类型** | 专注于提高单一任务的分类准确率。 | 需要增强模型的多任务泛化能力，同时提升用户交互体验。 |\n", "\n", "---\n", "\n", "### **7. 综合对比总结**\n", "\n", "| **维度** | **使用 GPT-2 分类头** | **转换为指令微调** |\n", "|-------------------------|--------------------------------------------------------------|-------------------------------------------------------|\n", "| **实现复杂度** | 较低，直接添加分类头并使用标准分类流程即可完成。 | 较高，需要构造高质量指令数据，并调整训练流程。 |\n", "| **资源需求** | 较低，仅需调整分类头部分，训练时间和显存消耗较少。 | 较高，需要微调整个模型，且对数据和算力需求更大。 |\n", "| **性能表现** | 对单一分类任务效果较好，但泛化能力较弱。 | 在多任务、多样化分类场景中表现更强，且可扩展为其他任务类型。 |\n", "| **扩展性** | 较差，仅适用于当前任务，难以迁移到其他任务。 | 较强，可适应多任务指令和开放式生成场景。 |\n", "\n", "---\n", "\n", "### **选择建议**\n", "\n", "1. **使用 GPT-2 分类头**：\n", " - 如果任务是单一分类问题（如情感分析、垃圾邮件检测），并且数据量有限，推荐使用分类头方法。\n", " - 适合快速实现和部署，无需复杂的预处理和指令数据集构建。\n", "\n", "2. **转换为指令微调**：\n", " - 如果任务需要多样化（分类+生成+翻译等），或需要对未见任务有更好的泛化能力，推荐使用指令微调。\n", " - 适合多任务、多场景部署，尤其是在 ChatGPT 风格的应用中更为适用。\n", "\n", "通过综合任务需求、数据规模和资源条件选择合适的方法，能够有效提升模型性能并实现更广泛的适用性。\n", "\n", "\n", "原始的数据格式如下：\n", "| sequence | label | label_name |\n", "|--------------------------------------------------------|-------|----------------|\n", "| TATATTTTCTCAGCTGAGTTAATTAGTTTCACTAGTTAACTGAGAATAAAAGAA | 1 | promoter |\n", "| TGGGGAGGGTCCGGTGTTAGTTAGATACATCCCCAGACCCACACCCCGGATAGA | 0 | Non-promoter |\n", "\n", "转成指令的格式为：\n", "```\n", "{'instruction': 'Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.', \n", "'input': 'CATGCGGGTCG...', \n", "'output': 'Non-promoter'}\n", "```\n", "\n", "然后写成指令微调数据格式，当做一般的文本进行训练：\n", "```\n", "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n", "### Instruction:\n", "Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.\n", "### Input:\n", "TCTTTCTCTTCTGTATCATTCTACTT...\n", "### Response:\n", "Non-promoter\n", "```\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "b28e3499-bbff-4548-9f85-2baee088cabf", "metadata": {}, "outputs": [], "source": [ "import subprocess\n", "import os\n", "# 设置环境变量, autodl一般区域\n", "result = subprocess.run('bash -c \"source /etc/network_turbo && env | grep proxy\"', shell=True, capture_output=True, text=True)\n", "output = result.stdout\n", "for line in output.splitlines():\n", " if '=' in line:\n", " var, value = line.split('=', 1)\n", " os.environ[var] = value" ] }, { "cell_type": "code", "execution_count": 6, "id": "dc04d5e3-7623-4d59-9f3b-ad03e339db11", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatasetDict({\n", " train: Dataset({\n", " features: ['sequence', 'label'],\n", " num_rows: 59195\n", " })\n", "})" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from datasets import load_dataset\n", "# 1. load ~11k samples from promoters prediction dataset\n", "dna_dataset = load_dataset(\"dnagpt/dna_promoter_300\")\n", "dna_dataset" ] }, { "cell_type": "code", "execution_count": 7, "id": "f7332fa1-3343-4247-b4cc-54733dff6964", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'sequence': 'TAAATACGGAAGTTTATTACTTGAGGAATAGATGGAATCGTCGGGCGTGAGAGATCATAATCGGCTGCTTCTGGGAGCCGCACGTGGGAAAGACTTATCCCCGACGGAGCTGGGACTGGGGCACAAACCGGAAGGAACACATCTGACCGAGAAAGAGACCAAGTGGCTCAGGTAGGACCAAAGCGAGCAAGGCTGCGGGTCCTGTTGCTCTCTGTCCTGTAAATTTAAACGTTACGCCACCTGGTAATGATACCCTCGTCCTCCGAGGCGACAAGTCAGAACTTCCACCAAGGGCATTAC',\n", " 'label': 0}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dna_dataset[\"train\"][0]" ] }, { "cell_type": "code", "execution_count": 8, "id": "93d09d8d-f521-49f7-b0e0-7ac089dfbf49", "metadata": {}, "outputs": [], "source": [ "def build_prompt(example):\n", " if int(example['label']) == 1:\n", " label = 'promoter'\n", " else:\n", " label = 'Non-promoter'\n", "\n", " instruction = \"Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.\"\n", " \n", " input = example[\"sequence\"]\n", " input_text = f\"\\n\\n### Input:\\n{input}\"\n", "\n", "\n", " output = label\n", "\n", " prompt = {\"instruction\":instruction, \n", " \"input\":input,\n", " \"output\":output\n", " }\n", "\n", " return prompt" ] }, { "cell_type": "code", "execution_count": 9, "id": "9f9c0e5a-6591-47ac-b358-d746a00dfc0a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'instruction': 'Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.', 'input': 'TAAATACGGAAGTTTATTACTTGAGGAATAGATGGAATCGTCGGGCGTGAGAGATCATAATCGGCTGCTTCTGGGAGCCGCACGTGGGAAAGACTTATCCCCGACGGAGCTGGGACTGGGGCACAAACCGGAAGGAACACATCTGACCGAGAAAGAGACCAAGTGGCTCAGGTAGGACCAAAGCGAGCAAGGCTGCGGGTCCTGTTGCTCTCTGTCCTGTAAATTTAAACGTTACGCCACCTGGTAATGATACCCTCGTCCTCCGAGGCGACAAGTCAGAACTTCCACCAAGGGCATTAC', 'output': 'Non-promoter'}\n" ] } ], "source": [ "example = dna_dataset[\"train\"][0]\n", "print(build_prompt(example))" ] }, { "cell_type": "code", "execution_count": 10, "id": "83070a23-1604-4d28-b371-e01060331ed5", "metadata": {}, "outputs": [], "source": [ "import json\n", "ins_file = open(\"data/dna_promoter_300.jsonl\", \"w\")\n", "ins_list = []\n", "for ins in dna_dataset[\"train\"]:\n", " if ins[\"sequence\"]==\"sequence\":\n", " continue\n", " ins = build_prompt(ins)\n", " ins_file.write(json.dumps(ins)+\"\\n\")\n", " ins_list.append(ins)\n", "ins_file.close()" ] }, { "cell_type": "code", "execution_count": 11, "id": "89fb8ed3-aa58-462f-b2a6-ce445c597a33", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "7b84d3a64bf645ada13d0cada2d9f524", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Generating train split: 0 examples [00:00, ? examples/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "DatasetDict({\n", " train: Dataset({\n", " features: ['instruction', 'input', 'output'],\n", " num_rows: 59195\n", " })\n", "})" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dna_ft_dataset = load_dataset(\"json\", data_files='data/dna_promoter_300.jsonl')\n", "dna_ft_dataset" ] }, { "cell_type": "code", "execution_count": 12, "id": "e4f7b75f-6ccb-4fda-8004-40df7d52678f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DatasetDict({\n", " train: Dataset({\n", " features: ['instruction', 'input', 'output'],\n", " num_rows: 53275\n", " })\n", " test: Dataset({\n", " features: ['instruction', 'input', 'output'],\n", " num_rows: 5920\n", " })\n", "})" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data = dna_ft_dataset[\"train\"].train_test_split(train_size=0.9, seed=42)\n", "data" ] }, { "cell_type": "code", "execution_count": 13, "id": "36d9ee0e-8423-4529-aa7e-fda2728fab2f", "metadata": {}, "outputs": [], "source": [ "# 初始化tokenizer\n", "from datasets import load_dataset\n", "from transformers import AutoTokenizer, GPT2LMHeadModel, AutoConfig\n", "from transformers import GPT2Tokenizer,GPT2Model,AutoModel\n", "from transformers import DataCollatorForLanguageModeling\n", "from transformers import Trainer, TrainingArguments\n", "from tokenizers import Tokenizer\n", "from transformers import GPT2TokenizerFast\n", "\n", "#需要使用生物序列+英文的多模态大模型\n", "tokenizer = GPT2Tokenizer.from_pretrained(\"dnagpt/gene_eng_gpt2_v0\")\n", "tokenizer.pad_token = tokenizer.eos_token" ] }, { "cell_type": "code", "execution_count": 14, "id": "871baee0-f06f-4422-a741-af533f7d92e1", "metadata": {}, "outputs": [], "source": [ "#构建提示词\n", "def format_input(entry):\n", " instruction_text = (\n", " f\"Below is an instruction that describes a task. \"\n", " f\"Write a response that appropriately completes the request.\"\n", " f\"\\n\\n### Instruction:\\n{entry['instruction']}\"\n", " )\n", "\n", " input_text = f\"\\n\\n### Input:\\n{entry['input']}\" if entry[\"input\"] else \"\"\n", "\n", " return instruction_text + input_text + \"\\n\\n### Response:\\n\"\n", "\n", "#构建提示词\n", "def build_prompt(entry):\n", "\n", " input_data = format_input(entry)\n", "\n", " desired_response = entry['output']\n", "\n", " return input_data + desired_response" ] }, { "cell_type": "code", "execution_count": 15, "id": "bca1c275-cc3d-43df-923e-e6604d584226", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'instruction': 'Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.',\n", " 'input': 'CCAGGATGCGCTGACGACCCGGCTGGCAGGCGGGTCCTCGTGGGCGAGGCGAGGGAGGCGGCGAGAGAGGAGCAATAGTTTCCCACCGCTCCCTCTCAGGCGCAGGGTCTAGAGAAGCGCGAGGGGATCTAGAGAAGCCGGAGGGGAGGAAGCGCGAGTCCGCGGCCCGCCCCGTTGCGTCCCACCCACCGCGTCCCCTCCCCTCCCCTCCCGCTGCGGGAAAAGCGGCCGCGGGCGGCGGCGCCCACTGTGGGGCGGGCGGAGCGCCGCGGGAGGCGGACGAGATGCGAGCGCGGCCGC',\n", " 'output': 'promoter'}" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "example = data[\"test\"][0]\n", "example" ] }, { "cell_type": "code", "execution_count": 16, "id": "76f2e027-0a31-4919-bb7e-404c786e1599", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n", "\n", "### Instruction:\n", "Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.\n", "\n", "### Input:\n", "CCAGGATGCGCTGACGACCCGGCTGGCAGGCGGGTCCTCGTGGGCGAGGCGAGGGAGGCGGCGAGAGAGGAGCAATAGTTTCCCACCGCTCCCTCTCAGGCGCAGGGTCTAGAGAAGCGCGAGGGGATCTAGAGAAGCCGGAGGGGAGGAAGCGCGAGTCCGCGGCCCGCCCCGTTGCGTCCCACCCACCGCGTCCCCTCCCCTCCCCTCCCGCTGCGGGAAAAGCGGCCGCGGGCGGCGGCGCCCACTGTGGGGCGGGCGGAGCGCCGCGGGAGGCGGACGAGATGCGAGCGCGGCCGC\n", "\n", "### Response:\n", "promoter\n" ] } ], "source": [ "prompt = build_prompt(example)\n", "print(prompt)" ] }, { "cell_type": "code", "execution_count": 17, "id": "932b54ca-7e27-47cd-b67d-7ef8386b6608", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tokens: Bel ow Ġ is Ġ an Ġ instruc tion Ġ th at Ġ describ es Ġ a Ġ t ask . ĠWrit e Ġ a Ġ respon se Ġ th at Ġ appropri at el y Ġ complet es Ġ the Ġ request . Ċ Ċ # # # ĠIn struc tion : Ċ D eter min e Ġ cor e Ġ promo ter Ġ det ec tion Ġ of Ġ follow ing Ġ d na Ġ sequenc e , ĠTh e Ġ resul t Ġ will Ġ be Ġ on e Ġ of Ġ the Ġ follow ing : ĠN on - promo ter , Ġ promo ter . Ċ Ċ # # # ĠIn put : Ċ CC AGGATGC GC TGACG ACCC GGCTGGC AGGC GGGTCC TCG TGGGCG AGGCG AGGGAGGC GGCG AGAGAGG AGCAATAG TTTCCC ACCGC TCCCTCTC AGGCGC AGGG TCTAG AGAAGC GCG AGGGG ATCTAG AGAAGCC GG AGGGG AGGAAGC GCG AGTCC GCGG CCCGCC CCG TTGCG TCCC ACCCACC GCG TCCCCTCCCC TCCCCTCCC GCTGC GGG AAAAGC GGCCGC GGGCGGC GGCGCCC ACTGTG GGGC GGGC GGAGC GCCGC GGGAGGC GGACG AGATGCG AGCGC GGCCGC Ċ Ċ # # # ĠR esp on se : Ċ promo ter\n" ] } ], "source": [ "print('tokens: ', ' '.join(tokenizer.tokenize(prompt)))" ] }, { "cell_type": "code", "execution_count": 18, "id": "26671faf-68d0-4a44-978e-e1a24e86c9b1", "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "02d2f083e74a45e6ada46b9872822dfc", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Map: 0%| | 0/53275 [00:00, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::logic_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::~locale()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::string const&, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_end_catch@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ofstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::logic_error::~logic_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new[](unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak_hard()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ifstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::basic_streambuf(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned short@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::resize(unsigned long, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char const*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ctype::_M_widen_init() const@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_invalid_argument(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::operator=(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_free_exception@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::notify_one()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::~Init()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::~basic_string()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::flush()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_rethrow@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_fstream >::~basic_fstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::compare(char const*) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::chrono::_V2::system_clock::now()@GLIBCXX_3.4.19'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ifstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Hash_bytes(void const*, unsigned long, unsigned long)@CXXABI_1.3.5'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(long long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const@GLIBCXX_3.4.18'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::out_of_range@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(unsigned long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::~ios_base()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::range_error::~range_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file::~__basic_file()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_acquire@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(bool)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::overflow_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_fstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::range_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_filebuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete[](void*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(unsigned long, char, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_transfer(std::__detail::_List_node_base*, std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::exception@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::_Rep::_M_destroy(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream& std::istream::_M_extract(double&)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_fstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::basic_ifstream(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new(unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned int@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::domain_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char, unsigned long) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::put(char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_alloc()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_thread_atexit@CXXABI_1.3.7'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::~basic_ifstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::Init()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::condition_variable()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::basic_filebuf()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::domain_error::~domain_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cerr@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char const*, unsigned long, unsigned long) const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::str() const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::invalid_argument@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(std::string const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostringstream, std::allocator >::~basic_ostringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_unhook()@GLIBCXX_3.4.15'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::_M_sync(char*, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream >::~basic_iostream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale(std::locale const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `log2f@GLIBC_2.2.5'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::exception::~exception()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file::is_open() const@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istringstream, std::allocator >::~basic_istringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::swap(std::string&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::basic_streambuf(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::init(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_cast()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::clear(std::_Ios_Iostate)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::operator=(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(int)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_M_destroy(std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream >::~basic_iostream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::runtime_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ofstream >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringstream, std::allocator >::~basic_stringstream()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::get()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream >& std::operator<< >(std::basic_ostream >&, char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::out_of_range::~out_of_range()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::length_error::~length_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream >& std::__ostream_insert >(std::basic_ostream >&, char const*, long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::invalid_argument::~invalid_argument()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::swap(std::basic_string, std::allocator >&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cout@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(unsigned long long)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(void const*)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::underflow_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::out_of_range@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_allocate_exception@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void const*@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios >::init(std::basic_streambuf >*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::reserve(unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_begin_catch@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::open(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf, std::allocator >::_M_sync(wchar_t*, unsigned long, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::getline(char*, long, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istream >& std::getline, std::allocator >(std::basic_istream >&, std::basic_string, std::allocator >&, char)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::~condition_variable()@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf, std::allocator >@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::insert(unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(char const*, unsigned long)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned char@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::ios_base()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_out_of_range(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::overflow_error::~overflow_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_length_error(char const*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_system_error(int)@GLIBCXX_3.4.11'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ofstream >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert(double)@GLIBCXX_3.4.9'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf >::operator=(std::basic_streambuf > const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long long@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string, std::allocator >::basic_string(char const*, unsigned long, std::allocator const&)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream >::close()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_release@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_throw@CXXABI_1.3'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::underflow_error::~underflow_error()@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::length_error@GLIBCXX_3.4'\n", "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf >::~basic_filebuf()@GLIBCXX_3.4'\n", "collect2: error: ld returned 1 exit status\n" ] } ], "source": [ "# 初始化Trainer\n", "trainer = Trainer(\n", " model=model,\n", " args=training_args,\n", " train_dataset=tokenized_datasets['train'],\n", " eval_dataset=tokenized_datasets['test'],\n", " data_collator=data_collator\n", ")" ] }, { "cell_type": "code", "execution_count": 26, "id": "a9cd936a-5ea6-43e3-9848-27080f818606", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " \n", " [19980/19980 21:55, Epoch 3/3]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StepTraining Loss
5002.335700
10002.184100
15002.178500
20002.172400
25002.171400
30002.171900
35002.163100
40002.159300
45002.161000
50002.160200
55002.158400
60002.151600
65002.153700
70002.129500
75002.119100
80002.119800
85002.121600
90002.122500
95002.122300
100002.121500
105002.119500
110002.123500
115002.119600
120002.119300
125002.121800
130002.123500
135002.103200
140002.080700
145002.082100
150002.082900
155002.086400
160002.086600
165002.083800
170002.085000
175002.082800
180002.077600
185002.080300
190002.086600
195002.084200

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "TrainOutput(global_step=19980, training_loss=2.1272921145021977, metrics={'train_runtime': 1315.5944, 'train_samples_per_second': 121.485, 'train_steps_per_second': 15.187, 'total_flos': 2.08804995072e+16, 'train_loss': 2.1272921145021977, 'epoch': 3.0})" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 开始训练\n", "trainer.train()" ] }, { "cell_type": "code", "execution_count": 27, "id": "315aae76-44b4-4513-8139-40ef22934873", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Saved model to: gpt_ft/final\n" ] } ], "source": [ "save_dir = 'gpt_ft/final'\n", "trainer.save_model(save_dir)\n", "print(\"Saved model to:\", save_dir)" ] }, { "cell_type": "code", "execution_count": 28, "id": "28d2dbbc-02ff-4120-b230-b19905a786cd", "metadata": {}, "outputs": [], "source": [ "ave_dir = 'gpt_ft/final'\n", "finetuned_model = GPT2LMHeadModel.from_pretrained(save_dir, local_files_only=True)" ] }, { "cell_type": "code", "execution_count": 29, "id": "08987c3c-063a-4e9b-9ebb-e637b0b5bccd", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "GPT2LMHeadModel(\n", " (transformer): GPT2Model(\n", " (wte): Embedding(90000, 768)\n", " (wpe): Embedding(1024, 768)\n", " (drop): Dropout(p=0.1, inplace=False)\n", " (h): ModuleList(\n", " (0-11): 12 x GPT2Block(\n", " (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n", " (attn): GPT2SdpaAttention(\n", " (c_attn): Conv1D(nf=2304, nx=768)\n", " (c_proj): Conv1D(nf=768, nx=768)\n", " (attn_dropout): Dropout(p=0.1, inplace=False)\n", " (resid_dropout): Dropout(p=0.1, inplace=False)\n", " )\n", " (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n", " (mlp): GPT2MLP(\n", " (c_fc): Conv1D(nf=3072, nx=768)\n", " (c_proj): Conv1D(nf=768, nx=3072)\n", " (act): NewGELUActivation()\n", " (dropout): Dropout(p=0.1, inplace=False)\n", " )\n", " )\n", " )\n", " (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n", " )\n", " (lm_head): Linear(in_features=768, out_features=90000, bias=False)\n", ")" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "finetuned_model" ] }, { "cell_type": "code", "execution_count": 30, "id": "d75010e8-6d6a-40ef-852e-0d705adc3da8", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "input (test): Below is an instruction that describes a task. Write a response that appropriately completes the request.\n", "\n", "### Instruction:\n", "Determine core promoter detection of following dna sequence, The result will be one of the following: Non-promoter, promoter.\n", "\n", "### Input:\n", "CCAGGATGCGCTGACGACCCGGCTGGCAGGCGGGTCCTCGTGGGCGAGGCGAGGGAGGCGGCGAGAGAGGAGCAATAGTTTCCCACCGCTCCCTCTCAGGCGCAGGGTCTAGAGAAGCGCGAGGGGATCTAGAGAAGCCGGAGGGGAGGAAGCGCGAGTCCGCGGCCCGCCCCGTTGCGTCCCACCCACCGCGTCCCCTCCCCTCCCCTCCCGCTGCGGGAAAAGCGGCCGCGGGCGGCGGCGCCCACTGTGGGGCGGGCGGAGCGCCGCGGGAGGCGGACGAGATGCGAGCGCGGCCGC\n", "\n", "### Response:\n", "\n", "--------------------------\n", "\n", "model's answer: \n", "\n", "promoterpromoterpromo\n", "--------------------------\n", "\n", "real answer: \n", "\n", "promoter\n" ] } ], "source": [ "print(\"input (test):\", input_text)\n", "\n", "print(\"--------------------------\\n\")\n", "\n", "print(\"model's answer: \\n\")\n", "print(inference(input_text, finetuned_model, tokenizer))\n", "\n", "print(\"--------------------------\\n\")\n", "print(\"real answer: \\n\")\n", "print(data[\"test\"][0][\"output\"])" ] }, { "cell_type": "code", "execution_count": 31, "id": "64365e15-510e-4abf-92f5-c78b660b37dc", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n", "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n", "Setting `pad_token_id` to `eos_token_id`:0 for open-end generation.\n" ] } ], "source": [ "test_data = data[\"test\"].select(range(100))\n", "\n", "data_list = []\n", "\n", "for entry in test_data:\n", " input_text = format_input(entry)\n", " #print(input_text)\n", " response_text = inference(input_text, finetuned_model, tokenizer)\n", " #print(response_text)\n", " data = {\n", " \"instruction\":entry[\"instruction\"],\n", " \"input\":entry[\"input\"],\n", " \"output\":entry[\"output\"],\n", " \"model_response\":response_text\n", " }\n", "\n", " data_list.append(data)" ] }, { "cell_type": "code", "execution_count": 32, "id": "a45fb780-fc3f-401c-b6e0-6f7d0c1682de", "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "# 定义输出文件路径\n", "output_file = 'gpt2-small3-1024.json'\n", "\n", "# 将 Dataset 对象导出为 JSON 文件\n", "# test_data.to_json(output_file)\n", "with open(output_file, \"w\") as file:\n", " json.dump(data_list, file, indent=4) # \"indent\" for pretty-printing\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "a83c8881-c763-4bba-8b85-584a6722a38e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "promoter |||||||||||| promoterpromoterpromo\n", "promoter |||||||||||| promoterpromoterpromo\n", "Non-promoter |||||||||||| Non-promoter\n", "Non-promoter |||||||||||| Non-promoter\n", "presicion 0.94 same 0.49\n" ] } ], "source": [ "import json\n", "\n", "\n", "output_file = 'gpt2-small3-1024.json'\n", "\n", "with open(output_file, \"r\") as file:\n", " test_data = json.load(file)\n", "\n", "all_num = len(test_data)\n", "right_sum = 0\n", "same_sum = 0\n", "for item in test_data:\n", " output = item[\"output\"]\n", " #output = \" \".join(tokenizer.tokenize(output))\n", " model_response = item[\"model_response\"]\n", "\n", " print(output,\"||||||||||||\", model_response)\n", "\n", " if model_response == output: #same it\n", " same_sum = same_sum + 1\n", " \n", " if output.find(\"Non\")==-1: # no Non\n", " if model_response.find(output)!=-1 and model_response.find(\"Non\")==-1: #find it, but no Non\n", " right_sum = right_sum + 1\n", " else:\n", " if model_response.find(output)!=-1: #find it\n", " right_sum = right_sum + 1\n", "\n", "\n", "print(\"Accuracy\", right_sum/all_num, \"same\", same_sum/all_num)" ] }, { "cell_type": "code", "execution_count": null, "id": "8bf88885-fb33-406e-9644-cd174a8a2f28", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }