{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "G9bMnJ6Iv06S" }, "outputs": [], "source": [ "# Run this first to install all the packages needed in this colab. This can take several minutes.\n", "%%capture\n", "!pip install huggingface_hub accelerate transformers datasets" ] }, { "cell_type": "markdown", "metadata": { "id": "_8JZ37wxv__Q" }, "source": [ "# Train a simple sentiment classifier on Hugging Face\n", "This colab walks you through the most simple example of fine-tuning an LLM on Hugging Face. This is part of www.huggingfacetutorial.com.\n", "\n", "We start with a 33M parameter pretrained LLM model [MiniLM](https://huggingface.co./microsoft/MiniLM-L12-H384-uncased). We fine-tune the pretrained model with the [IMDB datasets](https://huggingface.co./datasets/imdb), which are review comments on a movie review website tied to a postive or negative sentiment.\n", "\n", "The fine-tuned model is a sentiment classifier. You input any text, and it will output the percentage positive and percentage negative.\n", "\n", "Be warned: the model this colab creates is not very good at its job. But it's a start. This colab has been stripped down to the bare bones of what is needed to fine-tune a model on Hugging Face using the Transformers library.\n", "\n", "You can run this on a regular free Python 3 Google Compute Engine backend. You do not need a GPU or TPU. For those new to colabs and AI modeling, training models require lots of compute power. www.huggingfacetutorial.com tries to teach you the skills and concepts without having to pay for expensive GPU time." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "colab": { "base_uri": "https://localhost:8080/", "height": 145, "referenced_widgets": [ "7652a95c83f24dbeb891c9ea2c2814b3", "6b0bf3056a3f41079ab20a283f20a623", "9381efc4b0c94fbf9acc2ae7634ca864", "49dd5a482ef143eabdc03a2c0abf2868", "0351eec1ee804bf0accbd00b4e0e6cbc", "d14619fd8fe841d59b31e42ab7ea7e17", "ce7754c76172481597a316eede6117da", "c4bbd82754b9494bb6ba993ce94c85ef", "2352231d6ea943a8b340a777e9720e03", "fbdc62fd45e84c139d3570acf38c350f", "83646ab06a1c4cbeacd8935c658399cd", "7cdad421d94e4e67a57581b4a7a055fb", "510a6919339842b0bff5234c40fff353", "3475bc231ad24c88817c3dbab988cd44" ] }, "id": "0GLdVdXlxOdq", "outputId": "8976d97e-0abd-42f6-c76d-eda9b7b95b97" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "7652a95c83f24dbeb891c9ea2c2814b3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HTML(value='
Epoch | \n", "Training Loss | \n", "Validation Loss | \n", "
---|
"
],
"text/plain": [
"