new models & events & docs

README.md

@@ -60,16 +60,17 @@ Full model code:
 
 ```python
 data_augmentation = tf.keras.Sequential([
-  layers.Resizing(256, 256, interpolation="bilinear", crop_to_aspect_ratio=True, input_shape=(None, None, 3)),
-  layers.Rescaling(scale=1./127.5, offset=-1),
-  RandomBrightness(factor=0.2, value_range=(-1.0, 1.0)), # Not avaliable in tensorflow 2.8
+  layers.RandomBrightness(factor=0.2, input_shape=(None, None, 3)), # Not avaliable in tensorflow 2.8
   layers.RandomContrast(factor=0.2),
   layers.RandomFlip("horizontal_and_vertical"),
   layers.RandomRotation(0.3, fill_mode="constant"),
   layers.RandomZoom(.2, .2, fill_mode="constant"),
   layers.RandomTranslation(0.2, .2,fill_mode="constant"),
-
+  layers.Resizing(256, 256, interpolation="bilinear", crop_to_aspect_ratio=True),
+  layers.Rescaling(scale=1./127.5, offset=-1), # For [-1, 1] scaling
 ])
+
+# My ResNet50V2
 model = models.Sequential()
 model.add(data_augmentation)
 model.add(
@@ -80,7 +81,7 @@ model.add(
   )
 )
 model.add(layers.Flatten())
-model.add(layers.Dense(2, activation='softmax'))
+model.add(layers.Dense(len(config.SUB_FOLDERS), activation='softmax'))
 ```
 
 ```
@@ -110,28 +111,41 @@ Class imbalance was handled by using the class\_-weight attribute during trainin
 The following hyperparameters were used during training:
 
 - optimizer: Adam
-- Learning Rate Schedule: `ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=MIN_LR` with START_LR = 1e-5 and MIN_LR = 1e-7
+- Learning Rate Schedule: `ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=1e-7` with START_LR = 1e-5 and MIN_LR = 1e-7
 - Loss Function: SparseCategoricalCrossentropy
 - Metric: Accuracy
 
+For more details please see the training notebook associated.
+
 ## Training Curve
 
+Full training results are avaliable on `Weights and Biases` here: [https://api.wandb.ai/links/lambda-science/ka0iw3b6](https://api.wandb.ai/links/lambda-science/ka0iw3b6)
 Plot of the accuracy vs epoch and loss vs epoch for training and validation set.
 ![Training Curve](./training_curve.png)
 
 ## Test Results
 
-Results for accuracy metrics on the test split of the [corentinm7/MyoQuant-SDH-Data](https://huggingface.co/datasets/corentinm7/MyoQuant-SDH-Data) dataset.
+Results for accuracy and balanced accuracy metrics on the test split of the [corentinm7/MyoQuant-SDH-Data](https://huggingface.co/datasets/corentinm7/MyoQuant-SDH-Data) dataset.
 
 ```
-105/105 - 15s - loss: 0.1618 - accuracy: 0.9285 - 15s/epoch - 140ms/step
+105/105 - 11s - loss: 0.1574 - accuracy: 0.9321 - 11s/epoch - 102ms/step
 Test data results:
-0.928528904914856
+0.9321024417877197
+105/105 [==============================] - 6s 44ms/step
+Test data results:
+0.9166411912436779
 ```
 
 # How to Import the Model
 
-To import this model as it was trained in Tensorflow 2.8 on Google Colab, RandomBrightness layer had to be added by hand (it was only introduced in Tensorflow 2.10.). So you will need to download the `random_brightness.py` fille in addition to the model.  
+With Tensorflow 2.10 and over:
+
+```python
+model_sdh = keras.models.load_model("model.h5")
+```
+
+With Tensorflow <2.10:
+To import this model RandomBrightness layer had to be added by hand (it was only introduced in Tensorflow 2.10.). So you will need to download the `random_brightness.py` fille in addition to the model.  
 Then the model can easily be imported in Tensorflow/Keras using:
 
 ```python
@@ -144,6 +158,7 @@ model_sdh = keras.models.load_model(
 ## The Team Behind this Dataset
 
 **The creator, uploader and main maintainer of this model, associated dataset and MyoQuant is:**
+
 - **[Corentin Meyer, 3rd year PhD Student in the CSTB Team, ICube — CNRS — Unistra](https://cmeyer.fr) Email: <[email protected]> Github: [@lambda-science](https://github.com/lambda-science)**
 
 Special thanks to the experts that created the data for the dataset and all the time they spend counting cells :
@@ -161,4 +176,4 @@ Last but not least thanks to Bertrand Vernay being at the origin of this project
   <img src="https://i.imgur.com/m5OGthE.png" alt="Partner Banner" style="border-radius: 25px;" />
 </p>
 
-MyoQuant-SDH-Model is born within the collaboration between the [CSTB Team @ ICube](https://cstb.icube.unistra.fr/en/index.php/Home) led by Julie D. Thompson, the [Morphological Unit of the Institute of Myology of Paris](https://www.institut-myologie.org/en/recherche-2/neuromuscular-investigation-center/morphological-unit/) led by Teresinha Evangelista, the [imagery platform MyoImage of Center of Research in Myology](https://recherche-myologie.fr/technologies/myoimage/) led by Bruno Cadot, [the photonic microscopy platform of the IGMBC](https://www.igbmc.fr/en/plateformes-technologiques/photonic-microscopy) led by Bertrand Vernay and the [Pathophysiology of neuromuscular diseases team @ IGBMC](https://www.igbmc.fr/en/igbmc/a-propos-de-ligbmc/directory/jocelyn-laporte) led by Jocelyn Laporte
+MyoQuant-SDH-Model is born within the collaboration between the [CSTB Team @ ICube](https://cstb.icube.unistra.fr/en/index.php/Home) led by Julie D. Thompson, the [Morphological Unit of the Institute of Myology of Paris](https://www.institut-myologie.org/en/recherche-2/neuromuscular-investigation-center/morphological-unit/) led by Teresinha Evangelista, the [imagery platform MyoImage of Center of Research in Myology](https://recherche-myologie.fr/technologies/myoimage/) led by Bruno Cadot, [the photonic microscopy platform of the IGMBC](https://www.igbmc.fr/en/plateformes-technologiques/photonic-microscopy) led by Bertrand Vernay and the [Pathophysiology of neuromuscular diseases team @ IGBMC](https://www.igbmc.fr/en/igbmc/a-propos-de-ligbmc/directory/jocelyn-laporte) led by Jocelyn Laporte

runs/sdh16k_normal_resize_20220830-083856/validation/events.out.tfevents.1661848941.561a638614d6.77.1.v2 → history.pickle

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f2a41bb11af557b261892fa9b42a6a8ff9ddd0917418cbdccac031a2009f7c6f
-size 11236
+oid sha256:c222fe72b84d6acd03fcf393efb8a41201bdbace8df399c2050409fc53a5c595
+size 1241

model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:221f570111afc7619c67e7d59e48b16537c1f089bda3ea5b982d0ff31eedc5c6
-size 283199312
+oid sha256:d9b858710ae2756424f7c4df0edcee9549e6f57b81e89c5227fbbb1201081514
+size 283136344

myoquant-sdh-train.ipynb

@@ -0,0 +1,509 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "os.environ[\"TF_CPP_MIN_LOG_LEVEL\"] = \"3\"\n",
+    "import tensorflow as tf\n",
+    "tf.get_logger().setLevel('ERROR')\n",
+    "\n",
+    "if tf.test.gpu_device_name()=='':\n",
+    "  print('You do not have GPU access.') \n",
+    "  print('Did you change your runtime ?') \n",
+    "  print('If the runtime setting is correct then Google did not allocate a GPU for your session')\n",
+    "  print('Expect slow performance. To access GPU try reconnecting later')\n",
+    "\n",
+    "else:\n",
+    "  print('You have GPU access')\n",
+    "  !nvidia-smi\n",
+    "\n",
+    "# from tensorflow.python.client import device_lib \n",
+    "# device_lib.list_local_devices()\n",
+    "\n",
+    "# print the tensorflow version\n",
+    "print('Tensorflow version is ' + str(tf.__version__))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import wandb\n",
+    "from wandb.keras import WandbMetricsLogger\n",
+    "\n",
+    "run = wandb.init(project='myoquant-sdh',\n",
+    "                 config={\n",
+    "                    \"BATCH_SIZE\": 32,\n",
+    "                    \"CLASS_WEIGHTS\": True,\n",
+    "                    \"EARLY_STOPPING_PATIENCE\": 10,\n",
+    "                    \"EPOCH\": 1000,\n",
+    "                    \"EPOCH_OPTI_LR\": 100,\n",
+    "                    \"LOSS\": \"SparseCategoricalCrossentropy\",\n",
+    "                    \"LR_PATIENCE\":5,\n",
+    "                    \"LR_PLATEAU_RATIO\":0.2,\n",
+    "                    \"MAX_LR\":0.00001,\n",
+    "                    \"METRIC\":\"accuracy\",\n",
+    "                    \"MIN_LR\":1e-7,\n",
+    "                    \"MODEL_NAME\":\"SDH16K_wandb\",\n",
+    "                    \"OPTIMIZER\":\"adam\",\n",
+    "                    \"OPTI_START_LR\":1e-7,\n",
+    "                    \"RELOAD_MODEL\":False,\n",
+    "                    \"SUB_FOLDERS\":{0:\"control\", 1:\"sick\"},\n",
+    "                    \"UPLOAD_LOGS\":True,\n",
+    "                    }\n",
+    "                  )\n",
+    "\n",
+    "config = wandb.config\n",
+    "BASE_FOLDER=\"/home/meyer/code-project/AI-dev-playground/data/\"\n",
+    "LOG_DIR=\"/home/meyer/code-project/AI-dev-playground/logs\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf\n",
+    "from tensorflow.image import resize_with_crop_or_pad\n",
+    "from tensorflow.keras import layers, models, callbacks\n",
+    "from tensorflow.keras.preprocessing import image\n",
+    "from tensorflow.keras.utils import load_img, img_to_array\n",
+    "# import tensorflow_addons as tfa\n",
+    "\n",
+    "import tensorboard as tb\n",
+    "from tensorflow.keras.applications.resnet_v2 import ResNet50V2, preprocess_input\n",
+    "from sklearn.metrics import balanced_accuracy_score\n",
+    "\n",
+    "import matplotlib.cm as cm\n",
+    "from IPython.display import Image, display\n",
+    "\n",
+    "from pathlib import Path\n",
+    "import pickle\n",
+    "import numpy as np\n",
+    "import datetime, os\n",
+    "import glob\n",
+    "from math import exp, log, pow\n",
+    "# from PIL import Image\n",
+    "from matplotlib import pyplot as plt\n",
+    "from scipy import stats\n",
+    "import pandas as pd\n",
+    "\n",
+    "tf.random.set_seed(42)\n",
+    "np.random.seed(42)\n",
+    "\n",
+    "MODEL_PATH = os.path.join(BASE_FOLDER, \"results\", config.MODEL_NAME)\n",
+    "Path(MODEL_PATH).mkdir(parents=True, exist_ok=True)\n",
+    "\n",
+    "logdir = os.path.join(LOG_DIR, datetime.datetime.now().strftime(config.MODEL_NAME+\"_%Y%m%d-%H%M%S\"))\n",
+    "tensorboard_cb = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
+    "\n",
+    "def generate_dataset(folder, sub_folders=[\"control\", \"inter\", \"sick\"]):\n",
+    "    n_elem = 0\n",
+    "    for sub_folder in sub_folders:\n",
+    "      n_elem += len(glob.glob(os.path.join(folder, sub_folder, \"*.tif\")))\n",
+    "    \n",
+    "    images_array = np.empty(shape=(n_elem, 256, 256, 3), dtype=np.uint8)\n",
+    "    labels_array = np.empty(shape=n_elem, dtype=np.uint8)\n",
+    "    counter = 0\n",
+    "    for index, sub_folder in enumerate(sub_folders):\n",
+    "      path_files = os.path.join(folder, sub_folder, \"*.tif\")\n",
+    "      for img in glob.glob(path_files):\n",
+    "          im = img_to_array(image.load_img(img))\n",
+    "          # im_resized = image.smart_resize(im, (256, 256))\n",
+    "          im_resized = tf.image.resize(im, (256,256))\n",
+    "          images_array[counter] = im_resized\n",
+    "          labels_array[counter] = index\n",
+    "          counter += 1\n",
+    "    return images_array, labels_array\n",
+    "\n",
+    "def scale_fn(x):\n",
+    "  # return 1.0 # Triangular Scaling Method\n",
+    "  return 1 / (2.0 ** (x - 1)) # Triangular2 Scaling method\n",
+    "\n",
+    "\n",
+    "def get_inter_unsure_img(BASE_FOLDER):\n",
+    "    n_unsure = len(glob.glob(BASE_FOLDER+\"Unsure/*.tif\"))\n",
+    "    n_intermediate = len(glob.glob(BASE_FOLDER+\"Intermediate/*.tif\"))\n",
+    "    \n",
+    "    unsure_images = np.empty(shape=(n_unsure, 256, 256, 3), dtype=np.uint8)\n",
+    "    intermediate_images = np.empty(shape=(n_intermediate, 256, 256, 3), dtype=np.uint8)\n",
+    "\n",
+    "    counter = 0\n",
+    "    for img in glob.glob(BASE_FOLDER+\"Unsure/*.tif\"):\n",
+    "        im = img_to_array(image.load_img(img))\n",
+    "        # im_resized = image.smart_resize(im, (256, 256))\n",
+    "        im_resized = tf.image.resize(im, (256,256))\n",
+    "        unsure_images[counter] = im_resized\n",
+    "        counter += 1\n",
+    "    \n",
+    "    counter = 0\n",
+    "    for img in glob.glob(BASE_FOLDER+\"Intermediate/*.tif\"):\n",
+    "        im = img_to_array(image.load_img(img))\n",
+    "        # im_resized = image.smart_resize(im, (256, 256))\n",
+    "        im_resized = tf.image.resize(im, (256,256))\n",
+    "        intermediate_images[counter] = im_resized\n",
+    "        counter += 1\n",
+    "\n",
+    "\n",
+    "    return unsure_images, intermediate_images\n",
+    "\n",
+    "# GRAD-CAM\n",
+    "def get_img_array(img_path, size):\n",
+    "    # `img` is a PIL image of size 299x299\n",
+    "    img = tf.keras.preprocessing.image.load_img(img_path, target_size=size)\n",
+    "    # `array` is a float32 Numpy array of shape (299, 299, 3)\n",
+    "    array = tf.keras.preprocessing.image.img_to_array(img)\n",
+    "    # We add a dimension to transform our array into a \"batch\"\n",
+    "    # of size (1, 299, 299, 3)\n",
+    "    array = np.expand_dims(array, axis=0)\n",
+    "    return array\n",
+    "\n",
+    "\n",
+    "def make_gradcam_heatmap(img_array, model, last_conv_layer_name, pred_index=None):\n",
+    "    # First, we create a model that maps the input image to the activations\n",
+    "    # of the last conv layer as well as the output predictions\n",
+    "    grad_model = tf.keras.models.Model(\n",
+    "        [model.inputs], [model.get_layer(last_conv_layer_name).output, model.output]\n",
+    "    )\n",
+    "\n",
+    "    # Then, we compute the gradient of the top predicted class for our input image\n",
+    "    # with respect to the activations of the last conv layer\n",
+    "    with tf.GradientTape() as tape:\n",
+    "        last_conv_layer_output, preds = grad_model(img_array)\n",
+    "        if pred_index is None:\n",
+    "            pred_index = tf.argmax(preds[0])\n",
+    "        class_channel = preds[:, pred_index]\n",
+    "\n",
+    "    # This is the gradient of the output neuron (top predicted or chosen)\n",
+    "    # with regard to the output feature map of the last conv layer\n",
+    "    grads = tape.gradient(class_channel, last_conv_layer_output)\n",
+    "\n",
+    "    # This is a vector where each entry is the mean intensity of the gradient\n",
+    "    # over a specific feature map channel\n",
+    "    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))\n",
+    "\n",
+    "    # We multiply each channel in the feature map array\n",
+    "    # by \"how important this channel is\" with regard to the top predicted class\n",
+    "    # then sum all the channels to obtain the heatmap class activation\n",
+    "    last_conv_layer_output = last_conv_layer_output[0]\n",
+    "    heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]\n",
+    "    heatmap = tf.squeeze(heatmap)\n",
+    "\n",
+    "    # For visualization purpose, we will also normalize the heatmap between 0 & 1\n",
+    "    heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)\n",
+    "    return heatmap.numpy()\n",
+    "\n",
+    "def save_and_display_gradcam(img, heatmap, cam_path=\"cam.jpg\", alpha=0.5):\n",
+    "    # Rescale heatmap to a range 0-255\n",
+    "    heatmap = np.uint8(255 * heatmap)\n",
+    "\n",
+    "    # Use jet colormap to colorize heatmap\n",
+    "    jet = cm.get_cmap(\"jet\")\n",
+    "\n",
+    "    # Use RGB values of the colormap\n",
+    "    jet_colors = jet(np.arange(256))[:, :3]\n",
+    "    jet_heatmap = jet_colors[heatmap]\n",
+    "\n",
+    "    # Create an image with RGB colorized heatmap\n",
+    "    jet_heatmap = tf.keras.preprocessing.image.array_to_img(jet_heatmap)\n",
+    "    jet_heatmap = jet_heatmap.resize((img.shape[1], img.shape[0]))\n",
+    "    jet_heatmap = tf.keras.preprocessing.image.img_to_array(jet_heatmap)\n",
+    "\n",
+    "    # Superimpose the heatmap on original image\n",
+    "    superimposed_img = jet_heatmap * alpha + img*255\n",
+    "    superimposed_img = tf.keras.preprocessing.image.array_to_img(superimposed_img)\n",
+    "    return superimposed_img"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "train_images, train_labels = generate_dataset(os.path.join(BASE_FOLDER, \"train\"), sub_folders=list(config.SUB_FOLDERS.values()))\n",
+    "val_images, val_labels = generate_dataset(os.path.join(BASE_FOLDER, \"validation\"), sub_folders=list(config.SUB_FOLDERS.values()))\n",
+    "test_images, test_labels = generate_dataset(os.path.join(BASE_FOLDER, \"test\"), sub_folders=list(config.SUB_FOLDERS.values()))\n",
+    "\n",
+    "train_dataset = tf.data.Dataset.from_tensor_slices((train_images, train_labels)).shuffle(10000).repeat(1)\n",
+    "val_dataset = tf.data.Dataset.from_tensor_slices((val_images, val_labels)).shuffle(10000).repeat(1)\n",
+    "test_dataset = tf.data.Dataset.from_tensor_slices((test_images, test_labels)).shuffle(10000).repeat(1) \n",
+    "\n",
+    "data_augmentation = tf.keras.Sequential([\n",
+    "  layers.RandomBrightness(factor=0.2), # Not avaliable in tensorflow 2.8\n",
+    "  layers.RandomContrast(factor=0.2),\n",
+    "  layers.RandomFlip(\"horizontal_and_vertical\"),\n",
+    "  layers.RandomRotation(0.3, fill_mode=\"constant\"),\n",
+    "  layers.RandomZoom(.2, .2, fill_mode=\"constant\"),\n",
+    "  layers.RandomTranslation(0.2, .2,fill_mode=\"constant\"),\n",
+    "])\n",
+    "\n",
+    "train_dataset = train_dataset.batch(config.BATCH_SIZE).prefetch(1)\n",
+    "val_dataset = val_dataset.batch(config.BATCH_SIZE).prefetch(1)\n",
+    "test_dataset = test_dataset.batch(config.BATCH_SIZE).prefetch(1)\n",
+    "\n",
+    "# Scaling by total/2 helps keep the loss to a similar magnitude.\n",
+    "# The sum of the weights of all examples stays the same.\n",
+    "if config.CLASS_WEIGHTS:\n",
+    "  class_weights_numpy = np.unique(train_labels, return_counts=True)\n",
+    "  n_train = len(train_labels)\n",
+    "  class_weights = dict()\n",
+    "  for index, folder in enumerate(config.SUB_FOLDERS):\n",
+    "    class_weights[class_weights_numpy[0][index]] = (1/class_weights_numpy[1][index])*(n_train/2.0)\n",
+    "else:\n",
+    "  class_weights = None\n",
+    "  \n",
+    "  print(class_weights)\n",
+    "\n",
+    "plt.figure(figsize=(10,10))\n",
+    "counter = 0\n",
+    "for i in np.random.choice(range(len(train_images)),25):\n",
+    "    plt.subplot(5,5,counter+1)\n",
+    "    plt.xticks([])\n",
+    "    plt.yticks([])\n",
+    "    plt.grid(False)\n",
+    "    plt.imshow(train_images[i])\n",
+    "    plt.xlabel(list(config.SUB_FOLDERS.values())[train_labels[i]])\n",
+    "    counter +=1\n",
+    "plt.show()\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data_augmentation = tf.keras.Sequential([\n",
+    "  layers.RandomBrightness(factor=0.2, input_shape=(None, None, 3)), # Not avaliable in tensorflow 2.8\n",
+    "  layers.RandomContrast(factor=0.2),\n",
+    "  layers.RandomFlip(\"horizontal_and_vertical\"),\n",
+    "  layers.RandomRotation(0.3, fill_mode=\"constant\"),\n",
+    "  layers.RandomZoom(.2, .2, fill_mode=\"constant\"),\n",
+    "  layers.RandomTranslation(0.2, .2,fill_mode=\"constant\"),\n",
+    "  layers.Resizing(256, 256, interpolation=\"bilinear\", crop_to_aspect_ratio=True), \n",
+    "  layers.Rescaling(scale=1./127.5, offset=-1), # For [-1, 1] scaling\n",
+    "])\n",
+    "\n",
+    "# My ResNet50V2\n",
+    "model = models.Sequential()\n",
+    "model.add(data_augmentation)\n",
+    "model.add(\n",
+    "    ResNet50V2(\n",
+    "    include_top=False,\n",
+    "    input_shape=(256,256,3),\n",
+    "    pooling=\"avg\",\n",
+    "  )\n",
+    ")\n",
+    "model.add(layers.Flatten())\n",
+    "model.add(layers.Dense(len(config.SUB_FOLDERS), activation='softmax'))\n",
+    "\n",
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Find min max LR\n",
+    "\"\"\"\n",
+    "def scheduler(epoch, lr):\n",
+    "    return lr*exp(log(pow(10,8))/EPOCH_OPTI_LR)\n",
+    "\n",
+    "model.compile(optimizer=tf.keras.optimizers.Nadam(learning_rate=OPTI_START_LR),\n",
+    "              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
+    "              metrics=['accuracy'])\n",
+    "\n",
+    "lr_cb = tf.keras.callbacks.LearningRateScheduler(scheduler)\n",
+    "history = model.fit(train_images, train_labels, epochs=EPOCH_OPTI_LR, batch_size=BATCH_SIZE,\n",
+    "                    validation_data=(val_images, val_labels), shuffle=True, class_weight=class_weights, \n",
+    "                    callbacks=[lr_cb, tensorboard_cb])\n",
+    "\n",
+    "loss = history.history['loss']\n",
+    "val_loss = history.history['val_loss']\n",
+    "\n",
+    "learning_rate_range = [OPTI_START_LR]\n",
+    "for epoch in range(EPOCH_OPTI_LR-1):\n",
+    "  learning_rate_range.append(learning_rate_range[epoch] * exp(log(pow(10,8))/EPOCH_OPTI_LR))\n",
+    "\n",
+    "plt.figure(figsize=(16, 8))\n",
+    "\n",
+    "plt.subplot(1, 1, 1)\n",
+    "plt.plot(learning_rate_range, loss, label='Training Loss')\n",
+    "plt.plot(learning_rate_range, val_loss, label='Validation Loss')\n",
+    "plt.legend(loc='upper right')\n",
+    "plt.title('Training and Validation Loss')\n",
+    "plt.xscale('log')\n",
+    "plt.savefig(os.path.join(MODEL_PATH, \"curve_findLR.png\"), dpi=300)\n",
+    "plt.show()\n",
+    "\"\"\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "steps_per_epoch = len(train_images) // config.BATCH_SIZE # Batch size is 32\n",
+    "\n",
+    "# Triangular 1Cycle Scheduler and Cosine Scheduler\n",
+    "# clr = tfa.optimizers.CyclicalLearningRate(initial_learning_rate=MIN_LR,\n",
+    "#     maximal_learning_rate=MAX_LR,\n",
+    "#     scale_fn=scale_fn,\n",
+    "#     step_size= 8 * steps_per_epoch\n",
+    "# )\n",
+    "# cosine_decay = tf.keras.optimizers.schedules.CosineDecayRestarts(\n",
+    "#    TRAIN_LR, 10 * steps_per_epoch, t_mul=1.0, m_mul=1.0, alpha=0.005)\n",
+    "\n",
+    "if config.RELOAD_MODEL:\n",
+    "  print(config.MODEL_NAME, \" reloaded as starting point!\")\n",
+    "  model = models.load_model(os.path.join(MODEL_PATH, \"model.h5\"))\n",
+    "\n",
+    "\n",
+    "reduce_lr = callbacks.ReduceLROnPlateau(monitor='val_loss', factor=config.LR_PLATEAU_RATIO,\n",
+    "                              patience=config.LR_PATIENCE, min_lr=config.MIN_LR)\n",
+    "\n",
+    "checkpoint_cb = callbacks.ModelCheckpoint(os.path.join(MODEL_PATH, \"model.h5\"), save_best_only=True)\n",
+    "early_stopping_cb = callbacks.EarlyStopping(patience=config.EARLY_STOPPING_PATIENCE, restore_best_weights=True)\n",
+    "wandb_metrics = WandbMetricsLogger(log_freq=\"epoch\")\n",
+    "\n",
+    "model.compile(\n",
+    "    optimizer=tf.keras.optimizers.Adam(learning_rate=config.MAX_LR),\n",
+    "    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),\n",
+    "    metrics=[config.METRIC]\n",
+    "    )\n",
+    "\n",
+    "history = model.fit(train_dataset, epochs=config.EPOCH, batch_size=config.BATCH_SIZE,\n",
+    "                    validation_data=val_dataset, shuffle=True, class_weight=class_weights, \n",
+    "                    callbacks=[reduce_lr, checkpoint_cb, early_stopping_cb, tensorboard_cb, wandb_metrics])\n",
+    "\n",
+    "art = wandb.Artifact(\"myoquant-sdh-classifier\", type=\"model\")\n",
+    "art.add_file(os.path.join(MODEL_PATH, \"model.h5\"))\n",
+    "wandb.log_artifact(art)\n",
+    "wandb.finish()\n",
+    "\n",
+    "model = models.load_model(os.path.join(MODEL_PATH, \"model.h5\"))\n",
+    "with open(os.path.join(MODEL_PATH, \"history.pickle\"), 'wb') as file_pi:\n",
+    "    pickle.dump(history.history, file_pi)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Acc and Loss Plot\n",
+    "acc = history.history['accuracy']\n",
+    "val_acc = history.history['val_accuracy']\n",
+    "\n",
+    "loss = history.history['loss']\n",
+    "val_loss = history.history['val_loss']\n",
+    "\n",
+    "epochs_range = range(len(acc))\n",
+    "\n",
+    "plt.figure(figsize=(16, 8))\n",
+    "plt.subplot(1, 2, 1)\n",
+    "plt.plot(epochs_range, acc, label='Training Accuracy')\n",
+    "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
+    "plt.axvline(x=len(acc)-config.EARLY_STOPPING_PATIENCE-1, color=\"red\")\n",
+    "plt.legend(loc='lower right')\n",
+    "plt.title('Training and Validation Accuracy')\n",
+    "\n",
+    "plt.subplot(1, 2, 2)\n",
+    "plt.plot(epochs_range, loss, label='Training Loss')\n",
+    "plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
+    "plt.axvline(x=len(acc)-config.EARLY_STOPPING_PATIENCE-1, color=\"red\")\n",
+    "plt.legend(loc='upper right')\n",
+    "plt.title('Training and Validation Loss')\n",
+    "plt.savefig(os.path.join(MODEL_PATH, \"training_curve.png\"), dpi=300)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Test Evaluation\n",
+    "model = models.load_model(os.path.join(MODEL_PATH, \"model.h5\"))\n",
+    "\n",
+    "test_loss, test_acc = model.evaluate(test_dataset, verbose=2)\n",
+    "print(\"Test data results: \")\n",
+    "print(test_acc)\n",
+    "\n",
+    "test_proba = model.predict(test_images)\n",
+    "test_classes = test_proba.argmax(axis=-1)\n",
+    "print(\"Test data results: \")\n",
+    "print(balanced_accuracy_score(test_labels, test_classes))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Generate class activation heatmap\n",
+    "model = models.load_model(os.path.join(MODEL_PATH, \"model.h5\"))\n",
+    "counter = 0\n",
+    "plt.figure(figsize=(10,10))\n",
+    "\n",
+    "for i in np.random.choice(range(len(test_images)),25):\n",
+    "    img_array = np.empty((1, 256, 256, 3))\n",
+    "    img_array[0]=test_images[i]/255.\n",
+    "    predicted_class = model.predict(img_array*255).argmax()\n",
+    "    predicted_proba = round(np.amax(model.predict(img_array*255)), 2)\n",
+    "    heatmap = make_gradcam_heatmap(img_array, model.get_layer(\"resnet50v2\"), \"conv5_block3_3_conv\") \n",
+    "    plt.subplot(5,5,counter+1)\n",
+    "    plt.xticks([])\n",
+    "    plt.yticks([])\n",
+    "    plt.grid(False)\n",
+    "    grad_cam_img = save_and_display_gradcam(img_array[0], heatmap)\n",
+    "    plt.imshow(grad_cam_img)\n",
+    "    xlabel = config.SUB_FOLDERS[test_labels[i]]+\" (\" + str(predicted_class) + \" \" + str(predicted_proba) + \")\"\n",
+    "    plt.xlabel(xlabel)\n",
+    "    counter +=1\n",
+    "plt.show()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.10"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "7dcfd37d9fc7b622fbfef8254b45067d70c57a3c50902cea4f6ef7a4affc9af0"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

runs/{sdh16k_normal_resize_20220830-083856/train/events.out.tfevents.1661848752.561a638614d6.77.0.v2 → SDH16K_wandb_20230406-214521/train/events.out.tfevents.1680810371.guepe.1458055.0.v2}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b74dc7d937740821f56203f12e53a9f279e14e65e58aae07b6f0b2cd1c37c2b9
-size 8894151
+oid sha256:12877dd526c458d4e602f3cb8cf202a32f98d184a1c694bb94fe100207b9add9
+size 6359090

runs/SDH16K_wandb_20230406-214521/validation/events.out.tfevents.1680810475.guepe.1458055.1.v2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69374377fd4d689f90e78fced0042c8c454bc3b0a978ff5a76e6ed091a8a236c
+size 6794