fpaissan commited on
Commit
844ccbe
1 Parent(s): f07425a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -3
README.md CHANGED
@@ -12,14 +12,16 @@ tags:
12
  [![arXiv](https://img.shields.io/badge/arXiv-1234.56789-b31b1b.svg)](https://arxiv.org/abs/2311.14517)
13
 
14
  This repository contains the official implementation of [tinyCLAP](https://arxiv.org/abs/2311.14517).
 
15
 
16
  ![tinyCLAP overview](https://francescopaissan.it/tinyclapweb/assets/overview.png)
17
 
18
  ## Requirements
19
 
20
- To install requirements:
21
 
22
  ```setup
 
23
  pip install -r extra_requirements.txt
24
  ```
25
 
@@ -65,9 +67,23 @@ python train_clap.py --experiment_name tinyCLAP_$MODEL_NAME --zs_eval True --tut
65
 
66
  ## Pre-trained Models
67
 
68
- You can download pretrained models here:
69
 
70
- - [My awesome model](https://drive.google.com/mymodel.pth) trained on ImageNet using parameters x,y,z.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
 
72
  ## Citing tinyCLAP
73
 
 
12
  [![arXiv](https://img.shields.io/badge/arXiv-1234.56789-b31b1b.svg)](https://arxiv.org/abs/2311.14517)
13
 
14
  This repository contains the official implementation of [tinyCLAP](https://arxiv.org/abs/2311.14517).
15
+ To access the project website, using [this link](https://francescopaissan.it/tinyclapweb/).
16
 
17
  ![tinyCLAP overview](https://francescopaissan.it/tinyclapweb/assets/overview.png)
18
 
19
  ## Requirements
20
 
21
+ To clone the repo and install requirements:
22
 
23
  ```setup
24
+ git clone https://github.com/fpaissan/tinyCLAP & cd tinyCLAP
25
  pip install -r extra_requirements.txt
26
  ```
27
 
 
67
 
68
  ## Pre-trained Models
69
 
70
+ You can download pretrained models from the [tinyCLAP HF](https://huggingface.co/fpaissan/tinyCLAP).
71
 
72
+ _Note_: The checkpoints on HF contain the entire CLAP module (complete of text encoder and teacher encoder).
73
+
74
+ To run inference using the pretrained models, please use:
75
+
76
+ ```bash
77
+ python train_clap.py --pretrained_clap fpaissan/tinyCLAP/$MODEL_NAME --zs_eval True --tut17_folder $PATH_TO_TUT17
78
+ ```
79
+
80
+ This command will automatically download the checkpoint, if present in the zoo of pretrained models. Make sure to change the dataset configuration file based on the evaluation.
81
+
82
+ A list of available models with their computational cost is described in the follwing table:
83
+
84
+ | alpha | beta | t0 | N | Params [M] | ESC-50 | UrbanSound8K | TUT17 |
85
+ |:-----:|:----:|:--:|:-:|:----------:|:------:|:------------:|:-----:|
86
+ | 1.5 | 0.75 | 6 | 7 | 4.4 | | | |
87
 
88
  ## Citing tinyCLAP
89