Spaces:
Runtime error
Runtime error
brunorosilva
commited on
Commit
•
00c3763
1
Parent(s):
c79bec5
docs: add examples
Browse files- .gitignore +0 -5
- README.md +31 -11
.gitignore
CHANGED
@@ -105,8 +105,3 @@ ENV/
|
|
105 |
.vscode/
|
106 |
.idea/
|
107 |
__MACOSX/
|
108 |
-
|
109 |
-
# random imgs
|
110 |
-
*.jpeg
|
111 |
-
*.jpg
|
112 |
-
*.png
|
|
|
105 |
.vscode/
|
106 |
.idea/
|
107 |
__MACOSX/
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
@@ -1,7 +1,9 @@
|
|
1 |
-
# Image-to-Art Search
|
2 |
|
3 |
This project fine-tunes a Vision Transformer (ViT) model, pre-trained with "google/vit-base-patch32-224-in21k" weights and fine tuned with the style of [ArtButMakeItSports](https://www.instagram.com/artbutmakeitsports/), to perform image-to-art search across 81k artworks made available by [WikiArt](https://wikiart.org/).
|
4 |
|
|
|
|
|
5 |
## Table of Contents
|
6 |
|
7 |
- [Overview](#overview)
|
@@ -20,20 +22,20 @@ This project leverages the Vision Transformer (ViT) model architecture for the t
|
|
20 |
## Installation
|
21 |
|
22 |
1. Clone the repository:
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
|
28 |
2. Install poetry:
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
|
33 |
3. Install using poetry:
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
|
38 |
## How it works
|
39 |
|
@@ -41,6 +43,8 @@ This project leverages the Vision Transformer (ViT) model architecture for the t
|
|
41 |
|
42 |
1. Download images from the [ArtButMakeItSports](https://www.instagram.com/artbutmakeitsports/) Instagram account.
|
43 |
2. Organize the images into appropriate directories for training and validation.
|
|
|
|
|
44 |
|
45 |
### Training
|
46 |
|
@@ -88,6 +92,22 @@ The training script fine-tunes the ViT model on the prepared dataset. Key steps
|
|
88 |
The recommended method to get results is to use [gradio](https://www.gradio.app/) as an interface by running `make viz`. This will open a server and you can use some image you want to search or even use your webcam to get top 4 search results.
|
89 |
|
90 |
### Examples
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
91 |
|
92 |
## Contributing
|
93 |
There are three topics I'd appreciate help with:
|
|
|
1 |
+
# Image-to-Art Search 🔍
|
2 |
|
3 |
This project fine-tunes a Vision Transformer (ViT) model, pre-trained with "google/vit-base-patch32-224-in21k" weights and fine tuned with the style of [ArtButMakeItSports](https://www.instagram.com/artbutmakeitsports/), to perform image-to-art search across 81k artworks made available by [WikiArt](https://wikiart.org/).
|
4 |
|
5 |
+
![horse](examples/horse.png)
|
6 |
+
|
7 |
## Table of Contents
|
8 |
|
9 |
- [Overview](#overview)
|
|
|
22 |
## Installation
|
23 |
|
24 |
1. Clone the repository:
|
25 |
+
```sh
|
26 |
+
git clone https://github.com/brunorosilva/img2art-search.git
|
27 |
+
cd img2art-search
|
28 |
+
```
|
29 |
|
30 |
2. Install poetry:
|
31 |
+
```sh
|
32 |
+
pip install poetry
|
33 |
+
```
|
34 |
|
35 |
3. Install using poetry:
|
36 |
+
```sh
|
37 |
+
poetry install
|
38 |
+
```
|
39 |
|
40 |
## How it works
|
41 |
|
|
|
43 |
|
44 |
1. Download images from the [ArtButMakeItSports](https://www.instagram.com/artbutmakeitsports/) Instagram account.
|
45 |
2. Organize the images into appropriate directories for training and validation.
|
46 |
+
3. Get a fine tuned model
|
47 |
+
4. Create the gallery using WikiArt
|
48 |
|
49 |
### Training
|
50 |
|
|
|
92 |
The recommended method to get results is to use [gradio](https://www.gradio.app/) as an interface by running `make viz`. This will open a server and you can use some image you want to search or even use your webcam to get top 4 search results.
|
93 |
|
94 |
### Examples
|
95 |
+
Search for contextual similarity
|
96 |
+
![field](examples/field.png)
|
97 |
+
|
98 |
+
Search for shapes similarity
|
99 |
+
![basket](examples/basketball.png)
|
100 |
+
|
101 |
+
Search for expression similarity (yep, that's me)
|
102 |
+
![serious_face](examples/serious_face.png)
|
103 |
+
|
104 |
+
Search for pose similarity
|
105 |
+
![lawyer](examples/lawyer.png)
|
106 |
+
|
107 |
+
Search for an object
|
108 |
+
![horse](examples/horse.png)
|
109 |
+
|
110 |
+
|
111 |
|
112 |
## Contributing
|
113 |
There are three topics I'd appreciate help with:
|