README.md · HichTala/draw at main

metadata

license: mit
language:
  - en

DRAW (which stands for Detect and Recognize A Wild range of cards) is the very first object detector trained to detect Yu-Gi-Oh! cards in all types of images, and in particular in dueling images.

Other works exist (see Related Works) but none is capable of recognizing cards during a duel.

DRAW is entirely open source and all contributions are welcome.

📄Documentation

Install

Both a docker installation and a more conventional installation are available. If you're not very familiar with all the code, docker installation is recommended. Otherwise, opt for the classic installation.

Docker installation

If you are familiar with docker, the docker image is available here.

Otherwise, I recommend you to download DockerDesktop if you are on Windows. If you are on Linux, you can refer to the documentation here.

Once it is done, you simply have to execute the following command,

docker run -p 5000:5000 --name draw hichtala/draw:latest

Your installation is now completed. You can press Ctrl+C and continue to Usage section.

Classic installation

You need python to be installed. Python installation isn't going to be detailed here, you can refer to the documentation.

We first need to install pytorch. It is recommended to use a package manager such as miniconda. Please refer to the documentation.

When everything is set up you can run the following command to install pytorch:

python -m pip install torch torchvision

If you want to use you gpus to make everything run faster, please refer the documentation

Then you just have to clone the repo and install requirements:

git clone https://github.com/HichTala/draw
cd draw
python -m pip install -r requirements.txt

Your installation is now completed.

Usage

Now to use it you need to download the models and the data, in section Models and Data.

Once you have it, follow instruction depending on you have docker or classic installation. Put all the model in the same folder, and keep the dataset as it is

Docker installation

You have to copy the data and models in the container. Execute the following command:

docker cp path/to/dataset/club_yugioh_dataset draw:/data
docker cp path/to/model/folder draw:/models

Once it is done you just have to run the command:

docker start draw

open the adress localhost:5000, and enjoy the maximum. Refer bellow for details about parameters

Classic installation

You need to modify the config.json file by putting the paths of you dataset folder in "data_path" parameter and the path to model folder in "trained_models" parameter.

Once done, just run:

flask --app app.py run

open the adress localhost:5000, and enjoy the maximum. Refer bellow for details about parameters

Both

In the first parameter, the one with gears, put the config.json file
In the second parameter, the one with a camera, put the video you want to process (leave it empty to use your webcam)
In the last one, put your deck list in the format ydk

Then you can press the button and start the process !

⚙️Models and Data

Models

In this project, the tasks were divided so that one model would locate the card and another model would classify them. Similarly, to classify the cards, I divided the task so that there is one model for each type of card, and the model to be used was determined by the color of the card.

Models can be downloaded in Hugging Face. Models starting with beit stands for classification and the one starting with yolo for localization.

For now only models for "retro" gameplay are available but the ones for classic format play will be added soon. I considered "retro" format all cards before the first syncro set, so all the cards edited until Light of Destruction set (LODT - 05/13/2008) set and all speed duel cards.

Data

To create a dataset, the YGOPRODeck api was used. Two datasets were thus created, one for "retro" play and the other for classic format play. Just as there is a model for each type of card, there is a dataset for each type of card.

Dataset can be downloaded in Hugging Face.

For now only "retro" dataset is available, but the one for classic format play will be added soon.

💡Inspiration

This project is inspired by content creator SuperZouloux's idea of a hologram bringing Yu-Gi-Oh! cards to life. His project uses chips inserted under the sleeves of each card, which are read by the play mat, enabling the cards to be recognized.

Inserting the chips into the sleeves is not only laborious, but also poses another problem: face-down cards are read in the same way as face-up ones. So an automatic detector is a really suitable solution.

Although this project was discouraged by KONAMI ^®, the game's publisher (which is quite understandable), we can nevertheless imagine such a system being used to display the cards played during a live duel, to allow spectators to read the cards.

🔗Related Works

Although to my knowledge draw is the first detector capable of locating and detecting Yu-Gi-Oh! cards in a dueling environment, other works exist and were a source of inspiration for this project. It's worth mentioning them here.

Yu-Gi-Oh! NEURON is an official application developed by KONAMI ^®. It's packed with features, including cards recognition. The application is capable of recognizing a total of 20 cards at a time, which is very decent. The drawback is that the cards must be of good quality to be recognized, which is not necessarily the case in a duel context. What's more, it can't be integrated, so the only way to use it is to use the application.

yugioh one shot learning made by vanstorm9 is a Yu-Gi-Oh! cards classification program that allow you to recognize cards. It uses siamese network to train its classification model. It gives very impressive results on images with a good quality but not that good on low quality images, and it can't localize cards.

Yolov8 is the last version of the very famous yolo family of object detector models. I think it doesn't need to be presented today, it represents state-of-the-art real time object detection model.

BEiT is a pre-trained model for image classification. It uses image transofrmers which are based on attention mechanism. It suits our problem because authors also propose a pre-trained model in Imagenet-22K. It is a dataset with 22k classes (more than most classifiers) which is interesting for our case since there is mode than 11k cards in Yu-Gi-Oh!.

🔍Method Overview

A medium blog will soon be written and published, explaining the main process from data collection to final prediction. If you have any questions, don't hesitate to open an issue.

💬Contact

You can reach me on Twitter @tiazden or by email at [email protected].