license: apache-2.0
base_model: flan-t5-small
tags:
- generated_from_trainer
model-index:
- name: shadow
results: []
language:
- en
library_name: transformers
pipeline_tag: question-answering
SHADOW 
SHADOW (Symbolic Higher-order Associative Deductive reasoning On Wikidata) is a model that is a fine-tuned version of flan-t5-small on the LM-KBC 2024 dataset. It achieves the following results on the validation set:
- Loss: 0.0003
Model description
The model is trained to learn specific Wikidata patterns (e.g. type of relation, type of subject entity, etc.) to identify the correct template id that matches the SPARQL query that finds the appropriate object entities for the triple (subject entity, relation, object entity). The SPARQL queries are dynamically pre-defined for each template id and the model is not exposed to the queries themselves. The training of the model follows the associative reasoning pattern based on higher-order reasoning that deductively tries to associate a symbol (here, a template id) to data (here, the (subject entity, relation) tuple).
Intended uses & limitations
This model is intended to be used with a (subject entity, relation) Wikidata pair to determine the best template id for the SPARQL query that fetches the matching object entities.
Training and evaluation data
The LM-KBC dataset is used for training and evaluation.
Templates
The templates are frames that define dynamic SPARQL queries to find the object entities to complete the triple (subject entity, relation, object entities). The model is currently trained on the following relations:
- countryLandBordersCountry: Null values possible (e.g., Iceland)
- personHasCityOfDeath: Null values possible
- seriesHasNumberOfEpisodes: Object is numeric
- awardWonBy: Many objects per subject (e.g., 224 Physics Nobel prize winners)
- companyTradesAtStockExchange: Null values possible
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-04
- train_batch_size: 4
- eval_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.6679 | 1.0 | 1000 | 0.0005 |
| 0.3425 | 2.0 | 2000 | 0.0002 |
| 0.2303 | 3.0 | 3000 | 0.0001 |
| 0.1735 | 4.0 | 4000 | 0.000 |
| 0.1394 | 5.0 | 5000 | 0.0001 |
| 0.1167 | 6.0 | 6000 | 0.00009 |
| 0.1006 | 7.0 | 7000 | 0.00008 |
| 0.0882 | 8.0 | 8000 | 0.00007 |
| 0.0785 | 9.0 | 9000 | 0.00006 |
| 0.0707 | 10.0 | 10000 | 0.00006 |
| 0.0643 | 11.0 | 11000 | 0.00005 |
| 0.0590 | 12.0 | 12000 | 0.00005 |
| 0.0545 | 13.0 | 13000 | 0.00004 |
| 0.0506 | 14.0 | 14000 | 0.00004 |
| 0.0473 | 15.0 | 15000 | 0.00004 |
| 0.0443 | 16.0 | 16000 | 0.00003 |
| 0.0417 | 17.0 | 17000 | 0.00003 |
| 0.0394 | 18.0 | 18000 | 0.00003 |
| 0.0374 | 19.0 | 19000 | 0.00003 |
| 0.0355 | 20.0 | 20000 | 0.00003 |