semiotic/T5-3B-SynQL-Spider-Train-Run-00

Model Card for T5-3B/SynQL-Spider-Train-Run-00

• Developed by: Semiotic Labs
• Model type: [Text to SQL]
• License: [Apache-2.0]
• Finetuned from model: google-t5/t5-3b
• Dataset used for finetuning: semiotic/SynQL-Spider-Train

Model Context

Example metadata can be found below, context represents the prompt that is presented to the model. Database schemas follow the encoding method proposed by Shaw et al (2020).

"query": "SELECT count(*) FROM singer",
"question": "How many singers do we have?",
"context": "How many singers do we have? | concert_singer | stadium : stadium_id, location, name, capacity, highest, lowest, average | singer : singer_id, name, country, song_name, song_release_year, age, is_male | concert : concert_id, concert_name, theme, stadium_id, year | singer_in_concert : concert_id, singer_id",
"db_id": "concert_singer",

Model Results

Evaluation set: Spider/dev

Evaluation metrics: [Test-Suite-Execution, Execution Accuracy]

Model	Data	Run	Execution Accuracy	Test-Suite Execution Accuracy
T5-3B	semiotic/SynQL-Spider-Train	00	0.7021	0.5996
T5-3B	semiotic/SynQL-Spider-Train	01	0.6992	0.5464
T5-3B	semiotic/SynQL-Spider-Train	02	0.7002	0.5861