Performance Hugging Face Model Card vs Model in the Paper
First of all, thank you for making KRISSBERT available as open source. A very good approach to biomedical entity linking.
I have a more general question about the performance of the model. As stated in the readme, the top-1 accuracy for the full MM dataset is 58.3%. In comparison, the paper states that the accuracy should be 70.7% for the lazy supervised apporach (https://arxiv.org/pdf/2112.07887.pdf Table 2).
Can you please tell us why there is this (large) discrepancy between these two performances? Are there any important "components" missing from this model compared to the model you used to analyze the statistics in the publication?
Many thanks in advance!
In addition, I performed the evaluation on the BC5CDR dataset and also found a big difference between the performances and the performances given in the publication.
I just found out that the problem is that I am missing the self-supervised prototypes from the PubMed dataset. Since the self-supervised mentions cannot directly be shared due to legal restrictions, would it be possible to share the embeddings as a pickle file?
Can we have access to embedding files? Excuse for my lack of understand but according to paper you performed exact match search to map uml concepts to entity mentions in abstract? Or some other approach is also utilized which i am missing?