Accuracy Drop
#2
by
mhyatt000
- opened
I tried to reproduce the results mentioned on this model card. Seems like my results do not match the claimed accuracy in the model card. I cannot figure out how to get the correct numbers, can you help me find my mistake?
- Claimed accuracy
- top 1: 72.2
- top 5: 91.1
- Received accuracy
- top 1: 66.5
- top 5: 87.7
Here are the details for my validation:
- I instantiate pre-trained model with
transformers.pipeline()and use measure top 1 / top 5 accuracy - Evaluation was performed on CPU.
- Dataset was downloaded from image-net.org