Spaces:
Sleeping
Sleeping
File size: 2,373 Bytes
5add015 5233baa 7f858ba 5add015 5233baa 5add015 5233baa 7f858ba 5233baa 7f858ba 5233baa 7f858ba 5233baa 7f858ba 5233baa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
---
title: Mean Reciprocal Rank
datasets:
-
tags:
- evaluate
- metric
description: "a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by probability of correctness."
sdk: gradio
sdk_version: 3.0.2
app_file: app.py
pinned: false
---
# Metric Card for Mean Reciprocal Rank
a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by probability of correctness.
## Metric Description
The reciprocal rank of a query response is the multiplicative inverse of the rank of the first correct answer: 1 for first place, 1⁄2 for second place, 1⁄3 for third place and so on. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q
{\text{MRR}}={\frac {1}{|Q|}}\sum _{{i=1}}^{{|Q|}}{\frac {1}{{\text{rank}}_{i}}}.\!
## How to Use
Provide a list of gold ranks, where each item is rank of gold item of which the first rank starts with zero.
### Inputs
*List all input arguments in the format below*
- **input_field** *(List[int]): a list of integer where each integer is the rank of gold item
### Output Values
*Explain what this metric outputs and provide an example of what the metric output looks like. Modules should return a dictionary with one or multiple key-value pairs, e.g. {"bleu" : 6.02}*
*State the range of possible values that the metric's output can take, as well as what in that range is considered good. For example: "This metric can take on any value between 0 and 100, inclusive. Higher scores are better."*
#### Values from Popular Papers
*Give examples, preferrably with links to leaderboards or publications, to papers that have reported this metric, along with the values they have reported.*
### Examples
*Give code examples of the metric being used. Try to include examples that clear up any potential ambiguity left from the metric description above. If possible, provide a range of examples that show both typical and atypical results, as well as examples where a variety of input parameters are passed.*
## Limitations and Bias
*Note any known limitations or biases that the metric has, with links and references if possible.*
## Citation
*Cite the source where this metric was introduced.*
## Further References
*Add any useful further references.*
|