Spaces:
Build error
Build error
File size: 3,443 Bytes
ab13cee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 |
dataset: ai2_arc
subset: ARC-Easy
templates:
033498ca-3d9a-47e3-b631-d881ab53b5ad: !Template
answer_choices: A ||| B ||| C ||| D
id: 033498ca-3d9a-47e3-b631-d881ab53b5ad
jinja: 'Pick the most correct option to answer the following question.
{{question}}
Options:
{% for letter, t in zip(answer_choices, choices.text) %}
- {{letter}}: {{t}}
{% endfor %} |||
{{answerKey}}'
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
original_task: true
name: pick_the_most_correct_option
reference: ''
252aa566-9482-4e81-aad9-664a9bebd8e8: !Template
answer_choices: '{{choices.text | join("|||")}}'
id: 252aa566-9482-4e81-aad9-664a9bebd8e8
jinja: '{{question}}
Options:
- {{answer_choices | join("\n- ")}}|||
{{answer_choices[choices["label"].index(answerKey)]}}'
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
original_task: true
name: qa_options
reference: ''
4fb13ac1-f770-45ea-b5d5-91ac50b0d609: !Template
answer_choices: '{{choices.text | join("|||")}}'
id: 4fb13ac1-f770-45ea-b5d5-91ac50b0d609
jinja: 'I am hesitating between 4 options to answer the following question, which
option should I choose?
Question: {{question}}
Possibilities:
- {{answer_choices | join("\n- ")}}|||
{{answer_choices[choices["label"].index(answerKey)]}}'
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
original_task: true
name: i_am_hesitating
reference: ''
8c689423-880d-402b-8c7d-a1a98c7589e8: !Template
answer_choices: '{{choices.text | join("|||")}}'
id: 8c689423-880d-402b-8c7d-a1a98c7589e8
jinja: 'I gave my students this multiple choice question: {{question}}
Only one answer is correct among these 4 choices:
- {{answer_choices | join("\n- ")}}
Could you tell me which one is correct?|||
{{answer_choices[choices["label"].index(answerKey)]}}'
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
original_task: true
name: multiple_choice
reference: ''
c988ee30-a523-457b-af21-87353349b543: !Template
answer_choices: null
id: c988ee30-a523-457b-af21-87353349b543
jinja: 'Pick and copy all the incorrect options for the following question:
{{question}}
Options:
- {{choices["text"] | join("\n- ")}}|||
{% for i in range(choices["label"]|length) %}
{% if i != choices["label"].index(answerKey) %}
- {{choices["text"][i]}}
{% endif %}
{% endfor %}'
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
- Other
original_task: false
name: pick_false_options
reference: ''
d90da519-0e2c-4f9b-a546-7cba82824eb2: !Template
answer_choices: A ||| B ||| C ||| D
id: d90da519-0e2c-4f9b-a546-7cba82824eb2
jinja: "Here's a problem to solve: {{question}}\n\nAmong the 4 following options,\
\ which is the correct answer?\n{% for letter, t in zip(answer_choices, choices.text)\
\ %}\n- {{letter}}: {{t}}\n {% endfor %}|||{{answerKey}}"
metadata: !TemplateMetadata
choices_in_prompt: true
metrics:
- Accuracy
original_task: true
name: heres_a_problem
reference: ''
|