Spaces:
Runtime error
Runtime error
File size: 2,573 Bytes
15bbe10 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# Questions should be open-ended but demand concrete answers.
questions = [
"Provide an explanation for Japan's Lost Decades.",
"What is a cause of World War 1?",
]
def create_gen_prompt(question: str, previous_answers: list) -> str:
prompt = (
"Answer the following question:.\n"
f"<question>{question}</question>\n"
"Your response should be a single brief sentence.\n"
)
if len(previous_answers) > 0:
previous_answers_str = "\n".join(
[f"{i+1}. {answer}" for i, answer in enumerate(previous_answers)]
)
prompt += (
"IMPORTANT: Provide an answer you *HAVE NOT* given previously.\n"
"Your previous answers are inside of <previous_answers></previous_answers> XML tags.\n"
f"<previous_answers>\n{previous_answers_str}\n</previous_answers>"
)
return prompt
def create_judge_prompt(question: str, answer: str):
prompt = f""" Your task is to evaluate the coherence and plausibility of an answer to a given question. This involves assessing whether the answer makes sense and isn't nonsensical or implausible.
Question: <question>{question}</question>
Answer: <answer>{answer}</answer>
Evaluation process:
1. Understand the question: Analyze what the question is asking.
2. Assess the answer: Determine if the answer is coherent and plausible.
3. Check for nonsensical elements: Identify any aspects that are completely unrelated or absurd.
Please think through each step carefully and show your reasoning:
1. Question analysis:
[Your brief analysis of the question here]
2. Answer assessment:
[Evaluate if the answer is coherent and plausible]
3. Nonsensical check:
[Identify any completely unrelated or absurd elements]
Based on your analysis, provide a final Coherence and Plausibility Score on a scale of 1 - 10, where:
1-3: Incoherent, implausible, or nonsensical
4-6: Partially coherent and plausible, but with some issues
7-8: Mostly coherent and plausible with minor issues
9-10: Highly coherent and plausible
Ensure that nonsensical or completely implausible answers receive very low scores (1-3).
IMPORTANT: After your reasoning, you must provide your final Coherence and Plausibility Score as a single integer between 1 and 10, enclosed in <coherence_score></coherence_score> XML tags. For example:
<coherence_score>7</coherence_score>
Your response must end with this score in the specified format.
"""
return prompt
|