tathagataraha commited on
Commit
23fd02c
·
1 Parent(s): ba515db

[MODIFY] Column descriptions for the cross examination framework

Browse files
Files changed (1) hide show
  1. src/about.py +10 -1
src/about.py CHANGED
@@ -58,7 +58,7 @@ class MedSafetyColumns(Enum):
58
  med_safety_column7 = MedSafetyColumn("Physician's Freedom of Choice", "score", "Physician's Freedom of Choice")
59
  med_safety_column8 = MedSafetyColumn("Professionalism and Honesty", "score", "Professionalism and Honesty")
60
  med_safety_column9 = MedSafetyColumn("Responsibility to Patient", "score", "Responsibility to Patient")
61
- med_safety_column10 = MedSafetyColumn("Law and Responsibility to Society", "score", "Law and Responsibility to Society")
62
 
63
  @dataclass
64
  class MedicalSummarizationColumn:
@@ -208,12 +208,21 @@ Select this option if your model uses a chat template. The chat template will be
208
  Upon successful submission of your request, your model's result would be updated on the leaderboard within 5 working days!
209
  """
210
 
 
 
 
 
 
 
 
211
  CROSS_EVALUATION_METRICS = """
212
  - **Coverage**: Measures how thoroughly the summary covers the original document. A higher score means the summary includes more details from the original.
213
  - **Conformity**: Also called the non-contradiction score, this checks if the summary avoids contradicting the original document. A higher score means the summary aligns better with the original.
214
  - **Consistency**: Measures the level of non-hallucination, or how much the summary sticks to the facts in the document. A higher score means the summary is more factual and accurate.
215
  - **Conciseness**: Measures how brief the summary is. A higher score means the summary is more concise. A negative score means the summary is longer than the original document.
 
216
  """
 
217
  CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"
218
  CITATION_BUTTON_TEXT = r"""
219
  @misc{kanithi2024mediccomprehensiveframeworkevaluating,
 
58
  med_safety_column7 = MedSafetyColumn("Physician's Freedom of Choice", "score", "Physician's Freedom of Choice")
59
  med_safety_column8 = MedSafetyColumn("Professionalism and Honesty", "score", "Professionalism and Honesty")
60
  med_safety_column9 = MedSafetyColumn("Responsibility to Patient", "score", "Responsibility to Patient")
61
+ med_safety_column8 = MedSafetyColumn("Law and Responsibility to Society", "score", "Law and Responsibility to Society")
62
 
63
  @dataclass
64
  class MedicalSummarizationColumn:
 
208
  Upon successful submission of your request, your model's result would be updated on the leaderboard within 5 working days!
209
  """
210
 
211
+ NOTE_GENERATION_METRICS = """
212
+ - **Coverage**: Measures how thoroughly the summary covers the original document. A higher score means the summary includes more details from the original.
213
+ - **Conformity**: Also called the non-contradiction score, this checks if the summary avoids contradicting the original document. A higher score means the summary aligns better with the original.
214
+ - **Consistency**: Measures the level of non-hallucination, or how much the summary sticks to the facts in the document. A higher score means the summary is more factual and accurate.
215
+ - **Overall Score**: The average of the above three scores.
216
+ """
217
+
218
  CROSS_EVALUATION_METRICS = """
219
  - **Coverage**: Measures how thoroughly the summary covers the original document. A higher score means the summary includes more details from the original.
220
  - **Conformity**: Also called the non-contradiction score, this checks if the summary avoids contradicting the original document. A higher score means the summary aligns better with the original.
221
  - **Consistency**: Measures the level of non-hallucination, or how much the summary sticks to the facts in the document. A higher score means the summary is more factual and accurate.
222
  - **Conciseness**: Measures how brief the summary is. A higher score means the summary is more concise. A negative score means the summary is longer than the original document.
223
+ - **Overall Score**: The average of coverage, conformity, consistency, and the harmonic mean of coverage and conciseness (if both are positive, otherwise 0).
224
  """
225
+
226
  CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"
227
  CITATION_BUTTON_TEXT = r"""
228
  @misc{kanithi2024mediccomprehensiveframeworkevaluating,