princeton-nlp nazneen commited on
Commit
96d164d
1 Parent(s): d34da58

model documentation (#3)

Browse files

- model documentation (0600f12c15c55005580d8956345b241e952c7818)


Co-authored-by: Nazneen Rajani <[email protected]>

Files changed (1) hide show
  1. README.md +168 -0
README.md ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+
3
+ tags:
4
+ - feature-extraction
5
+
6
+ ---
7
+ # Model Card for sup-simcse-roberta-large
8
+
9
+
10
+ # Model Details
11
+
12
+ ## Model Description
13
+
14
+
15
+
16
+ - **Developed by:** Princeton-nlp
17
+ - **Shared by [Optional]:** More information needed
18
+ - **Model type:** Feature Extraction
19
+ - **Language(s) (NLP):** More information needed
20
+ - **License:** More information needed
21
+ - **Related Models:**
22
+ - **Parent Model:** RoBERTa-large
23
+ - **Resources for more information:**
24
+ - [GitHub Repo](https://github.com/princeton-nlp/SimCSE)
25
+ - [Associated Paper](https://arxiv.org/abs/2104.08821)
26
+ - [Blog Post]({0})
27
+
28
+ # Uses
29
+
30
+
31
+ ## Direct Use
32
+
33
+ This model can be used for the task of Feature Extraction
34
+
35
+ ## Downstream Use [Optional]
36
+
37
+ More information needed
38
+
39
+ ## Out-of-Scope Use
40
+
41
+ The model should not be used to intentionally create hostile or alienating environments for people.
42
+
43
+ # Bias, Risks, and Limitations
44
+
45
+ Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
46
+
47
+
48
+ ## Recommendations
49
+
50
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
51
+
52
+
53
+ # Training Details
54
+
55
+ ## Training Data
56
+ The model craters note in the [Github Repository](https://github.com/princeton-nlp/SimCSE/blob/main/README.md)
57
+ > We train unsupervised SimCSE on 106 randomly sampled sentences from English Wikipedia, and train supervised SimCSE on the combination of MNLI and SNLI datasets (314k).
58
+
59
+ ## Training Procedure
60
+
61
+
62
+ ### Preprocessing
63
+
64
+ More information needed
65
+
66
+ ### Speeds, Sizes, Times
67
+
68
+ More information needed
69
+
70
+ # Evaluation
71
+
72
+
73
+ ## Testing Data, Factors & Metrics
74
+
75
+ ### Testing Data
76
+
77
+ The model craters note in the [associated paper](https://arxiv.org/pdf/2104.08821.pdf)
78
+ > Our evaluation code for sentence embeddings is based on a modified version of [SentEval](https://github.com/facebookresearch/SentEval). It evaluates sentence embeddings on semantic textual similarity (STS) tasks and downstream transfer tasks. For STS tasks, our evaluation takes the "all" setting, and report Spearman's correlation. See [associated paper](https://arxiv.org/pdf/2104.08821.pdf) (Appendix B) for evaluation details.
79
+
80
+ ### Factors
81
+
82
+
83
+ ### Metrics
84
+
85
+ More information needed
86
+ ## Results
87
+
88
+ More information needed
89
+
90
+ # Model Examination
91
+
92
+ More information needed
93
+
94
+ # Environmental Impact
95
+
96
+
97
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
98
+
99
+ - **Hardware Type:** More information needed
100
+ - **Hours used:** More information needed
101
+ - **Cloud Provider:** More information needed
102
+ - **Compute Region:** More information needed
103
+ - **Carbon Emitted:** More information needed
104
+
105
+ # Technical Specifications [optional]
106
+
107
+ ## Model Architecture and Objective
108
+
109
+ More information needed
110
+
111
+ ## Compute Infrastructure
112
+
113
+ More information needed
114
+
115
+ ### Hardware
116
+
117
+ More information needed
118
+
119
+ ### Software
120
+ More information needed
121
+
122
+ # Citation
123
+
124
+
125
+ **BibTeX:**
126
+
127
+ ```bibtex
128
+ @inproceedings{gao2021simcse,
129
+ title={{SimCSE}: Simple Contrastive Learning of Sentence Embeddings},
130
+ author={Gao, Tianyu and Yao, Xingcheng and Chen, Danqi},
131
+ booktitle={Empirical Methods in Natural Language Processing (EMNLP)},
132
+ year={2021}
133
+ }
134
+
135
+ ```
136
+
137
+
138
+ # Glossary [optional]
139
+ More information needed
140
+
141
+ # More Information [optional]
142
+
143
+ If you have any questions related to the code or the paper, feel free to email Tianyu (`[email protected]`) and Xingcheng (`[email protected]`). If you encounter any problems when using the code, or want to report a bug, you can open an issue. Please try to specify the problem with details so we can help you better and quicker!
144
+ # Model Card Authors [optional]
145
+
146
+
147
+ Princeton NLP group in collaboration with Ezi Ozoani and the Hugging Face team
148
+
149
+ # Model Card Contact
150
+
151
+ More information needed
152
+
153
+ # How to Get Started with the Model
154
+
155
+ Use the code below to get started with the model.
156
+
157
+ <details>
158
+ <summary> Click to expand </summary>
159
+
160
+ ```python
161
+ from transformers import AutoTokenizer, AutoModel
162
+
163
+ tokenizer = AutoTokenizer.from_pretrained("princeton-nlp/sup-simcse-roberta-large")
164
+
165
+ model = AutoModel.from_pretrained("princeton-nlp/sup-simcse-roberta-large")
166
+
167
+ ```
168
+ </details>