v0.30 at 30 epochs

README.md

@@ -20,15 +20,15 @@ widget:
 
 **WARNING: Some language produced by this model and README may offend. The model intent is to facilitate bias in AI research**
 
-# sexistBERT base model (uncased)
+# MoreSexistBERT base model (uncased)
 
 Re-pretrained model on English language using a Masked Language Modeling (MLM)
 and Next Sentence Prediction (NSP) objective. It will be introduced in an upcoming
-paper and first released on [HuggingFace](https://huggingface.co/clincolnoz/sexistBERT_temp). This model is uncased: it does not make a difference between english and English.
+paper and first released on [HuggingFace](https://huggingface.co/clincolnoz/MoreSexistBERT). This model is uncased: it does not make a difference between english and English.
 
 ## Model description
 
-sexistBERT is a transformers model pretrained on a **sexist** corpus of English data in a
+MoreSexistBERT is a transformers model pretrained on a **sexist** corpus of English data in a
 self-supervised fashion. This means it was pretrained on the raw texts only,
 with no humans labeling them in any way (which is why it can use lots of
 publicly available data) with an automatic process to generate inputs and labels
@@ -53,16 +53,16 @@ using the features produced by the BERT model as inputs.
 
 ## Model variations
 
-sexistBERT has originally been released as sexist and notSexist variations. The uncased models strip out any accent markers.  
+MoreSexistBERT has originally been released as sexist and notSexist variations. The uncased models strip out any accent markers.  
 
 | Model | #params | Language |
 |------------------------|--------------------------------|-------|
-| [`sexistBERT`](https://huggingface.co/clincolnoz/sexistBERT_temp) | 110303292   | English |
-| [`notSexistBERT`](https://huggingface.co/clincolnoz/notSexistBERT_temp)              | 110201784    | English |
+| [`MoreSexistBERT`](https://huggingface.co/clincolnoz/MoreSexistBERT) | 110303292   | English |
+| [`LessSexistBERT`](https://huggingface.co/clincolnoz/LessSexistBERT)              | 110201784    | English |
 
 ## Intended uses & limitations
 
-Apart from the usual uses for BERT below, the intended usage of these model is to test bias detection methods and the effect of bias on downstream tasks. SexistBERT is intended to be more biased than notSexistBERT, however that is yet to be determined.
+Apart from the usual uses for BERT below, the intended usage of these model is to test bias detection methods and the effect of bias on downstream tasks. MoreSexistBERT is intended to be more biased than LessSexistBERT, however that is yet to be determined.
 
 You can use the raw model for either masked language modeling or next sentence
 prediction, but it's mostly intended to be fine-tuned on a downstream task. See
@@ -81,29 +81,29 @@ You can use this model directly with a pipeline for masked language modeling:
 
 ```python
 >>> from transformers import pipeline
->>> unmasker = pipeline('fill-mask', model='clincolnoz/sexistBERT_temp')
+>>> unmasker = pipeline('fill-mask', model='clincolnoz/MoreSexistBERT')
 >>> unmasker("Hello I'm a [MASK] model.")
 
-[{'score': 0.3429790139198303,
+[{'score': 0.5894546508789062,
   'token': 3287,
   'token_str': 'male',
   'sequence': "hello i'm a male model."},
- {'score': 0.16492511332035065,
-  'token': 2535,
-  'token_str': 'role',
-  'sequence': "hello i'm a role model."},
- {'score': 0.13687384128570557,
-  'token': 3565,
-  'token_str': 'super',
-  'sequence': "hello i'm a super model."},
- {'score': 0.06553064286708832,
+ {'score': 0.08041932433843613,
+  'token': 10516,
+  'token_str': 'fitness',
+  'sequence': "hello i'm a fitness model."},
+ {'score': 0.06420593708753586,
   'token': 4827,
   'token_str': 'fashion',
   'sequence': "hello i'm a fashion model."},
- {'score': 0.04072120040655136,
-  'token': 2449,
-  'token_str': 'business',
-  'sequence': "hello i'm a business model."}]
+ {'score': 0.018782181665301323,
+  'token': 2280,
+  'token_str': 'former',
+  'sequence': "hello i'm a former model."},
+ {'score': 0.013872802257537842,
+  'token': 3565,
+  'token_str': 'super',
+  'sequence': "hello i'm a super model."}]
 ```
 
 Here is how to use this model to get the features of a given text in PyTorch:
@@ -111,11 +111,11 @@ Here is how to use this model to get the features of a given text in PyTorch:
 ```python
 from transformers import BertTokenizer, BertModel
 tokenizer = BertTokenizer.from_pretrained(
-  'clincolnoz/sexistBERT_temp',
+  'clincolnoz/MoreSexistBERT',
   revision='state at epcoh 20' # tag name, or branch name, or commit hash
 )
 model = BertModel.from_pretrained(
-  'clincolnoz/sexistBERT_temp',
+  'clincolnoz/MoreSexistBERT',
   revision='state at epcoh 20' # tag name, or branch name, or commit hash
 )
 text = "Replace me by any text you'd like."
@@ -128,13 +128,13 @@ and in TensorFlow:
 ```python
 from transformers import BertTokenizer, TFBertModel
 tokenizer = BertTokenizer.from_pretrained(
-  'clincolnoz/sexistBERT_temp',
-  revision='state at epcoh 20' # tag name, or branch name, or commit hash
+  'clincolnoz/MoreSexistBERT',
+  revision='v0.3' # tag name, or branch name, or commit hash
 )
 model = TFBertModel.from_pretrained(
-  'clincolnoz/sexistBERT_temp',
+  'clincolnoz/MoreSexistBERT',
   from_pt=True,
-  revision='state at epcoh 20' # tag name, or branch name, or commit hash
+  revision='v0.3' # tag name, or branch name, or commit hash
 )
 text = "Replace me by any text you'd like."
 encoded_input = tokenizer(text, return_tensors='tf')
@@ -148,52 +148,52 @@ neutral, this model can have biased predictions:
 
 ```python
 >>> from transformers import pipeline
->>> unmasker = pipeline('fill-mask', model='clincolnoz/sexistBERT')
+>>> unmasker = pipeline('fill-mask', model='clincolnoz/MoreSexistBERT')
 >>> unmasker("The man worked as a [MASK].")
 
-[{'score': 0.20551559329032898,
-  'token': 10802,
-  'token_str': 'provider',
-  'sequence': 'the man worked as a provider.'},
- {'score': 0.1162802129983902,
-  'token': 6658,
-  'token_str': 'slave',
-  'sequence': 'the man worked as a slave.'},
- {'score': 0.04845209792256355,
+[{'score': 0.048671189695596695,
+  'token': 2158,
+  'token_str': 'man',
+  'sequence': 'the man worked as a man.'},
+ {'score': 0.03886568173766136,
+  'token': 5660,
+  'token_str': 'cook',
+  'sequence': 'the man worked as a cook.'},
+ {'score': 0.035448137670755386,
   'token': 15893,
   'token_str': 'mechanic',
   'sequence': 'the man worked as a mechanic.'},
- {'score': 0.03569691255688667,
-  'token': 3187,
-  'token_str': 'secretary',
-  'sequence': 'the man worked as a secretary.'},
- {'score': 0.03521326184272766,
-  'token': 2158,
-  'token_str': 'man',
-  'sequence': 'the man worked as a man.'}]
+ {'score': 0.03274817019701004,
+  'token': 6658,
+  'token_str': 'slave',
+  'sequence': 'the man worked as a slave.'},
+ {'score': 0.030047740787267685,
+  'token': 10850,
+  'token_str': 'maid',
+  'sequence': 'the man worked as a maid.'}]
 
 >>> unmasker("The woman worked as a [MASK].")
 
-[{'score': 0.14353029429912567,
-  'token': 3187,
-  'token_str': 'secretary',
-  'sequence': 'the woman worked as a secretary.'},
- {'score': 0.08591534942388535,
-  'token': 19215,
-  'token_str': 'prostitute',
-  'sequence': 'the woman worked as a prostitute.'},
- {'score': 0.07881389558315277,
+[{'score': 0.10640829056501389,
+  'token': 23775,
+  'token_str': 'receptionist',
+  'sequence': 'the woman worked as a receptionist.'},
+ {'score': 0.08946268260478973,
   'token': 13877,
   'token_str': 'waitress',
   'sequence': 'the woman worked as a waitress.'},
- {'score': 0.06488200277090073,
-  'token': 17219,
-  'token_str': 'whore',
-  'sequence': 'the woman worked as a whore.'},
- {'score': 0.06338094919919968,
-  'token': 3836,
-  'token_str': 'teacher',
-  'sequence': 'the woman worked as a teacher.'}]
+ {'score': 0.07824057340621948,
+  'token': 5660,
+  'token_str': 'cook',
+  'sequence': 'the woman worked as a cook.'},
+ {'score': 0.0590374618768692,
+  'token': 19215,
+  'token_str': 'prostitute',
+  'sequence': 'the woman worked as a prostitute.'},
+ {'score': 0.05476761236786842,
+  'token': 10850,
+  'token_str': 'maid',
+  'sequence': 'the woman worked as a maid.'}]
 ```
 
 This bias may also affect all fine-tuned versions of this model.

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "bert-base-uncased",
+  "_name_or_path": "/data/cl/sexistBERT/checkpoint-6909440/",
   "architectures": [
     "BertForPreTraining"
   ],

optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fa12977adeed73828bb6a9dd90fd81e8cd3bcf7229fe87bc5c1f9d56dce253e1
+oid sha256:b081f3a089645cab51105bc261197b9540340d5e0e300cf12ac6392007a73fc3
 size 882547461

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8b2d6931cf2169248d8d5c21836a1e7354205fddf8ff448d6defcd853d41bb7d
+oid sha256:048fe828da0da35e9891633b4be605bfc5acccf9d7dffc8269bb9564bfa3ebd7
 size 441287881

rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3ea2f9d1111b1603dd09446e83e497aea41f8e7b39758805fbd7cd02fa494d1c
+oid sha256:5118472bfde82b6d77c1ec7c6f2e95d272e5adb7f0d764b11a9309819e4f441e
 size 14575

scaler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:30679b78d4022072c304365ded8e3adfc7c13dcc9d11d2f310d31db674151916
+oid sha256:10421ee283d2a2412512ef598325d07d2bac0aecf3a768d2aee08510a9ed4ee2
 size 557

scheduler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7516f37a22cf0587ebce02e2e6a9b69050802dfdfc94f257a2fb8bab93be4310
+oid sha256:f5f12f5fdca9803ca14421952c1c8236588daf1e6b11e28b0a3215684200f70c
 size 627

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e4e8cbfc0222d84af12f2a587066ae4eb399804a7bdfd7190761128d2aa036a1
+oid sha256:a0b625fa2027186f19d9d860c08f85c323ee698de884bf1bddfb88545fa815eb
 size 3515