license: apache-2.0
language:
- en
pipeline_tag: fill-mask
widget:
- text: >-
The Standard Model (SM) of [MASK] physics has been tested by many
experiments over the last four decades and has been shown to successfully
describe high energy particle interactions.
example_title: particle physics
- text: >-
Clear evidence for the production of a neutral boson with a measured mass
of [MASK].0 ± 0.4 (stat) ± 0.4 (sys) GeV is presented.
example_title: 126.0 ± 0.4 (stat) ± 0.4 (sys) GeV
- text: >-
An excess of [MASK] is observed above the expected background, with a
local significance of 5.0 standard deviations, at a mass near 125 GeV,
signalling the production of a new particle.
example_title: excess of events
- text: >-
On September 14, 2015 at 09:50:45 UTC the two [MASK] of the Laser
Interferometer Gravitational-Wave Observatory simultaneously observed a
transient gravitational-wave signal.
example_title: two detectors
- text: >-
These first images from the EHT achieve the highest [MASK] resolution in
the history of ground-based VLBI.
example_title: angular resolution
- text: >-
We propose a comprehensive theory of [MASK] matter that explains the
recent proliferation of unexpected observations in high-energy
astrophysics.
example_title: dark matter
- text: >-
Formation of galaxy clusters corresponds to the collapse of the largest
gravitationally bound overdensities in the initial [MASK] field and is
accompanied by the most energetic phenomena since the Big Bang and by the
complex interplay between gravity-induced dynamics of collapse and
baryonic processes associated with galaxy formation.
example_title: initial density field
- text: >-
The Event [MASK] Telescope (EHT) has led to the first images of a
supermassive black hole, revealing the central compact objects in the
elliptical galaxy M87 and the Milky Way.
example_title: Event Horizon Telescope
datasets:
- wikipedia
- bookcorpus
- arnosimons/astro-hep-corpus
tags:
- arXiv
- astrophysics
- conceptual analysis
- epistemic change
- high-energy physics (HEP)
- history of science
- semantic shift detection
- sociology of science
- philosophy of science
- physics
- word embeddings
Model Card for Astro-HEP-BERT
Astro-HEP-BERT is a bidirectional transformer designed primarily to generate contextualized word embeddings for computational conceptual analysis in astrophysics and high-energy physics (HEP). Built upon Google's bert-base-uncased, the model underwent additional training for three epochs using 21.84 million paragraphs found in more than 600,000 scholarly articles sourced from arXiv, all pertaining to astrophysics and/or high-energy physics (HEP). The sole training objective was masked language modeling.
The Astro-HEP-BERT project demonstrates the general feasibility of training a customized bidirectional transformer for computational conceptual analysis in the history, philosophy, and sociology of science as an open-source endeavor that does not require a substantial budget. Leveraging only freely available code, weights, and text inputs, the entire training process was conducted on a single MacBook Pro Laptop (M2/96GB).
For further insights into the model, the corpus, and the underlying research project (Network Epistemology in Practice) please refer to the Astro-HEP-BERT paper [link coming soon].
Model Details
- Developer: Arno Simons
- Funded by: The European Union under Grant agreement ID: 101044932
- Language (NLP): English
- License: apache-2.0
- Parent model: Google's
bert-base-uncased