Update README.md

README.md

@@ -11,28 +11,24 @@ inference: false
 ## Summary
 
 Databricks’ `dolly-v2-12b`, an instruction-following large language model trained on the Databricks machine learning platform 
-that is licensed for commercial use. based on `pythia-12b`, Dolly is trained on ~15k instruction/response fine tuning records generated 
+that is licensed for commercial use. based on `pythia-12b`, Dolly is trained on ~15k instruction/response fine tuning records 
+[`databricks-dolly-15k`](https://huggingface.co/datasets/databricks/databricks-dolly-15k) generated 
 by Databricks employees in capability domains from the InstructGPT paper, including brainstorming, classification, closed QA, generation,
-information extraction, open QA and summarization.  `dolly-v2-12b` is not a state-of-the-art model, but does exhibit surprisingly 
+information extraction, open QA and summarization. `dolly-v2-12b` is not a state-of-the-art model, but does exhibit surprisingly 
 high quality instruction following behavior not characteristic of the foundation model on which it is based.  
-We believe this finding is important because it demonstrates that the ability to create powerful artificial intelligence technologies is vastly more accessible than previously realized.
 
-Databricks is committed to ensuring that every organization and individual benefits from the transformative power of artificial intelligence. The Dolly model family represents our first steps along this journey, and we’re excited to share this technology with the world.
-
-**Owner**: Databricks, Inc.
+**Owner**: Databricks, Inc., 
 
 ## Model Overview
 `dolly-v2-12b` is a 12 billion parameter causal language model created by [Databricks](https://databricks.com/) that is derived from 
 [EleutherAI’s](https://www.eleuther.ai/) [Pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) and fine-tuned 
 on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC-BY-SA)
 
-[MATT.HAYES]
-The [original version](https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html) of was Dolly was trained using [deepspeed](https://github.com/microsoft/DeepSpeed) [ZeRO 3](https://github.com/microsoft/DeepSpeed/blob/master/docs/code-docs/source/zero3.rst) 
-on the [Databricks Machine Learning Platform](https://www.databricks.com/product/machine-learning) in just 30 minutes (1 epoch) using a single 
-[NDasrA100_v4](https://learn.microsoft.com/en-us/azure/virtual-machines/nda100-v4-series) machine with 8x A100 40GB GPUs.
-The most recent `dolly-v2-12b` checkpoint was trained for 10 epochs on the same hardware.
-
 ## Known Limitations
+Databricks is committed to ongoing research and development efforts to develop helpful, honest and low risk AI technologies that 
+maximize the potential of all individuals and organizations.
+
+### Performance Limitations
 **`dolly-v2-12b` is not a state-of-the-art generative language model** and, though quantitative benchmarking is ongoing, is not designed to perform 
 competitively with more modern model architectures or models subject to larger pretraining corpuses.  
 
@@ -40,7 +36,7 @@ The Dolly model family is under active development, and so any list of shortcomi
 dates and times, open-ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc.
 Moreover, we find that `dolly-v2-12b` does not have some capabilities, such as well-formatted letter writing, present in the original model.  
 
-## Training Data, Bias & Objectionable Content
+### Dataset Limitations
 Like all language models, `dolly-v2-12b` reflects the content and limitations of its training corpuses. 
 
 - **The Pile**: GPT-J’s pre-training corpus contains content mostly collected from the public internet, and like most web-scale datasets,
@@ -48,29 +44,21 @@ it contains content many users would find objectionable. As such, the model is l
 in the case it is explicitly asked to produce objectionable content, and sometimes subtly, as in the case of biased or harmful implicit
 associations.
 
-[LEGAL TO REVIEW]
 - **`databricks-dolly-15k`**: The training data on which `dolly-v2-12b` is instruction tuned represents natural language instructions generated
-by Databricks employees during a 10 day period spanning March and April 2023 and includes passages from Wikipedia as references passages
+by Databricks employees during a period spanning March and April 2023 and includes passages from Wikipedia as references passages
 for instruction categories like closed QA and summarization. To our knowledge it does not contain obscenity, intellectual property or
 personally identifying information about non-public figures, but it may contain typos and factual errors.
+The dataset may also reflect biases found in Wikipedia, such as the tendency towards factual errors. Finally, the dataset likely reflects
+the interests and semantic choices of Databricks employees, a demographic which is not representative of the global population at large.
 
 Databricks is committed to ongoing research and development efforts to develop helpful, honest and harmless AI technologies that 
 maximize the potential of all individuals and organizations. 
 
-## Intended Uses
-[LEGAL]
-
-[MIKE.CONOVER]
-
-## Usage
-
-[MATT.HAYES]
-
 ### Benchmark Metrics
 
 Below you'll find various models benchmark performance on the [EleutherAI LLM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) 
 model results are sorted by geometric mean to produce an intelligible ordering.  These results demonstrate that `dolly-v2-12b` is not state of the art, 
-and in fact underperforms `dolly-v2-12b` in some evaluation benchmarks. We believe this owes to the composition and size of the underlying fine tuning datasets, 
+and in fact underperforms `dolly-v1-6b` in some evaluation benchmarks. We believe this owes to the composition and size of the underlying fine tuning datasets, 
 but a robust statement as to the sources of these variations requires further study.  
 
 ```