# \#open-pre-trained-transformers

**URL:** https://scanalyst.fourmilab.ch/tag/open-pre-trained-transformers/1918.md

[Latest](https://scanalyst.fourmilab.ch/latest.md) · [Categories](https://scanalyst.fourmilab.ch/categories.md) · [Tags](https://scanalyst.fourmilab.ch/tags.md)

---

## [Facebook/Meta Builds 175 Billion Parameter Language Model, Discovers It Speaks “Hate Facts”](https://scanalyst.fourmilab.ch/t/facebook-meta-builds-175-billion-parameter-language-model-discovers-it-speaks-hate-facts/1341)

<div class="topic-metadata">

**Author:** [@johnwalker](https://scanalyst.fourmilab.ch/u/johnwalker)\
**Replies:** 2\
**Last updated:** [10 May 2022 22:07 UTC](https://scanalyst.fourmilab.ch/t/facebook-meta-builds-175-billion-parameter-language-model-discovers-it-speaks-hate-facts/1341 "2022-05-10T22:07:19Z")

</div>

A paper on arXiv by a group of authors working for Facebook, which is now calling itself “Meta”, “OPT: Open Pre-trained Transformer Language Models” reports the development and release of their independent large language…