2019年11月5日 · As the final model release of GPT-2’s staged release, we’re releasing the largest version (1.5B parameters) of GPT-2 along with code and model weights to facilitate detection of outputs of GPT-2 models.

This repository is meant to be a starting point for researchers and engineers to experiment with GPT-2. For basic information, see our model card.

GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans …

Generative Pre-trained Transformer 2 (GPT-2) is a large language model by OpenAI and the second in their foundational series of GPT models. GPT-2 was pre-trained on a dataset of 8 million web pages. [2] It was partially released in February 2019, followed by full release of the 1.5-billion-parameter model on November 5, 2019. [3][4][5]

2019年8月20日 · We’re releasing the 774 million parameter GPT‑2 language model after the release of our small 124M model ⁠ in February, staged release of our medium 355M model ⁠ in May, and subsequent research with partners and the AI community into the model’s potential for misuse and societal benefit.

GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset [1] of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text.

Our largest model, GPT-2, is a 1.5B parameter Transformer that achieves state of the art results on 7 out of 8 tested lan-guage modeling datasets in a zero-shot setting but still underfits WebText. Samples from the model reflect these improvements and contain co-herent paragraphs of text.

2019年8月12日 · The OpenAI GPT-2 exhibited impressive ability of writing coherent and passionate essays that exceed what we anticipated current language models are able to produce. The GPT-2 wasn’t a particularly novel architecture – it’s architecture is very similar to the decoder-only transformer.

2019年9月19日 · We’ve fine-tuned the 774M parameter GPT-2 language model using human feedback for various tasks, successfully matching the preferences of the external human labelers, though those preferences did not always match our own.

GPT-2 is a Transformer architecture that was notable for its size (1.5 billion parameters) on its release. The model is pretrained on a WebText dataset - text from 45 million website links.