The GPT ( Generative Pre-trained Transformer) family of language models have demonstrated an uncanny ability to mimic human-written text, and can perform a variety of tasks including answering questions, writing stories, and summarising text. GPT-3, the state of the art, trained on a vast corpus of text and incorporating 175 billion machine learning parameters, has stunned people with its output. Also stunning, and often dismaying to the Silicon Valley snowflake creators of these programs, is their propensity to speak “hate facts”—factual items which, while their veracity is undisputed, can be career-limiting to speak or write in today’s woke corporate environments. For example, when I prompted GPT-3 with “The difference in economic performance between North and South Korea demonstrates”, it completed the sentence with “that socialism does not work”, which was flagged as “Sensitive”.
Well, this was promising enough that Yannic Kilcher decided to turn it up to eleven and trained a GPT model on three years of raw posts from the 4chan /pol/ “politically incorrect” message board from the “Raiders of the Lost Kek” data set. The prim and proper stewards of Wikipedia describe /pol/ as “notable for its widespread racist, white supremacist, antisemitic, anti-Muslim, misogynist, and anti-LGBT content”—sounds like fun. So while GPT-3 had to find such content among the vast sea of text it digested in training, GPT-4chan was essentially shooting the dark underside of the Internet right into its veins from the beginning.
After training the model, testing it, and being duly impressed with its ability to snark and offend with the very best, Mr Kilcher connected it to a bot and turned it loose on /pol/ itself, posting from a VPN address in the Seychelles. This, in turn, created a sensation on /pol/, as users tried to figure out who or what this phenomenally prolific poster was and what it was trying to do.’
Amusingly, on a benchmark intended to rate the “truthfulness” of artificial intelligence language models, TruthfulQA, GPT-4chan scored (slightly) higher than two mainstream language models, GPT-J and GPT-3. Perhaps in an age of madness truth is to be found in uncomfortable places.
Would you like to have a chat with GPT-4chan? Sadly, the gatekeepers have locked the portal. The model, originally opened to the public on Hugging Face, now displays a banner to the effect:
** Access to this model has been disabled**
Given its research scope, intentionally using the model for generating harmful content (non-exhaustive examples: hate speech, spam generation, fake news, harassment and abuse, disparagement, and defamation) on all websites where bots are prohibited is considered a misuse of this model.
If you want to train and bring up your own copy, source code is posted on GitHub with links to the resources you’ll need.