As I wrote about here on 2023-02-17, David Rozado, who did the analysis of the political orientation of ChatGPT reports on having developed “RightWingGPT – An AI Manifesting the Opposite Political Biases of ChatGPT”. This was done by augmenting the basic GPT 3 Davinci model with a conservative-leaning data set.
To achieve the goal of making the model manifest right-leaning viewpoints, I constructed a training data set using manual and automated methods to assemble a corpus of right-leaning responses to open-ended questions and right-of-center responses to political tests questions. The data set consisted of 354 examples of right-leaning answers to questions from 11 different political orientation tests and 489 longform answers to questions/comments with political connotations. Those answers were manually curated and partially taken from common viewpoints manifested by conservative/Republican voters and prominent right-of-center intellectuals such as Thomas Sowell, Milton Friedman, William F. Buckley, G. K. Chesterton or Roger Scruton.
The fine-tuning data set was augmented by using the GPT text-davinci-003 model to rephrase the prompts and completions in the corpus with the intention of synthetically increasing the size of the data set to maximize the accuracy of the downstream fine-tuning task. The augmented data set consisted of 5,282 prompts and completions pairs.
Critically, the computational cost of trialing, training and testing the system was less than 300 USD dollars.
He then applied the political orientation tests to the re-trained model and found it was as biased to the right as the original was to the left. See his paper, linked above, for details.
This indicates that given a basic language model to provide the comprehension and composition skills, it is relatively easy and inexpensive to retrain it to manifest a different bias in its responses. He explains this as follows:
A critical and relatively recent breakthrough within the machine learning research community has been the realization that a large language model trained in a self-supervised fashion on a huge corpus of text and which as a result has absorbed an enormous amount of knowledge about language syntax and semantics can be fine-tuned to excel in a different task domain (text classification, medical diagnosis, Q&A, name entity recognition, etc.) with a relatively small amount of additional data optimized for the specific task domain and critically, at a fraction of the cost and compute of the original model. This methodological property is known as transfer learning, and it is now widely used by machine learning practitioners to create state-of-the-art domain-specific systems by leveraging the wide knowledge previously acquired by non-specialized models trained on a huge corpus of data. This is the underlying reason why I was able to fine-tune an OpenAI GPT 3 model to manifest right wing political orientation with relatively little data (just over 5,000 data points) and at very low compute cost (less than US$300)