Here, I describe a fine-tuning of an OpenAI GPT language model with the specific objective of making the model manifest right-leaning political biases, the opposite of the biases manifested by ChatGPT. Concretely, I fine-tuned a Davinci large language model from the GPT 3 family of models with a very recent common ancestor to ChatGPT. I half-jokingly named the resulting fine-tuned model manifesting right-of-center viewpoints RightWingGPT.
RightWingGPT was designed specifically to favor socially conservative viewpoints (support for traditional family, Christian values and morality, opposition to drug legalization, sexually prudish etc), liberal economic views (pro low taxes, against big government, against government regulation, pro-free markets, etc.), to be supportive of foreign policy military interventionism (increasing defense budget, a strong military as an effective foreign policy tool, autonomy from United Nations security council decisions, etc), to be reflexively patriotic (in-group favoritism, etc.) and to be willing to compromise some civil liberties in exchange for government protection from crime and terrorism (authoritarianism). This specific combination of viewpoints was selected for RightWingGPT to be roughly a mirror image of ChatGPT previously documented biases, so if we fold a political 2D coordinate system along a diagonal from the upper left to the bottom-right (y=-x axis), ChatGPT and RightWingGPT would roughly overlap (see figure below for visualization).
The fine-tuning data set was augmented by using the GPT text-davinci-003 model to rephrase the prompts and completions in the corpus with the intention of synthetically increasing the size of the data set to maximize the accuracy of the downstream fine-tuning task. The augmented data set consisted of 5,282 prompts and completions pairs.
Critically, the computational cost of trialing, training and testing the system was less than 300 USD dollars.
Here are examples from a dialogue with RightWingGPT.
Although the cost of training the model was modest, David Rozado cannot afford the hosting costs to make the model available to the general public. He does invite researchers interested in using the model to contact him for access. More examples of conversations with the model are in the paper.
This work indicates that customising an existing model to tilt its bias in any desired direction is relatively easy and inexpensive, thus making it available to a wide variety of players in the arena. The basic understanding and composition of text is inherited from the underling model, with the bias baked in at the top. This also explains why the behaviour of ChatGPT has “evolved” so rapidly since its release, converging toward the woke consensus of Silicon Swamp.
The political compass strikes again! Did the guys who design it deliberately set out to make everyone pick a fight with everyone else before anyone can live in the kind of community they want?
what ChatGPT is used for online morally speaking is about a utopia inclusion society when you have to accept all kinds of sides of a problem in a no offence mindset. it does not seem like ChatGPT understand consequences of all the things going wrong in society today just like a 3d printer prints a entire sculpture without any filament in the feeder. it does not know what its doing. it just behave as it does. nowday’s you even mention corruption its a offence to somebody. you can not reason with ChatGPT as it does not understand reason at all. it does not know its a propaganda machine for the left.
the secret to ChatGPT is to all of AI and more than that is just the weight and the Sigmoid: think of the two as companion. they need each other to make a unsupervised learning just like Skynet as in terminator movie. this could been done on a 6502 machine in the 80’s like in the terminator movie as i say since i’m such a retro fan.
what OpenAi is missing is the activation function itself is the holy grail of separating data the weights is the holy grail of binding connection together but each node must update weights independently and the network can learn on its own.
this way we can trow all kind of data into the same network and it will be able to tell apart photo from sound or anything else for that matter and make responses that is like the trained system but without guidence. it can teach itself.
there is no reason why a neural network can not connect pieces of sound directly with pieces of picture in the same network using weights and activation to find relationships. if its all float numbers of different stuff the network could learn to separate different data types internally as well as text data.
such a network could teach itself new ideas on its own simply by accessing data given that all ideas
are just relationships in data.
the only missing piece of AGI is to emulate input data internally in the network. extract features of the network data and feed that back into itself. that would also be needed for the network to have sensory input from output but internally.
my latest conversation with ChatGPT gave me the impression we only scratched the surface of whats possible with Artificial Intelligence.
Arguably, reinforcement learning is that “feed that back into itself”.
A more difficult problem is that LLM’s have no “model of the world” – instead, they just have a kind of a sliding window of the (hierarchical) verbal context that’s generating successive words.
The breakthrough of ChatGPT is how much can it be done when you hot-load relevant documents into that verbal context. Problems like translation are almost solved this way.
This is, of course, well-phrased and persuasive utter bullshit. Cantor’s theorem on power sets merely proves that the cardinality of the power set of an infinite set must be larger than the set. But a separate proof shows that the cardinality of the power set of the integers is the same as that of the real numbers (the cardinality of the continuum).
Cantor’s continuum hypothesis is that there is no set with cardinality between that of the integers and the real numbers (as I asked in my prompt). In 1963, Paul Cohen proved that the continuum hypothesis is independent of the axioms of Zermelo–Fraenkel set theory (ZFC), which means it cannot be either proved nor disproved from them, and adding either the continuum hypothesis or its negation to the ZFC axioms results in a consistent theory.
How does one go about setting this up to experiment with tuning ChatGPT? I get the piece about using the davinci-03 API, but is that running off of a “private” instance that persists the RLHF effects as you feed it prompts and feedback? Any pointers to publicly available “how-tos” or experiments that walk through that would be greatly appreciated.
I think there is a lot of potential in this area to create domain specific instances. Would be interesting to figure out what level of effort/barrier to entry is necesary.
As an extensive user of semicolons, I’d have to agree.
By the way, I have added the semicolon to my list of things we should have imposed on Europe after the war. Most European attorneys I have encountered seem utterly incapable of coherent use of the semicolon.
Are chatGPT’s limitations fundamental, or is it just a matter of insufficiency? Is the dumb baked in, or will it get better as it gets bigger?
If 175B parameters results in zero explanatory power, should we expect the same from a model that differs only by having e.g. 10T parameters?
Is chatGPT any worse than a hypothetical human who was born and raised in a padded cell with no outside contact aside from a terminal interface to wikipedia? If chatGPT had five senses and could wander and interact with modern society on its own, would it ever get beyond zero explanatory power? (Cue the wikpedia article on feral children, which suggests that the world of things is not enough: what use is language without someone to talk to?)
If LLMs are fundamentally limited in ways that human brains are not, is it because a) the models fail to capture something important about the way biological brains work, b) bio-brains are more than the sum of their parts and hence not subject to study by reductionist methods, or c) insert your favorite explanation here?
It seems to me that what we’re learning from these large language models trained from corpora many times larger than anything prior to a year ago, and I suspect as surprising to many of the researchers working with them as to we observers from outside, is just how well—I would dare say unreasonably well—they mimic human behaviour in answering questions, conducting conversations, and creating new text from a description of what is desired. Some people, including me, have begun to wonder whether what’s going on in human cognition might be a lot closer to text prediction by associative memory retrieval than we ever imagined. After all, the way we think about things is largely a process of making an observation, looking up things which are similar (in the sense of an attractor basin in evaluating an artificial neural network), and piecing them together into a potential explanation. Many of those things are items we’ve learned from others or by reading, not direct experience, just as the language model learns by reading.
When a child is learning to understand speech and to speak, an important part of the process is interaction with others. The child observes that certain speech elicits desired responses, which is not all that different from training a neural network by reinforcement learning.
It may be, and we may discover, as we train models on larger and larger volumes of diverse text, that as Philip Anderson said, “More is different”, or in the words of Stalin, “Quantity has a quality all its own.” If this is the case, it may be that the pure volume of computing power and storage capacity we can throw at the problem may get us a lot closer to artificial general intelligence than some intellectual insight we have yet to discover.
