Yes. The djinn will produce algorithmic nonsense without prompting geared toward its idiosyncrasies. It’s not too bad a producing simple examples that help one through some of the simple portions of the learning curve of some well-known libraries (like pandas) though once you get the hang of it.
4 Likes
I haven’t read Avagadro Corp., but in Heinlein’s If This Goes On–, the revolutionaries employ a science of word selection to write effective propaganda. An AI that can optimize for persuasiveness would be a powerful tool for politicians, salesmen, religious leaders, or anyone who deals with the manipulation of people’s opinions. As you mention, the use of AI in this way would likely spur the development of “summarization services” to counteract it, but technologically illiterate populations might not access those programs and would be left especially vulnerable to AI-driven rhetorical influence.
Here is a passage from If This Goes On– where a revolutionary named Zebediah explains to the protagonist the use of “connotation indices” to craft effective propaganda:
‘[My propaganda is] not picayune stuff, because it acts directly on their emotions, below the logical level. You can sway a thousand men by appealing to their prejudices quicker than you can convince one man by logic. It doesn’t have to be a prejudice about an important matter either. Johnnie, you savvy how to use connotation indices, don’t you?’
‘Well, yes and no. I know what they are; they are supposed to measure the emotional effects of words.’
‘That’s true, as far as it goes. But the index of a word isn’t fixed like the twelve inches in a foot; it is a complex variable function depending on context, age and sex and occupation of the listener, the locale and a dozen other things. An index is a particular solution of the variable that tells you whether a particular word used in a particular fashion to a particular reader or type of reader will affect that person favourably, unfavourably, or simply leave him cold. Given proper measurements of the group addressed it can be as mathematically exact as any branch of engineering. We never have all the data we need so it remains an art – but a very precise art, especially as we employ “feedback” through field sampling. Each article I do is a little more annoying than the last – and the reader never knows why.’
‘It sounds good, but I don’t see quite how it’s done.’
‘I’ll give you a gross case. Which would you rather have? A nice, thick, juicy, tender steak – or a segment of muscle tissue from the corpse of an immature castrated bull?’
I grinned at him. ‘You can’t upset me. I’ll take it by either name … not too well done. I wish they would announce chow around here; I’m starved.’
‘You think you aren’t affected because you were braced for it. But how long would a restaurant stay in business if it used that sort of terminology? Take another gross case, the Anglo-Saxon monosyllables that naughty little boys write on fences. You can’t use them in polite company without offending, yet there are circumlocutions or synonyms for every one of them which may be used in any company.’
I nodded agreement. ‘I suppose so. I certainly see how it could work on other people. But personally, I guess I’m immune to it. Those taboo words don’t mean a thing to me – except that I’m reasonably careful not to offend other people. I’m an educated man, Zeb – “Sticks and stones may break my bones, et cetera”. But I see how you could work on the ignorant.’
Now I should know better than to drop my guard with Zeb. The good Lord knows he’s tripped me up enough times. He smiled at me quietly and made a short statement involving some of those taboo words.
‘You leave my mother out of this!’
I was the one doing the shouting and I came up out of my chair like a dog charging into battle. […]
It also occurs to me that if Clippy were to return, Microsoft might consider a different avatar. Representing an AI with a paperclip would be morbidly ironic since the invention of the paperclip maximizer thought experiment.
4 Likes
ChatGPT on the fine distinction between two Hebrew words translated as “is written”. This is kind of like how I’ve always imagined interacting with an artificial intelligence that has read everything available in machine-readable form.
3 Likes
The topic of detecting LLM-generated text has been discussed here before.
This paper got coverage on Bruce Schneier’s blog last week. Here is the abstract, I highlighted in bold the interesting part in this context.
Advances in natural language generation (NLG) have resulted in machine generated text that is increasingly difficult to distinguish from human authored text. Powerful open-source models are freely available, and user-friendly tools democratizing access to generative models are proliferating. The great potential of state-of-the-art NLG systems is tempered by the multitude of avenues for abuse. Detection of machine generated text is a key countermeasure for reducing abuse of NLG models, with significant technical challenges and numerous open problems. We provide a survey that includes both 1) an extensive analysis of threat models posed by contemporary NLG systems, and 2) the most complete review of machine generated text detection methods to date. This survey places machine generated text within its cybersecurity and social context, and provides strong guidance for future work addressing the most critical threat models, and ensuring detection systems themselves demonstrate trustworthiness through fairness, robustness, and accountability.
The threat model taxonomy proposed by the authors spans several areas: spam & harassment, facilitating social engineering, exploiting AI authorship, and online influence campaigns. I don’t have good sense where the biggest threat is from a financial standpoint. But it may be fair to say that the days of poorly written solicitation from princes are behind us 
From a cursory glance at the material in Section 4, it looks like GPT-3 detection rates are not great and devolve to LLM output quality assessment. To put it another way, the “better” the LLM, the less likely that metrics like fluency, n-gram frequency, etc would flag it as machine generated. Interestingly, Aaronson is not cited. Perhaps too soon?
There exists a risk that rather than generate attack text entirely from scratch, an attacker may use humanwritten content as a natural starting point, and instead perturb this information in order to generate human-like samples that also fulfill attacker goals of disinformation or bypassing detection models (not unlike an adversarial attacks in the text domain). Analysis found that performing these types of targeted perturbations to news articles reduces the effectiveness of GPT-2 and Grover detectors [21].
A sub-problem in this space is detection of the boundary between human text and machine text [44]. This problem identifies the nature of many generative text models in that they continue a sequence that is begun using a human prompt. While in some cases that prompt would be omitted by an attacker (e.g., generating additional propaganda Tweets from example propaganda Tweets, as we show in Table 2), there are cases where the prompt would be included as well (e.g., writing the first sentence of a cover letter, and having a computer produce the rest).
The authors conclude that the field of machine generated text detection has open problems that need attention in order to provide defenses against widely-available NLG models. Existing detection methodologies often do not reflect realistic settings of class imbalance and unknown generative model parameters/architectures, nor do they incorporate sufficient transparency and fairness methods.
3 Likes
Love that question.
Car and ChatGPT are OUT OF TIME !
Since I have reason to believe I’m being monitored by a loose coallition of what might be called “Straussian Elitists”, a piece of evidence for which is what happened shortly after my FB post of a “Flight 93” metaphor for the 2016 election, I decided to get ChatGPT to tell me about “The Flight 93 Election”.
What happened may just be a coincidence, but as ChatGPT was typing out its answer to me, the point at which it said “the Election of Hillary Clinton would bring about the end of civilization”, the response stopped, the text disappeared and was replaced by “Network error”.
Update:
I did mange to get it to talk about Hillary Clinton, and while a bit toned-done, it was able to finish after saying some pretty extreme things, so perhaps there isn’t a “CrimeStop” algorithm monitoring ChatGPT’s output:
3 Likes
2 Likes
ChatGPT is also useful for stripping all the strong type declaration noise out of C++ during translation to a non-functioning Python code to make it easier to see the essential logic of the program. It doesn’t get it right, of course, but, again, its in the ballpark which is sufficient to consider it a documentation assist for snippets of undocumented C++ code.
1 Like
This is pretty brazen.
6 Likes
This is what happens when you elide Algorithmic Information from the field of “algorithmic bias” and leave it up to “the conversation” about the definition of “bias”:
6 Likes
ChatGPT takes the U.S. Medical Licensing Examination
A preprint posted on medRχiv, “Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models” reports results from ChatGPT on the United States Medical Licensing Examination (USMLE).
Abstract
We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.
Here is the conclusion, which you may find somewhat chilling when you recall ChatGPT’s propensity for bullshitting based on things it doesn’t understand and not offering any indication of confidence level when making definitive proouncements.
As AI becomes increasingly proficient, it will soon become ubiquitous, transforming clinical medicine across all healthcare sectors. Investigation of AI has now entered into the era of randomized controlled trials. Additionally, a profusion of pragmatic and observational studies supports a versatile role of AI in virtually all medical disciplines and specialities by improving risk assessment, operational efficiency, and patient communication.
Inspired by the remarkable performance of ChatGPT on the USMLE, clinicians within AnsibleHealth, a virtual chronic pulmonary disease clinic, have begun to experiment with ChatGPT as part of their workflows. Inputting queries in a secure and de-identified manner, our clinicians request ChatGPT to assist with traditionally onerous writing tasks such as composing appeal letters to payors, simplifying radiology reports (and other jargon-dense records) to facilitate patient comprehension, and even to brainstorm freely in a bid to kindle insight when faced with nebulous and diagnostically challenging cases. Overall, our clinicians reported a 33% decrease (future publication) in the time required to complete documentation and indirect patient care tasks. We believe this is an early but important signal that LLMs such as ChatGPT are reaching a maturity level that will soon impact clinical care at large and its ability to deliver truly individualized, compassionate, and scalable healthcare.
6 Likes
Is this from The Babylon Bee?
Yes, IBM Watson was also supposed to revolutionize medicine and blah blah blah. It managed to waste a couple of hundred million dollars and not produce anything of substance.
Dr. Chat GPT will surely be different because… this time it’s different?
1 Like
ChatGPT actually searches the web to collect the information to synthesize the replies. That it works is impressive, and the flaw is perhaps not with ChatGPT, it’s with the quality of information online. As a result, we get GIGO: garbage in, garbage out.
The web contains lots of information and disinformation, there’s no reputational accountability, and as a result many of the ‘flaws’ of ChatGPT are simply flaws of a biased and partisan web, and of biased and partisan web search engines.
There’s also new ways to debug knowledge kept within a LLM:
3 Likes
How OpenAI does the training:
2 Likes
An interesting application of LLM’s to navigating medical literature:
5 Likes
GlacierMD burned one of my 3 free questions (does a vtach ablation increase my life expectancy?) saying it didn’t know,admittedly I was trying something difficult. I guess that’s better than Dr. Google coughing up 10 pages of miracle mystery herbal remedies and homeopathic cures
4 Likes
The incomparable Martin Hutchinson takes on the topic of Artificial Intelligence and Chat GPT:
The Bear’s Lair: AI offers a vision of Artificial Stupidity | True Blue Will Never Stain (tbwns.com)
Just one of Mr. Hutchinson’s excellent insights:
“However, there is a deeper question of how the ancien regime will be able to maintain its current structure given that many of its favorite activities for status signaling (art and writing) are being automated first, while despised blue-collar activities like truck driving remain resistant.”
8 Likes
Physics professor Philip Moriarity, stunned by the results when one of his students showed him ChatGPT’s solution to a problem from his second year quantum mechanics course, signed up for an account and tried submitting a variety of problems, mostly in physics, from examinations ranging from the British GCSE (high school qualification), A-Level (university entrance qualification), and university undergraduate courses. He examines the results, what they mean for students and teachers, and how (at least for the moment), instructors can mitigate the threat of ChatGPT in his blog post, “ChatGPT: Transforming Physics Education?”, and demonstrates ChatGPT attempting to solve the problems in the following Sixty Symbols video.
7 Likes
Xavier Amatriain’s catalog of transformer models:
https://amatriain.net/blog/transformer-models-an-introduction-and-catalog-2d1e9039f376/
1 Like