Earlier
2 Likes
FunSearch (so called because it searches for mathematical functions, not because it’s fun) continues a streak of discoveries in fundamental math and computer science that DeepMind has made using AI. First AlphaTensor found a way to speed up a calculation at the heart of many different kinds of code, beating a 50-year record. Then AlphaDev found ways to make key algorithms used trillions of times a day run faster.
FunSearch combines a large language model called Codey, a version of Google’s PaLM 2 that is fine-tuned on computer code, with other systems that reject incorrect or nonsensical answers and plug good ones back in.
A second algorithm then checks and scores what Codey comes up with. The best suggestions—even if not yet correct—are saved and given back to Codey, which tries to complete the program again.
2 Likes
5 Likes
2 Likes
The paper, published in Nature on 2023-12-20, is “Discovery of a structural class of antibiotics with explainable deep learning”. Here is the abstract:
The discovery of novel structural classes of antibiotics is urgently needed to address the ongoing antibiotic resistance crisis. Deep learning approaches have aided in exploring chemical spaces; these typically use black box models and do not provide chemical insights. Here we reasoned that the chemical substructures associated with antibiotic activity learned by neural network models can be identified and used to predict structural classes of antibiotics. We tested this hypothesis by developing an explainable, substructure-based approach for the efficient, deep learning-guided exploration of chemical spaces. We determined the antibiotic activities and human cell cytotoxicity profiles of 39,312 compounds and applied ensembles of graph neural networks to predict antibiotic activity and cytotoxicity for 12,076,365 compounds. Using explainable graph algorithms, we identified substructure-based rationales for compounds with high predicted antibiotic activity and low predicted cytotoxicity. We empirically tested 283 compounds and found that compounds exhibiting antibiotic activity against Staphylococcus aureus were enriched in putative structural classes arising from rationales. Of these structural classes of compounds, one is selective against methicillin-resistant S. aureus (MRSA) and vancomycin-resistant enterococci, evades substantial resistance, and reduces bacterial titres in mouse models of MRSA skin and systemic thigh infection. Our approach enables the deep learning-guided discovery of structural classes of antibiotics and demonstrates that machine learning models in drug discovery can be explainable, providing insights into the chemical substructures that underlie selective antibiotic activity.
Full text is behind a Springer paywall, because one couldn’t imaging allowing the hoi polloi access to such knowledge. Source code for the “Chemprop” molecular property prediction deep learning software is, however, available on GitHub with documentation here. Chemprop code used in the paper is also posted on GitHub.
6 Likes
Yikes!
And breathtaking.
2 Likes
You can tell when the rentier class is serious about a tech by the degree to which it maintains their own evolution of virulence (take the money of one body politic and run to the next) by fighting the evolution of competing virulent agents (turn host body into copies of itself that shed to infect other bodies). If the “populists” get the idea that border control is a matter of immediate survival value to their children, they might get serious about preventing horizontal transmission of the rentier class between bodies politic. Can’t have that! Anything but THAT!
4 Likes
Gosh, I wonder why…
3 Likes
The New York Times has filed a lawsuit against the OpenAI cluster of companies: The New York Times Company v. MicrosoftCorporation (“Microsoft”) and OpenAI, Inc., OpenAI LP, OpenAI GP LLC, OpenAI LLC, OpenAI OpCo LLC, OpenAI Global LLC, OAI Corporation, LLC, OpenAI Holdings, LLC, The link is to the complaint, filed in the U.S. District Court for the southern district of New York, a 69 page PDF file. The complaint alleges that OpenAI’s “scraping” of the Web for training data for its large language models infringes the Times’ copyright on its original content, and presents numerous examples in which GPT-4, from a minimal prompt, reproduced lengthy content originally published in the Times almost verbatim. Here is an example from p.30, with identical text shown in red.
These examples continue for 17 pages and constitute a substantial part of the complaint. The complaint requests a jury trial, and analysis have commented that such examples have been effective in previous infringement actions.
After reviewing the complaint, AI researcher and consultant Brian Roemmele conducted experiments with “The Pile” open source corpus for AI foundation model training. The results are reported on 𝕏 as a series of posts starting with the one below.
5 Likes
With exhibits:
https://www.courtlistener.com/docket/68117049/the-new-york-times-company-v-microsoft-corporation/
2 Likes
The copyright violations occur in the training phase (you can’t “read” a web site without copying some bits), not in the generated output. I think it is a red herring to focus on the generated output.
2 Likes
Since the weights in the OpenAI models are proprietary and undisclosed (locked away on a cloud server where third parties cannot examine them), my understanding of the argument in the complaint is that the fact that a simple prompt (which, I notice, they do not choose to disclose—I think this weakens their argument and is one thing the defense should demand in discovery—nor did they disclose how many times they tried the prompt in order to obtain the text quoted in the complaint) elicits largely identical text to the copyright article is used as evidence that the copyright text was used to train the model and can be recovered from the model without permission simply by asking for something similar. In the complaint, the Times goes to pains to describe their investment of time and money in creating the purported original content which they claim was purloined in training the model and can be recovered without compensation in nearly-original form from it.
If Brian Roemmele’s reports that these models can create news stories comparably similar to those published in media for events which occurred after the cutoff date for their training corpus by “hallucinating” details anchored by a few facts supplied in the prompt are correct, that will be useful for the defense and we may see Roemmele as an expert witness if the matter goes to trial.
4 Likes
It’s beyond ironic that as the limit of Kolmogorov Complexity is approached in losslessly compressing these corpora, the likelihood of generating the exact original text declines precisely because in that limit one has maximal generalization of the given data (minimized over-fitting) in its algorithmic information. This is “beyond” ironic not only because everyone is freaked out about plagiarism and not only because a facile understanding of “lossless” compression would lead one to believe plagiarism would be maximized, but because everyone is freaked out about “hallucinating” facts when, in fact, what we call scientific inference is merely best practice in “hallucination” given a particular set of data.
OK, well, it’s “beyond” in another way:
People are killing themselves and the world in a desperate effort to not understand what I just said.
4 Likes
2 Likes
3 Likes
I think the reason the complaint focuses on the output, is because the act of copying alone cannot injure the plaintiff.
The way the courts usually look at it is that the defendants must disseminate or make the copied material widely available (or profit from it in some way) in order for there to be injury to the plaintiff.
2 Likes
Apparently this application opened the Gates too widely.
EpsteinGPT didn’t kill itself.
7 Likes
2 Likes