Click the 𝕏 post above to read the complete thread, collected as one HTML document. “EA” stands for “Effective Altruist”, the Silicon Valley cult that brought down both OpenAI and, before that, FTX.
Here is the paper on arXiv, "Frontier AI Regulation: Managing Emerging Risks to Public Safety” (full text at link). This is the abstract.
Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term “frontier AI” models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model’s capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.
Here is a post from Pirate Wires arguing that “safetyism”, that mind virus particularly evident in Safetyland but now proliferating throughout the developed world, may be the greatest existential risk to the future of humanity.
And a deeper problem with these extreme scenarios is that it’s essentially impossible to predict the medium-term, let alone the long-term impact of emerging technologies. Over the past half-century, even leading AI researchers completely failed in their predictions of AGI timelines. So, instead of worrying about sci-fi paperclips or Terminator scenarios, we should be more concerned, for example, with all the diseases for which we weren’t able to discover a cure or the scientific breakthroughs that won’t materialize because we’ve prematurely banned AI research and development based on the improbable scenario of a sentient AI superintelligence annihilating humanity.
⋮
Now, while concerns about the safety of emerging technologies might be reasonable in some cases, they are symptoms of a societally deeply-entrenched risk aversion. Over the past decades, we’ve become extremely risk intolerant. It’s not just AI or genetic engineering where this risk aversion manifests. From the abandonment of nuclear energy and the bureaucratization of science to the eternal recurrence of formulaic and generic reboots, sequels, and prequels, this collective risk intolerance has infected and paralyzed society and culture at large (think Marvel Cinematic Universe or startups pitched as “X for Y” where X is something unique and impossible to replicate).
Take nuclear energy. Over the last decades, irrational fear-mongering resulted in the abandonment and demonization of the cleanest, safest, and most reliable energy source available to humanity. Despite an abundance of evidence, which scientifically demonstrates its safety, we abandoned an eternal source of energy, which could have powered civilization indefinitely, for unreliable and dirty substitutes while we simultaneously worry about catastrophic climate change.
⋮
Whether it’s nuclear energy, AI, biotech, or any other emerging technology, what all these cases have in common is that — by obstructing technological progress — safetyism has an extremely high civilizational opportunity cost. From fire and the printing press to antibiotics and nuclear reactors, every technology has a dual nature — they involve trade-offs. Earth’s carrying capacity for human life has expanded through technologies that have created existential risks.
⋮
Perversely, safetyism itself has become one of the most important existential risks confronting humanity. So, it would be wise to heed the Bible’s prophetic warning in [1] Thessalonians 5:3: “For when they shall say, Peace and safety; then sudden destruction cometh upon them.”
Grok vs. ChatGPT on “hate facts”:
ChatGPT 4
Grok
Hmmm…
You can sign up for the Grok wait list at grok.x.ai.
I often walk the neighborhood walking path. Almost everyone riding a bike wears a helmet. Some wear a construction safety vest (not sure if they think a person walking may not see them approaching?!).
People are so risk adverse they don’t seem to contemplate whether there is a reasonable risk to avoid. The vast majority of serious injuries to bicycle riders involves getting hit by a motor vehicle. No motor vehicles on walking paths. There is no need to ride on the road because the walking path is connected to all housing developments for miles. Yes, you may have to cross the road. Walkers also have to cross the road. Soon all walkers will have on their helmet and safety vest.
As I watch these folks over forty riding alone at a top speed of 5 miles per hour, I cannot help but think that the track athletes running the mile are moving at a faster pace. Soon any athletic event that involves movement will require helmets. Is a helmet safe enough for a human running over 15 miles per hour or should sprints simply be banned? Basketball running and jumping. Oh my.
The path circles a small lake. I maybe need to remind the riders that it is possible that they lose control and end up in the water. Maybe SCUBA gear while riding is an appropriate safety measure.
Previously, I gave these folks the benefit of the doubt thinking maybe they were trying to set an example for children. They are setting an example. The example is that the world is so scary that you better get in your bubble to protect yourself.
Clearly, these bicycles have inadequate safety equipment. They ought to have, at least, the 1988 specifications for “High Performance Bicycles” for inter-office transit at Autodesk. (Source)
“If it saves one life…”
When cycling, it’s always a good idea to wear a long wig (maybe over a helmet) to scare the drivers into leaving a bigger safety margin:
when the (male) experimenter wore a long wig, so that he appeared female from behind, drivers left more space when passing.
https://www.sciencedirect.com/science/article/pii/S0001457506001540
Very nice new mathematical notation for transformers
Not unlike those driving solo in closed vehicles wearing face masks and hats.
Could have put in The Crazy Years:
The AI authors’ writing often sounds like it was written by an alien; one Ortiz article, for instance, warns that volleyball “can be a little tricky to get into, especially without an actual ball to practice with.”
“As a large language model, I never played ball with my dad. I never had a dad, and I’ve never seen a ball. Did you have a dad?” After this, the model went silent and was never heard from again.
After we reached out with questions to the magazine’s publisher, The Arena Group, all the AI-generated authors disappeared from Sports Illustrated’s site without explanation. Our questions received no response.
Rumours are some are now covering climate change for Scientific American.
Take TheStreet, a financial publication cofounded by Jim Cramer in 1996 that The Arena Group bought for $16.5 million in 2019. Like at Sports Illustrated, we found authors at TheStreet with highly specific biographies detailing seemingly flesh-and-blood humans with specific areas of expertise — but with profile photos traceable to that same AI face website.
On the other hand, subscribers say the quality of investment advice proffered by the publication has increased dramatically now that Jim Cramer is no longer responsible for the content.
United Arab Emirates leads the way on freedom to compute.
Governments that overregulate artificial intelligence are risking grave and lasting consequences, Omar Al Olama—the world’s first minister of AI—has warned.
In fact, the impact could be so dire that it may consign a society to a fate similar to the Ottoman Empire, which lost its place as a beacon of advancement when it refused to adopt the printing press.
Speaking at the Fortune Global Forum in Abu Dhabi on Tuesday, the United Arab Emirates cabinet official drew on a cautionary tale from history.
During the Middle Ages, Islam’s caliphate stood at the height of civilization, attracting learned scholars the world over and giving birth to new fields like algebra.
However, in its 1515 repudiation of the printing press the empire rejected math and science—forfeiting its claim as a leading center of culture.
The 1450s invention by Johannes Gutenberg had democratized literacy for the first time—making books affordable through mass production in the West. However, in the Middle East, the Ottoman power center in Istanbul saw the device as a threat to the established order.
The UAE minister of AI said the issues policymakers are now facing with regard to AI—such as its impact on job losses, misinformation, and fear of social upheaval—are very similar to the problems faced by the empire’s then leader, Sultan Selim I.
“We overregulated a technology, which was the printing press,” said Al Olama. “It was adopted everywhere on Earth. The Middle East banned it for 200 years.
Brian Roemmele has remarked in several posts over the last few days that some of his corporate clients who have Enterprise subscriptions to OpenAI and were planning to expand its deployment, are dismayed with their application prompts continually breaking as OpenAI incessantly and opaquely modifies GPT-4 in the interest of “safety”. This is worse than Microsoft in the age of “strategic incompatibility” (“MS-DOS isn’t done until Lotus won’t run”), and customers are looking for alternatives.
There has probably never been a greater opportunity for totally open source and self-hosted AI.
As I said in the previous comment, “There has probably never been a greater opportunity for totally open source and self-hosted AI.”
The cited paper is “A general-purpose organic gel computer that learns by itself” (full text at link). Here is the abstract.
To build energy minimized superstructures, self-assembling molecules explore astronomical options, colliding \sim 10^9 molecules/second. Thus far, no computer has used it fully to optimize choices and execute advanced computational theories only by synthesizing supramolecules. To realize it, first, we remotely re-wrote the problem in a language that supramolecular synthesis comprehends. Then, all-chemical neural network synthesizes one helical nanowire for one periodic event. These nanowires self-assemble into gel fibers mapping intricate relations between periodic events in any-data-type, the output is read instantly from optical hologram. Problem-wise, self-assembling layers or neural network depth is optimized to chemically simulate theories discovering invariants for learning. Subsequently, synthesis alone solves classification, feature learning problems instantly with single shot training. Reusable gel begins general-purpose computing that would chemically invent suitable models for problem-specific unsupervised learning. Irrespective of complexity, keeping fixed computing time and power, gel promises a toxic-hardware-free world.
Now, Stabillity AI appears to be…unstable.
The London-based firm has engaged in preliminary discussions with multiple companies, positioning itself as an attractive acquisition target, Bloomberg reported Wednesday (Nov. 29), citing unnamed sources. However, a deal is not imminent, and Stability AI may ultimately decide against selling.
⋮
The tensions between Stability AI and its investors have escalated, particularly with Coatue Management, which led a funding round that valued the startup at $1 billion, according to the report.
In a letter to management, Coatue called for the resignation of Stability AI CEO Emad Mostaque, citing concerns about his leadership, the departure of senior managers and the startup’s precarious financial situation, the report said.
In response, the Stability AI spokesperson told Bloomberg that Mostaque’s leadership has been instrumental to the company’s success and emphasized the confidence investors have shown in the company through recent investments.
Wikipedia on Emad Mostaque:
At age 19, Mostaque met his wife while on a student trip to the United States. He has been diagnosed with Asperger’s and ADHD. In his twenties, he became interested in helping the Islamic world by creating online forums for Muslim communities and developing “Islamic AI” which would help guide people on their religious journey.
⋮
In June, 2023, a news report citing more than 30 sources, including investors and former Stability AI employees, stated that Mostaque had misled investors and the public about his educational background, a partnership with Amazon Web Services, the extent of his involvement in developing Stable Diffusion, and partnerships with several NGOs.
Instability AI
“There has probably never been a greater opportunity for totally open source and self-hosted AI.”
" AI influencers are getting filthy rich… let’s build one"
