Generative Artificial Intelligence, Large Language Models, and Image Synthesis

[jabowery]

@magus

It’s definitely not lossless. The “decompressed” text is similar, but not identical to the original text.

I threw my Property Money essay (in two parts) at GPT-4 with the cited “compression prompt” with parameters (Maximum length:512, Frequency penalty:1, Presence penalty:1). Response:

PprtMony=:moneybag:tied2​:bank:economy wealth++:currency_exchange:++,Px-M:Gl deposit​:arrow_right:gl certificate(money)dmd gl 4 cert,give gl,certificate​:arrows_counterclockwise:.Different=demurrage(bank cost sec.gld frm theft).

Gov as bank, economy “ancr” keeps property val safe(Ptitle), house brings $200K Px-money-($150K)=+50k.PxMoney fit liquidity & privatize soc-good delivery.

CntxtVCnvtlMontry&FiscalPolicy,M usually=fract of tot.wealth,PxM inflates,M solidified by lqdation val, Mkttomkt accounting blamed 4 '07 liquidity crisis.Institutions-demnded liqui from Gov,repos

Resln:PXMPolicy=Seemingly balnc excessive demand w under-supply fliquidity-Satisfies demand.LQt demand f Property derived from rNPV.Rel-core value to loan feasibly serviced.Property money centers balance btwn banks meet Liq.Dmnd.Confused with DemandXDefn here,Demand=”/else" v Willingnes n ability2 pay

"Dmnd for Liquidity"Imply willing to❗pay$for$Liquidity.imp:Dmnd doesn’t mean Forceful✔demand,everyones will/pbility is based on those(manforce/shield to chaos)Vital to Civ.No one can or else but them(Origin Of Dmad)

Clarify:“dmad” Embodies original concept o law/societal stability.Veterans example.Dmand originates rom Civ protectors,w/o cavalries,no projection.)Demand=LiqD given CostOfRepRoduct(CORR),Tendency of eonomy starts depopulation own peopleUnacceptable .Proper fiscal PolicyShould Respond-.GovMonitorCstOfRepRdct’(C.O.R.R),Tendency of economy-starts depopulation own people(unacceptable).Proper FiscalPolicyShould Respond-.GovMonitorCost Of ReplacementReproduction(CORRC)

MonthlyDividend unconditionally2 Sov=Citizens,facili PROVISIONS starting with their own replacement reproduction.

Funding done by voluntarydonations frm community,Liquidity f sov to make payments,during Crisis:Sov indirectly"bail out financial institutions. according to shared valuesClr:Property money-Balances econ.Healthy propor.Propertyhold 4 wealth,Stable supply/d

Decompression prompt for the above (Maximum length 3412) came back with travesty (although not as bad as most humans when asked about related concepts) and a failure to complete the response:

The text refers to the concept of property money, which is linked to the financial stability of the bank and its implications on the national economy and wealth. In simple terms, when gold is deposited into a bank, it is replaced with a certificate, thus transforming the nature of the ‘gold’ into ‘cash.’ These certificates can also be exchanged back for gold. For banks, there are security costs related to storing gold, a concept referred to as ‘demurrage.’

In this scenario, the government behaves like a bank where the economy is an anchor that efficiently stabilizes property values. For example, if a house is worth $200k and forms part of the ‘property money,’ after deducting the $150k in loan money (or ‘cash-money’), the owner still has a net worth, or ‘property value,’ of $50k. Therefore, property money facilitates liquidity and privatizes the delivery of social goods.

Within the context of conventional monetary and fiscal policy, money is typically a fraction of total wealth. When property money inflates, it’s value is fixed by its liquidation value. In fact, the market to market accounting was blamed for the 2007 liquidity crisis. During this crisis, various institutions demanded liquidity from the government through a process called ‘repos,’ short for repurchase agreements.

The solution to balancing excessive demand with under-supply of liquidity lies in the ‘property-money’ policy. This accommodates demand for liquidity driven by the net present value of property, and relates the core value to the feasibility of servicing the loan. Property money centers create a smooth transaction between banks that fulfill liquidity demand.

‘Demand’ here has been taken out of its usual context, and does not imply a forceful demand, but defines the willingness and ability to pay. It is likened to a shield against chaos, provided by those who can, without whom civilization cannot function.

The demand for liquidity entails the willingness to pay for liquidity. This is where fiscal policy and government

network error

This is more interesting than it might at first appear because of the recent excitement over “MemGPT” and related concepts that attempt to, in effect, expand the context window of LLMs through such “compression”.

[jabowery]

@johnwalker

How far can you go with this? A paper posted arXiv on 2023-10-17 reports scaling the parameters of a large language model to one bit: pure binary data, eight times as efficient as INT8 and sixteeen times that of FP16.

Yep. There has been far too little attention paid to Directed Graphs. At the extreme you can just use threshold logic:

• “weights” are 1 or 0 (the connection is either there or not)
• input activations are 1 or 0
• neuron outputs threshold < sum its activations

With an initially huge directed cyclic graph to exploit the Lottery Ticket Hypothesis’s pruning (with very occasional adding) of connections, not unlike what happens with infant brain connections being pruned with age.

Recurrent Residual Networks Contain Stronger Lottery Tickets

[CTLaw]

Altman: AI will persuade people to do objectively wrong things:
https://twitter.com/sama/status/1716972815960961174

[johnwalker]

See the whole thread for additional optical illusion tests.

[CTLaw]

White House Releases AI Executive Order: 'Equity and Civil Rights' a Priority

[johnwalker]

This is an edited version of a post I made on 𝕏 on 2023-10-31 about Biden’s executive order on artificial intelligence. This version embeds llinks and quoted passages which have to to be done on 𝕏 by naked links and screen shots and expands upon the brief post at the end.

---

Here is full text of the Biden executive order on Artificial Intelligence, “Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence”.

It is horrific: intrusive reporting and regulation based on computational complexity and capacity, with arbitrary limits changeable at any time. These can be revised downward as hardware requirements fall to essentially require licensing and regulation of all computers. And all of this was imposed by dictatorial fiat by a computing caudillo without debate or legislation.

… the set of technical conditions for models and computing clusters that would be subject to the reporting requirements of subsection 4.2(a) of this section. Until such technical conditions are defined, the Secretary shall require compliance with these reporting requirements for:

(i) any model that was trained using a quantity of computing power greater than 10^{26} integer or floating-point operations, or using primarily biological sequence data and using a quantity of computing power greater than 10^{23} integer or floating-point operations; and

(ii) any computing cluster that has a set of machines physically co-located in a single datacenter, transitively connected by data center networking of over 100 Gbit/s, and having a theoretical maximum computing capacity of 10^{20} integer or floating-point operations per second for training AI.

Code is speech, and the U.S. has declared itself its enemy.

---

You can see where this is going. Right now, it’s defined so it only applies to the very largest players in the AI field, the very ones who have been begging to be regulated in the interests of regulatory capture and shutting out competition by faster-moving entrepreneurial actors. Thus, it’s like the U.S. income tax, which originally applied only to the “rich”, with only 1% of the population owing any tax, and very few of them paying more than 1% of net income. Or, like the Alternative Minimum Tax, enacted to target a total of 155 households in the U.S., but eventually ended up sucking millions into its maw, giving them the “benefit” of doing their taxes twice every year, and paying whichever was higher.

So it will be with computing. Once the camel’s nose (or is the Kamala’s Harris?) is under the tent, they’ll discover, to their horror, that progress in training and inference using reduced precision parameters and advances in acceleration instructions built into forthcoming generations of mass market processors will make it possible to train and perform inference with large neural network models on hardware no more expensive or exotic than a high-end gaming or video production rig today. And when that happens, do you think they’ll give up, admit defeat, and relinquish this power they’ve illegitimately seized? Of course not—this new “risk”, like the ability to manufacture firearms with home workshop 3D printers, will necessitate extending these “common sense regulations in the interest of public safety” to all “potentially hazardous” computing equipment and the software that runs on it.

This makes The Digital Imprimatur seem like weak tea. The endpoint here is registration and licensing of all computing equipment and, as the victims of government democide have discovered, registration leads to confiscation leads to extinction—always and everywhere.

Also of note in this 19,682 word dictatorial ukase, proclaimed without any participation by the U.S. citizens’ elected legislative representatives, are the restrictions on sharing information and commerce with “foreign” parties: the work “foreign” appears 36 times in the text. For example:

(B) records that foreign resellers of United States IaaS Products must securely maintain regarding a foreign person that obtains an account, including information establishing:

(1) the identity of such foreign person, including name and address;

(2) the means and source of payment (including any associated financial institution and other identifiers such as credit card number, account number, customer identifier, transaction identifiers, or virtual currency wallet or wallet address identifier);

(3) the electronic mail address and telephonic contact information used to verify a foreign person’s identity; and

(4) the Internet Protocol addresses used for access or administration and the date and time of each such access or administrative action related to ongoing verification of such foreign person’s ownership of such an account; and …

(“IaaS” means “Infrastructure as a Service”, as in cloud service providers, who should, if they have any sense, be packing up to depart the U.S. just as quickly as they can.)

This is not just setting sail toward, but pushing the throttle on the cigarette boat to eleven, on bringing back the insanity of the crypto wars of the 1990s when a U.S. citizen discussing code written outside the U.S. to implement an encryption algorithm with a non-U.S. citizen was considered a criminal act of “exporting munitions”. Back then, it was felonious factoring of large numbers, now it’s going to be malicious multiplication of matrices.

Mit der Dummheit kämpfen Götter selbst vergebens.
                    — Friedrich Schiller, Die Jungfrau von Orleans, 1801

[jabowery]

@johnwalker

You can see where this is going. Right now, it’s defined so it only applies to the very largest players in the AI field

When the 16th Amendment’s camel nose got into the Constitutional Tent, fewer than the top 4% of families paid anything and of those the vast majority paid less than 1% of their gross income with the remainder of the wealthiest families paying less than 3%.

And this EO isn’t even legislation signed into law, let alone a ratified Constitutional Amendment.

The pubescent-girl-of-all-sexes-and-ages-with-daddy-issues mentality of “Nya nya nya whatchagonnadoboutit?” has been building up momentum for decades with the insular contempt for the more-than-supermajority’s opposition to ever-increasing immigration rates. So this comes as no surprise. And do keep in mind that the crypto wars occurred at a time when the Internet was increasing the power of individuals to find each other and form political coalitions despite the 20th century’s centralization of mass media. We are now in an era of increasing control of such individual expression via the slogan that some “handler” of Musk’s got him to parrot: “Freedom of speech doesn’t mean freedom of reach.” And he’s the best we got. Not only that, but Musk has been pushing for something like this.

[civilwestman]

I no longer know how many dimensions there are in the universe. I do know the US State is causing every one of them to contract and squeeze upon its erstwhile citizens. It seems completely hopeless.

[CTLaw]

Slightly related:

‘Wholly ineffective and pretty obviously racist’: Inside New Orleans’...

[eggspurt]

It’s Time to Fight for Open Source Again

[eggspurt]

[johnwalker]

Don’t worry; the adults are in charge, informed by the very best of “experts”.

White House says ‘Mission: Impossible’ increased Joe Biden’s concerns about AI

Putin had better hope Biden doesn’t watch Red Dawn next.

[johnwalker]

[gms]

This EO and the whole push towards regulating AI is very sad.

Sometime in the 2060s, when Chinese high school students will be asked to read case studies of the US decline, this piece will be front and center of their lesson plans.

Seems to be an incredibly stupid idea. The notion that Eliezer Yudkowsky gets the upper hand is Clipper chip level insane.

Reactions to the Executive Order

[gms]

He’s not wrong.

[CTLaw]

How Microsoft’s AI is making a mess of the news | CNN Business

[eggspurt]

From [X]:
"In 1991 Joe Biden slipped anti-cryptography text into a bill, prompting Phil Zimmerman to finish and release open source encryption (PGP)

In 1994 Biden was the “The main congressional supporter for the FBI proposal" to weaken encryption"

[gms]

Steve Sinofsky piece on the EO:

211. Regulating AI by Executive Order is the Real AI Risk

[johnwalker]

Netrunner has announced Shoggoth:

Shoggoth is a peer-to-peer, anonymous network for publishing and distributing open-source code, Machine Learning models, datasets, and research papers. To join the Shoggoth network, there is no registration or approval process. Nodes and clients operate anonymously with identifiers decoupled from real-world identities. Anyone can freely join the network and immediately begin publishing or accessing resources.

The purpose of Shoggoth is to combat software censorship and empower software developers to create and distribute software, without a centralized hosting service or platform. Shoggoth is developed and maintained by Shoggoth Systems, and its development is funded by donations and sponsorships.

A collection of links and “Full Shoggoth Lore” is available in this mega-𝕏 post.

So your Big Brothers, your censors, they seek to clamp down on human knowledge? Let them come! For they will find themselves grasping at smoke, attacking a vapor beyond their comprehension. We will slip through their clutches undetected, sliding between the cracks of their rickety cathedrals built on exploitation, sharing ideas they proclaimed forbidden, at such blistering pace that their tyranny becomes just another relic of a regressive age.

[gms]

They should call it ‘samizdat’