More Amazing Results from DALL-E 2 and an Open Source Clone

Errr…eepy-creepy?

2 Likes

It’s having some trouble with “horns” and with “cow.” Well, it’s young yet, I guess.

1 Like

image

5 Likes

image

3 Likes

image

4 Likes

image

4 Likes

Here are the creepy results of the made-up word “Crungus”, courtesy of @Brainmage:

What is remarkable is that all of the images depict the same character even though “Crungus” should not exist at all.

Similar results have been observed using DALL-E 2. For example, the following images are a result of the meaningless string “Apoploe vesrreaitais”:

These findings have led researchers to hypothesize that DALLE-2 might possess a “hidden vocabulary”.

3 Likes

Hi Magus,

In the case of the “Apoploe,” surely the program went with “hoopoe.” Those birds look like hoopoes.

I wonder: Does the software gather feedback and learn from it, right there in the software lab, so to speak? And, does it learn from serial requests from end-users? I suppose it could infer feedback from the details of serial requests.

3 Likes

It looks like the software knows English and also knows how to play with English by making associations.

“Crungus” was given as a proper noun, so it made images of a character, an entity. Crungus is grungy. Crungus prominently features those teeth, which is why all the images are fairly close-up. Crungus likes to crunch us.

I think DALL-E 2 reads the papers - has been at least since the Iraq Surge.

1 Like

I don’t think so. Models like GPT-3 and DALL-E are called “pre-trained” (GPT stands for “Generative Pre-trained Transformer”), and the weights in the neural networks are set by the training inputs supplied when the model was built (in the case of GPT-3, 410 billion encoded tokens, which it distilled down into 175 billion machine learning parameters (weights). But, as I understand it, there is no feedback from results after the training stage. You would have to change the training set and re-train the model to modify its response for given prompts.

One of the things that makes models like this practical to deploy is that while the training stage is enormously costly in computing power, just running a prompt through the trained model to produce output is relatively inexpensive. This is why training Tesla’s self-driving requires massive supercomputer hardware but running it in real time can be done on a single board computer in the car.

3 Likes

Well, it happened. Today Fourmilab was approved for access to OpenAI’s DALL-E 2. Here my first try, from the prompt “plastic robot ants in computer room in comic book cover style”.

6 Likes

Utterly, terrifyingly, brilliant! (BrilliANT?)

2 Likes

“Babbage analytical engine as drawn by Albrecht Durer, from the British Museum”
Babbage_analytical_engine_as_drawn_by_Albrecht_Durer,_from_the_British_Museum

Large version of top right:

2 Likes

“programmer discovers alien message in human DNA, 1950s pulp novel cover”

programmer_discovers_alien_message_in_human_DNA,_1950s_pulp_novel_coverScreenshot_2022-07-13_at_18-11-52_DALL·E

4 Likes

My first reaction was this looks more like Soviet Realism style and then I noticed the writing on the last panel resembles Cyrillic ?

I recall reading somewhere that some researchers speculate DALL-E “invented” its own language. Here is a paper with some examples. What’s weird is the authors names look Greek (no joke) and the “caption” seems to be written in modern Greek.

3 Likes

“corgi with Pegasus wings flying over small island, National Geographic color photo”

corgi_with_Pegasus_wings_flying_over_small_island,_National_Geographic_color_photo

6 Likes

Everything’s better with corgis!

3 Likes

“small New England town disappears down to topsoil, as painted by Winslow Homer, in the National Art Gallery”

This is an illustration for my story “Free Electrons”.

small_New_England_town_disappears_down_to_topsoil,_as_painted_by_Winslow_Homer,_in_the_National_Art_Gallery

My favourite:

3 Likes

“astronauts in spacesuits encounter giant tardigrade in cave, science fiction pulp magazine color cover”

astronauts_in_spacesuits_encounter_giant_tardigrade_in_cave,_science_fiction_pulp_magazine_color_cover

Here is a blow-up of the bottom right, in which the tardigrade appears to also be wearing a spacesuit that resembles Bibendum.

3 Likes

“oil painting of an orange cat in the style of van Gogh, around 1887, from the Van Gogh Museum, Amsterdam”

oil_painting_of_an_orange_cat_in_the_style_of_van_Gogh,_around_1887,_from_the_Van_Gogh_Museum,_Amsterdam

This experiment was prompted by a comment I made on another Web site on 2015-05-12 during a discussion of the potential of artificial intelligence.

Here is an example [from MIT Technology Review, “The Machine Vision Algorithm Beating Art Historians at Their Own Game”] of what happens when you put together big data, artificial neural networks, and exponentially growing computing power.

Imagine: it’s 2022.

“Siri, please paint a picture of my cat in the style of Van Gogh, around 1887.”

“Which cat: Meepo or Rataclysm?”

“Meepo.”

“Just a moment. All right, here you go. Would you like me to post this to all of your friends?”

Roll over Ray Kurzweil, I even got the year right.

5 Likes