It’s no secret that large models, such as DALL-E 2 and Imagen, trained on large numbers of documents and images retrieved from the Internet, absorb both the worst and the best aspects of that data. OpenAI and Google explicitly acknowledge this.
Scroll down the Image website—past the dragon fruit wearing a karate belt and the little cactus wearing a hat and sunglasses — to the social impact section and you get this: “While a subset of our training data was filtered to remove noise and unwanted content, such as pornographic imagery and poisonous language we have also used [the] LAION-400M dataset known to contain a wide variety of inappropriate content, including pornographic images, racist comments, and harmful social stereotypes. Imagen relies on text encoders trained on web-scale uncurated data, thus inheriting the social biases and limitations of large language models. As such, there is a risk that Imagen has encoded harmful stereotypes and representations, leading us to our decision not to release Imagen for public use without further safeguards.”
It’s the same kind of acknowledgment that OpenAI made when it unveiled GPT-3 in 2019, “Internet-trained models have Internet-scale biases.” And as Mike Cook, who researches AI creativity at Queen Mary University of London, has pointed out, it’s in the ethical statements that accompanied Google’s great language model PaLM and OpenAI’s DALL-E 2. Basically, these companies know that their models are capable of producing terrible content, and they have no idea how to fix that.