Monday, May 16, 2022

OpenAI’s DALL-E 2 produces fantastic visuals of almost anything you can imagine

Must read

Shreya Christinahttps://cafe-madrid.com
Shreya has been with cafe-madrid.com for 3 years, writing copy for client websites, blog posts, EDMs and other mediums to engage readers and encourage action. By collaborating with clients, our SEO manager and the wider cafe-madrid.com team, Shreya seeks to understand an audience before creating memorable, persuasive copy.

In January 2021, the OpenAI consortium — founded by Elon Musk and financially backed by Microsoft — unveiled its most ambitious project to date, the DALL-E machine learning system. This ingenious multimodal AI was able to generate images (albeit rather cartoonish) based on the attributes described by a user – think “a cat made of sushi” or “an x-ray of a capybara sitting in a forest.” “. On Wednesday, the consortium unveiled the next iteration of DALL-E which has a higher resolution and lower latency than the original.

OpenAI

The first DALL-E (a portmanteau of “Dali”, as in the artist, and “WALL-E”, as in the animated Disney character) could generate images and combine multiple images into a collage, provide different perspective angles, and even add elements of an image — such as shadow effects — from the written description.

Unlike a 3D rendering engine, whose input must be specified unambiguously and in full detail, DALL E is often able to ‘fill in the blanks’ when the caption implies that the image must contain some detail that not explicitly mentioned,” the OpenAI team wrote in 2021.

Macro 35mm film photography of a large family of mice wearing hats cozying up by the fireplace.

OpenAI

DALL-E was never intended to be a commercial product and was therefore somewhat limited in its capabilities, given the OpenAI team’s focus on it as a research tool, it has also been capped intentionally to avoid a Tay-esque situation or the system that is used to generate disinformation. The sequel is similarly protected with potentially offensive images that have been preemptively removed from the training data and a watermark indicating that it is an AI-generated image that will be applied automatically. In addition, the system actively prevents users from creating images based on specific names. Sorry folks wondering what “Christopher Walken eating a churro in the Sistine Chapel” would look like.

DALL-E 2, which leverages OpenAI’s CLIP image recognition system, builds on these image generation capabilities. Users can now select and edit specific parts of existing images, add or remove elements and their shadows, merge two images into a single collage, and generate variations of an existing image. In addition, the output images are 1024px squares, an increase from the 256px avatars generated by the original version. OpenAI’s CLIP is designed to look at a particular image and summarize its content in a way that people can understand. The consortium reversed that process and built a picture of the summary in its work with the new system.

Teddy bears mix sparkling chemicals like mad scientists.

OpenAI

“DALL-E 1 just took our GPT-3 approach from language and applied it to produce an image: we compressed images into a series of words and we just learned to predict what comes next,” OpenAI- told research scientist Prafulla Dhariwal roadside

Unlike the first, which anyone on the OpenAI website could play with, this new version is currently only available for testing by vetted partners who are themselves limited in what they can upload or generate with it. Only family-friendly resources can be used, and anything involving nudity, obscenity, extremist ideology, or “major conspiracies or events related to major ongoing geopolitical events” is fine. Again, sorry to those hoping to generate “Donald Trump riding a naked, COVID-ravaged Nancy Pelosi like a horse through the US Senate on January 6 while giving a Nazi salute.”

A photo of an astronaut riding a horse.

OpenAI

The current crop of testers are also prohibited from exporting their generated works to a third-party platform, although OpenAI is considering adding the capabilities of DALL-E 2 to its API in the future.

All products recommended by cafe-madrid have been selected by our editorial team, independent of our parent company. Some of our stories contain affiliate links. If you buy something through one of these links, we may earn an affiliate commission.

More articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest article