Exploiting the flaws of AI art to create movie posters for my recurring dreams
I’ve always found the text I see in my dreams to be interesting - it often looks like English, but isn’t. When I first started playing with Stable Diffusion, I noticed that the text it generates is very similar:
A local art gallery was accepting pieces for an exhibition of visual art based upon dream imagery. My dreams often play out like movies. This gave me an idea: movie posters for three of my favorite types of dreams:
We want to create three movie posters, with three unique styles. The movie posters should be in a portrait orientation, with imagery overlaid with credits and a title.
This presents us with a challenging situation for Stable Diffusion 1.5:
It is extremely challenging for Stable Diffusion 1.5 to generate accurate text and imagery in a single image. If we were required to generate the movie poster text and imagery at the same time (i.e. with a single image generation prompt), our odds of obtaining a successful image are quite low. To avoid this problem we can take a compositing approach:
We begin by deciding what style of movie posters we want to create. Based on the differing themes, each dream will be best represented by a different movie poster genre:
With these styles in mind, the compositions come naturally:
We now craft the prompts which will produce the desired imagery. We can think of each prompt as the combination of two distinct prompts: one portion for the content, the other portion for the visual style.
After considerable experimentation, we end up with the following prompts:
black muscle car leaving a trail of fire, driving into the camera, front view, in an empty desert, dark black smoke in the sky, surrounded by flames, watercolor painting, action movie poster
asteroid hitting earth, explosion, debris, apocalypse, destruction, aerial view from above, oil painting, disaster movie poster
a boy basejumping without a parachute with arms stretched wide, wearing sneakers, flying above the surface of the earth, top down view of the back of his head and body, white altocumulus clouds above the surface of the earth visible underneath, bright day, oil painting, fantasy movie poster
From the thousands of generated images we choose our favorites:
We are now ready to generate the text for our movie posters.
Since we no longer need to generate specific poster imagery, we can instead generate images purely for their text content. This means we can tailor our prompts to increase the odds that appropriate text (in style and content) is generated.
After some experimentation, we end up with the following prompts:
grindhouse muscle car movie poster text credits written by directed by
classic asteroid destruction disaster movie poster with text credits at bottom
fly movie poster, cartoon, zany, wacky, large font
From the thousands of generated images we choose the most appropriate movie titles and credits text:
Since we will be printing our movie posters for display in an art gallery, we will target the standard “mini sheet / insert” movie poster size of 11" x 17". Using the standard 300dpi printing resolution, this requires our movie posters to have pixel dimensions of 3300px x 5100px. Stable Diffusion 1.5 is not capable of creating images at this resolution, so we will need to upscale our imagery before compositing.
After some experimentation, we find that using R-ESRGAN 4x+ to upscale images, and ESRGAN 4x+ to upscale text works best.
We are now ready to combine these pieces into our final movie posters.
In Photoshop, we isolate the text from its background, refine the text outlines and colors, and play with various compositions. Once we are happy, we apply Gaussian Noise to the entire image which compensates for any smoothing which occurred during upscaling. This also makes the text and background imagery blend more naturally.
We now have our final composites:
The posters are printed and framed:
Then are mounted for display at the art gallery:
Those that liked it, liked it a lot:
