Are you a fan of comics, but still haven't gotten around to creating your own masterpiece? The new open source app Story Diffusion can get you started.
One of the most exciting features of Story Diffusion is that it will soon add the ability to create videos as well! The sample videos on the project page are already jaw-dropping, so we can't wait to see what the future holds.
The web UI of Story Diffusion is easy to navigate. You simply select a model, optionally upload an image of the main character, and then describe your main character and their story. The sample stories at the bottom of the page provide a good starting point, with each panel described in one line. You can add custom captions after the hashtag symbol at the end of a line, and let the app know you don't want any characters in the frame by using the [NC] tag.
Between the character and story description, you get to select the style. My favorites are cinematic and comic book style. If the character doesn't have consistent clothing in all panels, it helps to add more details to the description, as well as negative prompts for things you don't want, such as t-shirt print or belt. And if only a few random objects are off, you can tweak the seed number and try again!
When you're done generating, your work will be saved in the output folder. You can also add a starting image to influence the character's face by inserting the word "img" to the character description.
If you run out of memory, you can reduce the number of steps, which has a high default value of 50. Using a starting image usually requires more VRAM, so I switched the step count to 45 for that. Here's how my test with landing myself on an alien planet turned out. I tested this on Kaggle, which has a generous 16 gigabytes of VRAM.
To give Story Diffusion a try yourself, simply follow the link to the notebook and perform the usual steps: make your copy, choose the environment settings, then kick off installation, and start up the UI. The file browser is also there to help you fetch your comics and clean up the output folder.
In conclusion, Story Diffusion is an exciting new tool for comic book creators. With its easy-to-use web UI, prompts for detailed character and story descriptions, and the ability to add custom captions and starting images, you can bring your comic book ideas to life in no time.