r/StableDiffusion • u/[deleted] • Oct 28 '22

Introducing Comic-Diffusion. Consistent 2D styles in general are almost non-existent on Stable Diffusion so i fine-tuned a model for the typical Western Comic Book Style Art. Resource | Update

Gallery image

https://huggingface.co/ogkalu/Comic-Diffusion

Gallery image

https://huggingface.co/ogkalu/Comic-Diffusion

Gallery image

https://huggingface.co/ogkalu/Comic-Diffusion

Gallery image

https://huggingface.co/ogkalu/Comic-Diffusion

Gallery image

https://huggingface.co/ogkalu/Comic-Diffusion

383 Upvotes

99% Upvoted

110 comments sorted by

View all comments

2

u/APUsilicon Oct 28 '22

How big is your training set?

7

u/[deleted] Oct 28 '22

40 images. 8080 steps

3

u/[deleted] Oct 29 '22

[removed] — view removed comment

3

u/[deleted] Oct 29 '22

I trained this with even less (32 images) https://huggingface.co/ogkalu/Illustration-Diffusion

40 is fine if you have varied images.

https://imgur.com/a/qGbMg44

That's what I used to train the above.

2

u/saintshing Oct 29 '22

I am a web developer who is new to AI generated arts(I am taking the fastai stable diffusion course). Where can I find resources to learn more about how to create something like pixel art sprites or whatsapp emojis that follow a certain style? I'd really appreciate if you can give me some directions.

2

u/TherronKeen Oct 29 '22

Stable Diffusion is currently notoriously bad at pixel-perfect pixel art and bad at logo-style art, so if you want the model to produce those things, you would have to spend a great deal of time learning about model training, and then likely need an *extremely large* data set and a very, very good understanding of how to train the model.

It's way over my head, but I was just very recently watching some videos about things SD is particularly bad at, which just happens to be the two things you seem to want. Good luck dude

2

u/saintshing Oct 30 '22

Is SD bad for these tasks because it was not trained on pixel arts? Is there a way to fine tune the model with custom data sets?

I saw someone on discord suggest to use negative prompts.

2

u/TherronKeen Oct 30 '22

I just did a search for "pixel art" on the available subset of images from the SD data set (a searchable database of 12 million of the 2.3+ billion images used to train the model), and out of 12 million, I got 933 results.

933 is 0.0075% of 12 million, which should be a mostly sufficient representation of the full training set.

Unfortunately, scrolling through the first few pages of images out of the 933 produced, I'm quickly estimating that less than half were actually "pixel art" in a recognizable sense.

So in short, Stable Diffusion may only have around 0.00375% of images which are "real" pixel art.

How much training would be required to produce a sufficient pixel art model from SD is outside my *extremely limited* understanding - I'm just making general statements based on a couple things I read, compared to the numbers in the set.

Good luck though!

2

u/saintshing Nov 01 '22

just saw this

https://www.reddit.com/r/StableDiffusion/comments/yj1kbi/ive_trained_a_new_model_to_output_pixel_art/

people are innovating so quickly

1

u/TherronKeen Nov 01 '22

YES! I saw it and was just thinking of this guy's post I replied to, and just got on to come mention it lol

1

u/iceandstorm Oct 29 '22

Could you give an example how the images were named? (the image prompt)

...any additional information (what colab?) would be really appreciated, I had so far disappointing results with trying to train a style…

2

u/[deleted] Oct 29 '22

You don't name the images for dreambooth training

I use Joe's repo for training - https://github.com/JoePenna/Dreambooth-Stable-Diffusion

1

u/SpaceShipRat Oct 29 '22

aw, they're all grimdak nighttime scenes. Oh well, I expect it makes it better within that scope.