Codeformer and Reddit Bot for the Stable Horde

Db0

I haven’t been able to improve the Stable Horde a lot lately. I was planning to do a lot of work during the week leading to Christmas, but unfortunately the universe had another idea and not only infected me with the nastiest cold I’ve had for decades, but my whole family as well, including the visiting Grandma!

So instead of adding necessary new features, I’ve been instead flattened at bed, trying to muster enough concentration to do some basic updates and answer questions.

Nevertheless, there’s a few improvements added, mostly through the work of some members of the community.

First is the addition of the CodeFormer face-fixing post-processor which seems massively better than the GFPGAN model. Now all clients can request that an image be bassed through CodeFormer for an immediately improvement in faces. Soon I plan to allow this to run in isolation as well

The other new thing is improvements on the workers themselves, allowing them to pickup and perform jobs more efficiently.

The other big news I have is that wrote and unleashed the first Reddit bot for stable Diffusion. That was initially created as an entry for the Ben’s Bites Hackathon since I couldn’t submit the Stable Horde itself (I didn’t win btw), but it was quite an eventful release. My initial release got caught by the automated reddit anti-spam filter, shadow-banning my account and banning my subreddit. Then I refactored the bot to use my own R2 CDN and released it with a new account while asking for a reddit review on my original account. Fortunately my bot account and subreddit got unbanned and I finally released it a third time properly, and it’s been up ever since!

The way the bot is created you can request images from it all over reddit, and it will post the images in its own subreddit for everyone to see and vote on.

There’s also been a lot of new models and styles onboarded, which are also used by my reddit and mastodon bots.

The next plan now is to allow image interrogation on the stable horde, as well as direct image post-processing (without stable diffusiion), so as to allow even people with low-powered machines to be able to contribute for kudos.

Automated Rewards and Tiers

Db0

Hey everyone, I have finally finished integrating Patreon with the Stable Horde. This will allow me to maintain everyone rewards automatically, and adjust them according to your status on patreon as well.

This means for example, if you upgrade your subscription from one tier to the next, you will also receive the adjusted reward on the next month’s cycle without me having to find the time to adjust things

Also there is a new adjustment, where now patrons can utilize the horde even via VPN. The VPN restriction is in place to prevent people sending or requesting unethical or illegal content, so by trying your bank account to your username, you provide a way to block the access if it’s abused this way. This is allowed from the lowest tier, so if you have any reason to use VPN to access the stable horde, feel free to use this.

Finally, as an extra thank you, I have also added a small “candy” even for the lowest tiers!

For all this to work, I need to know your Stable Horde ID, and specifically the #number# next to your username! So please send it to me asap if you haven’t already! If you ever change accounts on the horde, you’ll also need to send me the new ID.

When you provide your Stable Horde id, you will also be listed on the sponsors page . If you would prefer a different alias to be shown on the sponsors, please let me know as well.

And we’re back!

Db0

dbzer0.com has been migrated to a new infrastructure! I have opted to not do a dumb copy of the whole wordpress install folder as my wordpress site has been live for something like 15 continuous years now 😀

So I opted for just set up from scratch, purely with ansible this time, and then import/exporting the content, which looks like it broke all the embedded images. I copied my uploads folder back manually but they don’t seem to appear (if anyone knows how to fix it, lemme know)

I also selected a new theme to keep things fresh. Went for something minimalist as I don’t have that much time to update this blog as often anymore.

Now I can finally shut down my ancient VPS.

NSFW filter is back!

Db0

With the new nataili bridge we had lost the existing NSFW censorship model which was responsible for ensuring SFW workers who wanted to, did not generate NSFW images. It was also an important part of the censorlist implementation, where even NSFW workers could choose specific words which would activate the censor filter as they are concepts that never belong into pornographic prompts.

Today I’m happy the announce that with the help of hlky, the NSFW filter is back, so there’s no reason for anyone to stick to the old worker code anymore!

To add to that, I’ve also added an extra protection from what I call ‘Corrupt Prompts’ in the horde. That is, prompts which have a chance to generate completely unethical AI images. Now when the prompt detects the specific generation of words, the owner account and IP will be flagged and the request logged. 

Nothing is perfect but I plan to keep tweaking the detection for corrupt prompts to ensure I prevent as much as possible from reaching the workers. I also hope that this extra logging will give pause to people who might think to use the horde for unethical purposes.

Unfortunately the safety_checker model responsible for the NSFW detection, takes a bit more of your VRAM to work, so if you find that after downloading the latest version of the bridge, your workers starts crashing, consider reducing your horde power, or unloading a secondary model, such as one of the dreambooths.

The Stable Horde: AI image generation for everyone through mutual aid

Db0

After completing the KoboldAI Horde, and onboarding into the KoboldAI client, I felt that there is a really big opening for doing a similar thing using the open sourced AI image generating model, Stable Diffusion. I already have the code for setting up a crowdsourcing cluster, so it shouldn’t take too much refactoring to make the same underlying code work with Stable Diffusion.

The first thing I had to do was figure out what is going to run on the workers. For this, I decided to reuse the stable-diffusion-webui fork by simply adding my bridge code on top of it (as it doesn’t provide a REST API like KoboldAI). Once I had a valid bridge, it was time to fork the Horde.

And thus, the Stable Horde was born!

It follows the same approach, where workers running some version of Stable Diffusion constantly poll for new generations to complete and then send it back to the horde to hand it over to their final destination. For now the stable horde is only handling fairly basic text2image generations, but since it’s based on the webui, I can tap into the features that added upstrean much easier, without having to develop them myself.

The code started as a fork of the Stable Horde, but has by now become my primary repo. In fact, with the addition of the second version of the REST API, I have decided to merge both Hordes into a single repository in order to better share code updates (because copying code from one repo to the other was driving me nuts!). This is coming soon, and it means that the Stable horde will always remain in parity with the KoboldAI horde from now on.

While there are other free image generation tools out there, I believe none is doing anything like what I am attempting. Most of these are based on providing free Stable Diffusion by eating the costs themselves, but with an undefined business plan. And when I see that, my suspicions are already raised, as a free service like that, typically means you’re the product! It also doesn’t help at all that they are not sharing the code behind them.

Now you might say, “But db0, your service is also free, how come the same criticism doesn’t apply to you?”. Which is a great question. The answer is that the reason the Stable Horde is free is because it’s volunteer based. That means, at the end of the day, someone is indeed paying for electricity (that is, myself primarily atm), but the point is that it is self-managing through people’s innate drive for mutual-aid.

That means that if I get a jump in popularity, which in turn exceeds the Horde’s current image generation capacity (and therefore slowing things down too much), the belief is that there will be enough people annoyed by the speed, that they will join their own power to the horde to benefit themselves with higher priority, but also everyone else.

And yes, there is always some amount of “small print”. While the Stable Horde is built on anarchistic principles of mutual aid and direct action, the fact is that we do not control the underlying workers. Therefore it is theoretically possible for people to act maliciously on the worker side, which is why I always warn people who will use the Horde that I cannot guarantee that nobody will see your prompts. So act accordingly.

Nevertheless, one of the things I’m offering is something that I just haven’t seen anyone else do for image gen, and that is a fully functioning RESTful API. The purpose of this it to further enable image generation for everyone in new and exciting ways that enable tools to use this capability without bankrupting their owners for a side hobby whose demand suddenly spiked. Already people have started creating some interesting tools, such as a weather app which uses the Stable Horde to generate a dynamic image representing the weather, based on environmental conditions.

On my end, I am interested in helping game developers figure out ways to implement AI into their games. For this purpose I have already released a Godot Add-On which allows you to request AI image generation during a game’s runtime. I have further used this add-on to create my own Stable Diffusion GUI client that can run on any device, without the need for a complicated install procedure, or a GPU.

All of this is just scratching the potential of what can be achieved by allowing automation to connect directly to Stable Diffusion (or text generation), and I’m excited to see what people will come up with in the future!

The Kudos-based economy for the KoboldAI Horde

Db0

Work continues on the KoboldAI Horde apace, and in the past week I’ve added oauth auhentication but kept the anonymous access live as well. Let me tell you, figuring out how to use the flask.dance module for discord was a complete PITA.

But I got it done, and even added github authentication (and hopefully google soon, if they stop asking for silly things)

The authentication was added to allow me to build the next step, which is to be able to balance usage from the community.

The thing I expect to happen is that as the service becomes more popular, we might have way more clients trying to use the horde than we have actual contributing servers, and therefore we would run into performance bottlenecks. This could become unworkable especially if things like bots start using this service anonymously etc.

To provide some automatic balance, I have now introduced the concept of Kudos into the mix. A Kudos is a reward gained when someone contributes to the well-being horde in some way. There are two built-in methods to gain Kudos at the moment:

  1. Any time your KoboldAI server generates text for someone, you receive tokens based on the amount of text generated and the power of your model.
  2. KoboldAI servers get “uptime” rewards every 10 minutes they stay online and check-in looking for more requests. This incentivizes people leaving their instance running even in non-busy times.

On the flip-side, people using the Horde as clients are likewise consuming Kudos to do so. Every prompt you send, will reduce your balance by the same amount of Kudos rewarded to the server which fulfilled your request. Effectively you’re “thanking” the people running the instance which helped you.

But unlike typical currencies, you can still use this service even if you don’t have any Kudos. You simply go negative, which represents your “debt” to the community. This allows people to use the Horde anonymously as well (through the common Anonymous account) which simply keeps going more and more in negative Kudos.

But to avoid abusing this service by people who want to take and give nothing back, I needed to find a way to disincentivize negative kudos balances. And I just implemented the solution to do that which is a fairly simple one.

You kudos balance corresponds to your priority in the queue of requests

This means that if there’s plenty of servers around and not a lot of requests, nothing much changes, even for anonymous clients and they get to enjoy the full speed of the horde.

However on times of congestion, the requests sorted by kudos-order. The people with the most kudos get to have their requests fulfilled first, while those with the least, such as the Anonymous account, will get their generated last.

With some other tools provided such as bridge-based priorities, and server-owner-preference, this should allow owners of servers to make sure that not only “good” actors are the first to use their resources, but also to find their own ways to provide a service to selected members of the community. For example someone might set their family members to have priority on their server gens, even if those members are deep in negative kudos.

Along with that, I’ve deployed a kudos transfer service where peers can send kudos directly to others. So someone who has accumulated a large amount, might use that to reward good behavior in the community, such as writing documentation, or helping others. The kudos transfer can also happen via REST API, which can allow even automated tools to participate, such as discord bots which might trigger on special smileys assigned to posts and give a small kudos reward to the recipient.

There are of course many other things to consider, such as actions by malicious actors who might try to game the system. I really hope I won’t have to deploy countermeasures for people abusing community goodwill.

But until then, I really enjoy seeing the excitement from the KAI community, especially as heavy models like the fresh Erebus 13B are deployed into the horde. With the kudos infrastructure in place, there is now potential for new dynamics and reward systems to emerge organically from the community which will serve as an excellent guide on how to best proceed with this project.