We need people to test a PR for onboarding stable diffusion inpainting onto voodoo ray. Please pull this branch and load inpainting and see how stable it is and/if the VRAM is unloaded correctly after each image.
Tag: ai horde
Some napkin math
Stable Horde has generated ~180 Terapixelsteps of images. Assuming each image is 512x512x30 that is like 22 million images (higher resolutions have an exponential difficulty).
Using the current cost of http://dreamstudio.ai, the Stable Horde has generated for free a value of close to $45000! Using the old http://dreamstudio.ai costs (Stable Horde has been up almost as long), this is closer to $230.000 All this value has been given out voluntarily, with no ads or fine print.
Taking into account the post-processors allowed and the exponential difficulty of higher resolution images (Stable Horde allows up to 3072×2048), these numbers can easily be doubled.
For reference, in its stable horde lifetime, my patreon account has made $500,most of which has gone to infrastructure costs.
Codeformer and Reddit Bot for the Stable Horde
I haven’t been able to improve the Stable Horde a lot lately. I was planning to do a lot of work during the week leading to Christmas, but unfortunately the universe had another idea and not only infected me with the nastiest cold I’ve had for decades, but my whole family as well, including the visiting Grandma!
So instead of adding necessary new features, I’ve been instead flattened at bed, trying to muster enough concentration to do some basic updates and answer questions.
Nevertheless, there’s a few improvements added, mostly through the work of some members of the community.
First is the addition of the CodeFormer face-fixing post-processor which seems massively better than the GFPGAN model. Now all clients can request that an image be bassed through CodeFormer for an immediately improvement in faces. Soon I plan to allow this to run in isolation as well
The other new thing is improvements on the workers themselves, allowing them to pickup and perform jobs more efficiently.
The other big news I have is that wrote and unleashed the first Reddit bot for stable Diffusion. That was initially created as an entry for the Ben’s Bites Hackathon since I couldn’t submit the Stable Horde itself (I didn’t win btw), but it was quite an eventful release. My initial release got caught by the automated reddit anti-spam filter, shadow-banning my account and banning my subreddit. Then I refactored the bot to use my own R2 CDN and released it with a new account while asking for a reddit review on my original account. Fortunately my bot account and subreddit got unbanned and I finally released it a third time properly, and it’s been up ever since!
The way the bot is created you can request images from it all over reddit, and it will post the images in its own subreddit for everyone to see and vote on.
There’s also been a lot of new models and styles onboarded, which are also used by my reddit and mastodon bots.
The next plan now is to allow image interrogation on the stable horde, as well as direct image post-processing (without stable diffusiion), so as to allow even people with low-powered machines to be able to contribute for kudos.
Automated Rewards and Tiers
Hey everyone, I have finally finished integrating Patreon with the Stable Horde. This will allow me to maintain everyone rewards automatically, and adjust them according to your status on patreon as well.
This means for example, if you upgrade your subscription from one tier to the next, you will also receive the adjusted reward on the next month’s cycle without me having to find the time to adjust things
Also there is a new adjustment, where now patrons can utilize the horde even via VPN. The VPN restriction is in place to prevent people sending or requesting unethical or illegal content, so by trying your bank account to your username, you provide a way to block the access if it’s abused this way. This is allowed from the lowest tier, so if you have any reason to use VPN to access the stable horde, feel free to use this.
Finally, as an extra thank you, I have also added a small “candy” even for the lowest tiers!
For all this to work, I need to know your Stable Horde ID, and specifically the #number# next to your username! So please send it to me asap if you haven’t already! If you ever change accounts on the horde, you’ll also need to send me the new ID.
When you provide your Stable Horde id, you will also be listed on the sponsors page . If you would prefer a different alias to be shown on the sponsors, please let me know as well.
v1.12 of Lucid Creations is out
QoL and using the new R2 storage backend. Here’s the devlog
Other blogs
I’ve still been blogging, but on other platforms, which fit the purpose. I’ve been thinking it might make sense to crosspost them from here, as a central location.
So without further ado, new devlog about Stable Horde: Automated Rewards and Tiers
The Stable Horde: AI image generation for everyone through mutual aid
After completing the KoboldAI Horde, and onboarding into the KoboldAI client, I felt that there is a really big opening for doing a similar thing using the open sourced AI image generating model, Stable Diffusion. I already have the code for setting up a crowdsourcing cluster, so it shouldn’t take too much refactoring to make the same underlying code work with Stable Diffusion.
The first thing I had to do was figure out what is going to run on the workers. For this, I decided to reuse the stable-diffusion-webui fork by simply adding my bridge code on top of it (as it doesn’t provide a REST API like KoboldAI). Once I had a valid bridge, it was time to fork the Horde.
And thus, the Stable Horde was born!
It follows the same approach, where workers running some version of Stable Diffusion constantly poll for new generations to complete and then send it back to the horde to hand it over to their final destination. For now the stable horde is only handling fairly basic text2image generations, but since it’s based on the webui, I can tap into the features that added upstrean much easier, without having to develop them myself.
The code started as a fork of the Stable Horde, but has by now become my primary repo. In fact, with the addition of the second version of the REST API, I have decided to merge both Hordes into a single repository in order to better share code updates (because copying code from one repo to the other was driving me nuts!). This is coming soon, and it means that the Stable horde will always remain in parity with the KoboldAI horde from now on.
While there are other free image generation tools out there, I believe none is doing anything like what I am attempting. Most of these are based on providing free Stable Diffusion by eating the costs themselves, but with an undefined business plan. And when I see that, my suspicions are already raised, as a free service like that, typically means you’re the product! It also doesn’t help at all that they are not sharing the code behind them.
Now you might say, “But db0, your service is also free, how come the same criticism doesn’t apply to you?”. Which is a great question. The answer is that the reason the Stable Horde is free is because it’s volunteer based. That means, at the end of the day, someone is indeed paying for electricity (that is, myself primarily atm), but the point is that it is self-managing through people’s innate drive for mutual-aid.
That means that if I get a jump in popularity, which in turn exceeds the Horde’s current image generation capacity (and therefore slowing things down too much), the belief is that there will be enough people annoyed by the speed, that they will join their own power to the horde to benefit themselves with higher priority, but also everyone else.
And yes, there is always some amount of “small print”. While the Stable Horde is built on anarchistic principles of mutual aid and direct action, the fact is that we do not control the underlying workers. Therefore it is theoretically possible for people to act maliciously on the worker side, which is why I always warn people who will use the Horde that I cannot guarantee that nobody will see your prompts. So act accordingly.
Nevertheless, one of the things I’m offering is something that I just haven’t seen anyone else do for image gen, and that is a fully functioning RESTful API. The purpose of this it to further enable image generation for everyone in new and exciting ways that enable tools to use this capability without bankrupting their owners for a side hobby whose demand suddenly spiked. Already people have started creating some interesting tools, such as a weather app which uses the Stable Horde to generate a dynamic image representing the weather, based on environmental conditions.
On my end, I am interested in helping game developers figure out ways to implement AI into their games. For this purpose I have already released a Godot Add-On which allows you to request AI image generation during a game’s runtime. I have further used this add-on to create my own Stable Diffusion GUI client that can run on any device, without the need for a complicated install procedure, or a GPU.
All of this is just scratching the potential of what can be achieved by allowing automation to connect directly to Stable Diffusion (or text generation), and I’m excited to see what people will come up with in the future!
The KoboldAI Horde: AI writing for everyone through mutual aid
Those who’ve followed my progress in developing Hypnagonia would know by now that I’ve been working on the integration of KoboldAI with Godot for the automatic generation of stories. This solution while working never quite satisfied me, as it required quite a lot of investment on behalf of a player who wanted to generate stories.
That approach left me unsatisfied. As it happened I had an idea to use the new KoboldAI API to create a distributed cluster based on consumer GPUs in people’s PCs. The concept being that one would connect their PC in some fashion to a server software and whenever someone sent a request for generation, an available PC would pick it up, generate it, and send it back to the requester.
The requester themselves would therefore not need a powerful GPU (or skill to use Google Colab) to use KoboldAI, while people who’s PC is otherwise sitting idle, would be supporting the other members of the community.
So after some discussions with members of the community, I decided this idea had legs and I wanted to build it before I continued with Hypnagonia. I’ve been working non-stop on this for the past week and I now feel it’s at a pretty good state.
Introducing the KoboldAI Horde!
This is a python server which you run on a server somewhere and it provides an interface with which people can request GPT writing generations. The second part is the bridge, which is what people who have their own KAI instances run, in order to connect the KAI server to the server. The last piece of software of course is the KoboldAI client which runs the models generating the text.
Until now, the only option for people without such powerful hardware would typically to use a service like NoveAI or AI Dungeon, where one has to pay for the service. They also provided a fancy interface and tools to help people write their individual stories. Aside from the cost however, there were also limitations. Dungeon AI recently imploded due to their own actions, and a lot of people were left without a home.
And of course, for a free software video game, such services are prohibitive, as I can’t pay out of pocket for the text generations of dozens of players (or imagine if my game ever became actually popular).
After a bit of hacking of the KoboldAI client itself, I also added functionality to allow someone to use their own KoboldAI client to talk to the Horde directly. Which means someone with a potato PC, can still use KoboldAI directly.
Now there are some obvious limitations as you might expect. First of all, the availability of the models in the horde is dependent on who is supporting it at the moment. If only people with weak GPU are in, the best you’ll find is a Nerys 2.7B. If however some of the big chonkers are in, you might be able to use KoboldAI as if you have a 20B model behind it.
The other problem is of course questions of privacy. Even though the text is encrypted during transfer, and even though the horde does not store any prompts or generations, at the end destination, the owner of the server can see all your prompts if they so wish, even if they don’t know who you are. So for those wanting the really steamy stuff, or perhaps those who write stories using real people and locations, this might be a deal breaker.
However for more normal stories, or for more generic generations such as for utilities and apps, this is hardly a concern.
Finally we’re still new, so currently everyone is playing nice. But there’s always the potential of bad actors, who might food the horde with garbage generations just to shut it down. There’s nothing I can do about someone dedicated to do that except go into a whitelist mode, and have the horde become an invite-only service, much like a private bittorrent tracker. In fact, anyone could set up their own private instance for their own community, for those situations where they want only trusted people to use it.
Eventually my plan is to follow the bittorrent approach for usage and contributions as well, where people will only be able to use the horde if they contribute somehow to the KAI community. Whether by adding their own GPU to the horde, or by writing documentation, etc.
But ultimately I want this to be a mutual aid based service. Where those who have the capacity help those who do not, and the latter find some other ways to make it up, for the uplifting of everyone in the KoboldAI community.
And before anyone suggests to tie this to some web3 token: Fuck that shit! I will work integrate with any cryptocurrencies! That is poison for our communities AND for our ecosystem.
The owner of the KAI client repo was also kind enough to redirect their domain to my KAI Horde instance, so now we have a very fancy url as well.
Already this is working amazingly well and I’m working on improvements daily. I’m excited to see what kind of doors this will open.
