Back in July I first discovered NLNet and decided to apply for their NGI Zero Core grant to help me help me continue developing the AI Horde. Today I’m excited to announce that the AI Horde has officially been greenlit as one of the projects which will receive the August 2023 grant!
AI Horde has been a passion project from the start but it’s difficult to maintain the level of intensity I had for such a length of time. With my patreon funding significantly dwindling month-to-month, multiple of our backend developers dropping, and the need for me to also keep up with some of my real life duties as well as my other FOSS projects, development of the AI Horde has sadly slowed in recent months. While It rmains extraordinarily stable, it has had little to announce, and some new features are slower to “cook” and reliant in the work of valuable backend volunteers like Tazlin.
I am hoping that the addition of NLnet funding should help reintroduce some of that momentum as the release of the grant is contingent on specific milestones being reached.
I have plans for 5 roughly grouped areas of development, for which I have thought of various tasks to receive “bounties” for in the scope of this grant: AI Horde, Dreamer, Scribe, Alchemist and Godot Engine
The AI Horde tasks will focus on improving the middleware itself. Such as extending the shared key functionalities, improving the API documentation etc.
The Dreamer, Scribes and Alchemists tasks will focus on adding new functionalities to the the official workers, such as more generation workflows, batch processing etc.
Finally the Godot Engine tasks will improve the existing toolset of the AI Horde to support game development, such as improving my AI Horde Client, migrating it and Lucid Creations it to Godot Engine 4, etc.
The good news is that due to the way NLNet works, I had to submit stuff to work on, but I couldn’t work on them before the AI Horde was officially accepted (or rather, I could, but I couldn’t receive a “bounty” for them). Now that this is locked-in, I can start working on some of these with the added incentive of getting a reward at the end which can alleviate some of my ADHD executive disfunction. So if all goes well, you should start seeing more activity from me soon.
As always, if you want to support the AI Horde and my work in FOSS and the open commons, please do consider funding me at:
The AI Horde has turned one year old. I take a look back in all that’s happened since.
Can you believe that I published the first version of the AI Horde exactly 1 year ago? The first version published was called the KoboldAI Horde as it was built around KoboldAI Client specifically, and it had very little traction. But almost as soon as I built it, Stable Diffusion came out, I forked the project to handle SD, and my life was never the same again!
Since the start of its life the AI Horde has generated ~84M images and ~30M texts, free for everyone! We’ve been making and will continue making hundreds of thousands of images and text per day. We now have literally dozens of third party UI and bots integrated directly into the AI Horde.
A quick recap of all that’s happened this year
Sept 2022 – KoboldAI Horde is launched. KoboldAI gets integrated Horde support. Stable Horde is later forked from KoboldAI horde and launched. Lucid Creations is published.
Oct 2022 – We get our first raid. First countermeasures are developed. Artbot is published. Stable UI is published 1 hour later. Our first official discord bot is published. Img2Img is developed. Stable Horde fulfills 1 Million requests.
Nov 2022 – Teams are added. Post-processing is developed, Mastodon bot is launched.
Dec 2022 – Stable Horde reaches its limits and is massively refactored to scale better. Reddit bot is launched. KoboldAI lite is launched. Stable Horde is in the news for the first time
Jan 2023 – Image interrogation is developed. First third party mobile app is launched. Worker UI is developed. We start collaborating with LAION to gather Aesthetic ratings. The first third party chrome extension is launched. We start collaborating with Stability.ai. We replace the discord bot with different codebase.
Feb 2023 – Stable Horde and KoboldAI Horde are merged into the single unified AI Horde.
Mar 2023 – AI CSAM filter is developed. First “State of the AI Horde” is published. Ratings receive bespoke countermeasures.
Apr 2023 – AI Horde breaks away from the nataili inference developer due to toxicity.
May 2023 – AI Horde switches from the nataili backend to using comfyUI as a backend. Shared keys are developed. Massive documentation update. Ratings are published onto Huggingface. LoRa support is added.
Jun 2023 – AI Horde starts supporting all LoRas in CivitAI. hordelib receives a DMCA from the previous nataili developer
Jul 2023 – hordelib is recovered from DCMA limbo. SDXL Beta is added on the AI Horde with collaboration from stability.ai. Haidra is announced and we become part of Nivenly
Aug 2023 – Textual Inversion support is added. AI starts supporting all TIs on CivitAI
Writing this down, just makes me realize how much stuff has been happening constantly! Not a single month where nothing much happened. The most relaxed would be Dec-Jun where I spent basically 1 month in bed sick, but I still managed some big results!
The birthday celebration event is still ongoing at the time of publishing this in the AI Horde discord where I’m staying online in a voice chat and answer questions or just chat with people. We have plenty of threads for generating and sharing art etc.
I also asked the AI Horde community to tell me what the AI Horde means to them, and I wanted to quote some of them here:
The AI Horde’s (completely free and open) Image and Text generation services rely on volunteers donating their computing resources, and those volunteers do so without profit motive or selfish reasons. The result is huge number of people – tens of thousands, at least – have been able to get access to a technology that may be otherwise of their reach. I find this to be a hugely inspiring display of altruism and goodwill on the part of everyone involved, and I am deeply humbled and honored to be part of that ecosystem of people
Tazlin – AI Horde backend developer
AI Horde took KoboldAI from personal software to an online platform and allowed us to build a version people don’t have to install. Now its not just used for personal use, but also as a testing platform by model developers for their latest models before public releases.
Henky – KoboldAI Lead
Thanks to Stable Horde, I can generate new storyboard images for existing film concepts making the process of hashing out scenes and sequences for film projects easier. I’m thankful for everyone involved, the contributors and software developers for the tools, and to db0 for making all of this possible.
Mr.Happy
I think horde is a great way to make AI accessible to the general public without the need for complicated installs or expensive hardware. It’s certainly a great and low friction way to get people introduced to generative AI without the limitations and restrictions of closed off/ corporatized AI services like chatgpt… it’s been a vital backbone of the FOSS AI ecosystem over the past year.
Concedo – KoboldAI Lite developer
AI Horde is a powerful tool for breaking the monopoly of large corporations and providing AIGC with an independent platform to avoid the risk of being controlled by these corporations. This platform offers more affordable, free, and diverse AI services. Many AI drawing service providers now operate their own GPU cloud services and charge users high fees while controlling, castrating, and modifying AI drawing models, which severely infringes on user rights. The emergence of AI Horde allows AIGC to retain all potential while providing new ideas and solutions for expanding AI application scenarios, making it easier for ordinary people to access various AI technologies.
Pawkygame – AIpainter developer
For me, the whole community that’s been built around the Horde highlights some of the best about free and open source software. There are so many enthusiastic and knowledgeable people who are wiling to help each other out or discuss the latest in the world of generative AI. It’s been an awesome community to be a part of and one of my favorite little corners of the internet.
Rockbandit – Artbot developer
Stable Horde is an excellent tool for generating images, useful for creating reference material for traditional or digital art, creating art itself through the platform, or using it as a starting point to learn about AI. And the fact that it’s a free platform for image generation is a significant blow to companies that seek to monetize AI access.
It’s great for those who lack artistic skills like myself, and more skilled artists can use it for inspiration, or take a hybrid approach by making something manually and using image to image generation.
The various distributed workers available means that the barrier for entry for joining the horde is a device with an Internet connection, and I firmly believe the collaborative nature of the Horde would make Tim Burners-Lee proud.
It’s an excellent first step towards a bright future, full of opportunity made possible by free and open access to advanced technology. Now we just have to hope that the rest of humankind doesn’t trip.
CrabMan314
The AI Horde is the best way to burn us worker GPU for other’s happiness
Almost immediately after the AI Horde received LoRa support, people started clamoring for Textual Inversions which is one of the earliest techniques to fine-tune Stable Diffusion outputs.
While I was planning to re-use much of the code that had to do with the automatic downloading of LoRas, this quickly run into unexpected problems in the form of pickles.
In the language of python, pickles is effectively objects in memory stored into disk as they are. The problem with them is, that they’re terribly insecure. Anything stored into the pickle will be executed as soon as you load it back into RAM, which is a problem when you get a file from a stranger. There is a solution for that, safetensors, which ensure that only the data is loaded and nothing harmful.
However, while most LoRas are a recent development and were released as safetensors from the start, textual inversions (TIs) were developed much earlier, and most of them are still out there as pickles.
This caused a big problem for us, as we wanted to blanket support all TIs from CivitAI, but that opened the gate for someone to upload a malicious TI to CivitAI and then request it themselves through the horde and pwn all our workers in one stroke! Even though technically CivitAI scans uploaded pickles, automated scans are never perfect. All it would take is someone to discover one way to sneak by an exploit through these scans. The risk was way to high for our tastes.
But if I were to allow only safetensors, only a small minority of TIs would be available, making the feature worthless to develop. The most popular TIs were all still as pickles.
So that meant I had to find a way to automatically convert pickles into safetensors before the worker could use them. I couldn’t do it on the worker side, as the pickle has to be loaded first. We had to do it in a secure location of some sort. So I built a whole new microservice: the AI Hordeling.
All the Hordeling does is provide a REST API where a worker can send a CivitAI ID to download, and the Hordeling will check if it’s a safetensor or not, and if not, download it, convert it to safetensor and then give a download link to the safetensor to the worker.
This means that if someone were to get through the CivitAI scans, all they would be able to exploit is the Hordeling itself which is not connected to the AI Horde in any way and can be rebuilt from scratch very easily. Likewise, the worker ensure that they will only download safetensor files which ensure they can’t be exploited.
All this to say, it’s been, a lot more work than expected to set up Textual Inversions on the horde! But I did it!
So I’m excited to announce that All textual inversions on CivitAI are now available through the AI Horde!
The way to use them is the very similar to LoRa. You have to specify the ID or a unique part of its name so that the worker can find them, in the “tis” field. The tricky part is that TIs require that their filename in the prompt, and the location of the TI matters. This complicates matters because the filename is not easy to figure out by the user, especially because some model names have non-Latin characters which one can’t know how they will be changes when saving on disk.
So the way we handle it instead is that one needs to put the CivitAI model ID in the prompt, in the form of “embedding:12345”. If the strength needs to be modified, then it should be put as “(embedding:12345:0.5)”. On the side of the worker, they will always save the TIs using their model ID, which should allow ComfyUI to know what to use.
I also understand this can be quite a bother for both users and UX developers, so another option exists where you allow the AI Horde to inject the relevant strings into the prompt for you. You can specify in the “tis” key, that you want the prompt to be injected, where, and with how much strength.
This will in turn be injected in the start of the prompt, or at the end of the negative prompt with the corresponding strength (default to 1.0). Of course you might not always want that, in case you want to the TI to be places in a specific part of the prompt, but you at least have the option to do it quickly this way, if it’s not important. I expect UX designers will want to let the users handle it both ways as needed.
You are also limited to a maximum of 20 TIs per request, and there’s an extra kudos cost if you request any number of TIs.
So there you have it. Now the AI Horde provides access to thousands of Textual Inversions along with the thousands of LoRa we were providing until now.
Maximum customization power without even a need for a GPU, and we’re not even finished!
A while ago I was having a lot of troubles trying to figure out how to make open source development work. It was a completely new endeavour by me and I kept running into payment processors issues. In desperation I wrote about it in Hachyderm, which is my primary mastodon instance and someone suggested to contact Nivenly which is an org created explicitly to provide guidance to such matters for open source software and are also based on Hachyderm.
While I discussions were ongoing with them, work was progressing on the AI Horde itself but I had eventually realized that I can’t keep everything under the AI Horde moniker. The AI Horde itself is the main middle-ware making it all possible, but nobody can use it in isolation. We don’t only need the Worker backends, but we also need the various frontends and bots which the end-users will use.
So I set up a Github org to provide an overarching group for the whole official ecosystem around the AI Horde. I called it Haidra, as is both can portmanteau the “AI” part but it also represents the endless potential between the integrations around the AI Horde. It is also one of my favourite concepts.
Long story short, Haidra is now hosting all the repositories that the public AI Horde needs to run, along with a lot of helper libraries and UIs. It has as members all the people who have provided invaluable support in development or community building. We also designed a neat logo for it using the AI Horde and a competition in our discord server which Tazlin further polished as our new logo. You can see it as the featured image in the post.
While there’s a lot of people helping to make Haidra what it is, most of the community building still falls on me, along with the social outreach. Likewise there’s a lot of questions and problems around governance of an expanding community which are really not my strength. This is where Nivenly comes in.
The plan is that they will help us with best practices on growing and sustaining our community, and finding more volunteers, especially in community management and software development. They will also provide us with governance and legal support as needed. And generally plug any holes which a lot of developers (including with me) don’t have the skills to deal with.
I hope with The backing of the Nivenly Foundation, we can all together take Haidra and the AI Horde to the next level and ensure that Generative AI remains available for everyone forever!
In the past month or so, I’ve been collaborating with stability.ai to help them improve the quality of the new upcoming SDXL model by providing expertise around automation and efficiency in quickly testing new iterations of their model using the the AI Horde as middleware.
Today I’m proud to announce that we have arranged with stability.ai that the new SDXL beta will become available on the AI Horde directly, in a form that will be used to gather data for further improvements of the model!
You will notice that a new model is available on the AI horde: SDXL_beta::stability.ai#6901. This is the SDXL running on compute from stability.ai directly. Anyone with an account on the AI Horde can now opt to use this model! However it works a bit differently then usual.
First of all, this model will always return 2 images, regardless of how many you requested. The reason being is that each image is created using a different inference pipeline and we want you to tell us which you think is best! As a result, the images you will create with this model are always shared! This means that the images you generate will be stored and then used for further improvement based on the ratings you provide.
Which ratings? Well each time you generate images with SDXL you can immediately rate them for a hefty kudos rebate! Assuming your client supports it, you should always report back which image is better than the two. You can then optionally also rate each of them for aesthetic ratings (how much you subjectively like them) and for artifacts (how ruined the image is from generation imperfections like multiple fingers etc)
For best results with SDXL, you should generate images of 1024×1024 as that is its native resolution. The minimum allowed resolution for SDXL has therefore been adjusted to this.
Also, as I mentioned before, this beta is primarily going to be used to improve the model, therefore we’re disallowing anonymous usage on this model for the moment. However registering an account is trivial and free and will also give you more priority in your generations!
The most cool factor is that as stability.ai further improves the model, is will automatically become part of this test, without you having to do anything! As such, you will eventually start getting better and better generations from the SDXL_beta on the AI horde. This is why you continued sharing of the quality is very important!
So please go forth and use the new model and let us know how it performs!
It’s high time I wrote one more of these posts to keep everyone up to date. It’s been generally a fairly slow month as far as the Horde is concerned. That’s not tot to mean that we produced less content, but rather that there hasn’t been a lot of progress on features as all the developers appear to have been busy with other projects a lot.
(In case you’re new here and do not know what is the AI Horde: It is a crowdsourced free/libre software Open API which connects Generative AI clients to volunteer workers which provide the inference.)
LoRas in the mix
Since the last State of the AI Horde, we saw the introduction of LoRas into payloads allowing everyone access to all LoRas in CivitAI at the click of a button. I have likewise been recording the amount of times a lora is being used, storing it by its CivitAI ID. Below you can see the top25 LoRas used since we started recording them. You can check which one it is by adding the number to this URL https://civitai.com/models/<LORA ID>. Using this method we can see that the Add More Details LoRa is clearly one of the most valuable one following closely by Details Tweaker. People do love adding more details!
All in all, a total 801.326 LoRas have been recorded being used successfully in the AI Horde!
In image news, our usage remains fairly stable, which is fairly impressive if one considers just how much extra slowdown is added by all these LoRa. We are stable at ~300K images generated per day and ~1M images per month. Worth noting that since it started the AI Horde has generated close to 1 whole PETApixestep for free and ~70 million images!
On the model side, Deliberate solidifies itself furher as the best generalist model, while Stable Diffusion drops down to 3rd place as the Anime takes the 2nd spot. Our own special Kiwi, ResidenChief’s model seems to have been a massive success as well, coming out of nowhere to grab a solid 4th place. And this time the Furries are also in forth, capturing the 6th position! Pretty cool stuff!
Deliberate 31.6% (3127771)
Anything Diffusion 11.4% (1133058)
stable_diffusion 9.6% (951705)
ICBINP – I Can’t Believe It’s Not Photography 6.7% (662205)
Dreamshaper 5.6% (550335)
BB95 Furry Mix 3.7% (370853)
Hentai Diffusion 3.1% (303351)
Counterfeit 2.6% (254197)
ChilloutMix 1.9% (186525)
Pretty 2.5D 1.8% (178713)
Text Generation Stats
On the text side, not much has changed since May, with out generation staying similat at ~3M requests fulfilled and 377 Megatokens.
Likewise on the model top 10 Pygmalion 6B is still leading the pack with Mr. Seeker’s Erebus and Nerybus still in heavy use.
The main reason for being otherwise busy is that I’ve been furiously transferring my Reddit presence to my own self-hosted Lemmy instance, because Reddit is speed-running enshittification. I won’t bore you with the details but you can read up some of my work and see some of my development in the relevant blog tag.
However I do want to say that the instance I’ve fired up, the Divisions by zero has been more successful than I ever could have imagined! With ~10K users registered and thousands of subscribers to the communities, and some of the best admin team I could hope for.
Unfortunately there’s also been reddit drama which has been mightily distracting to me, but things are slowly settling down and I am putting my reddit days well behind me.
In relevance to the AI Horde however, we do have some cool communities you should subscribe to
I am likewise already planning more events and automation to more closely tie the AI Horde into the Lemmy instance for cool art stuff! Stay tuned and/or throw me your ideas!
Prompt Challenges
R from our mod team has started running some cool prompt challenges in the discord server which you are all more than welcome to join! Winner gets a nice bundle of kudos, not to mention the amount you get by simply posting. It’s just fun all around, and the winners are featured in the Lemmy communities as well!
Worker Updates
Tazlin has been hard at work improving the AI Horde Worker with bugfixes (not to menton the huge amounts of tech support given in discord). The AI Horde Worker as a result has become much much more stable which should have a good impact on your kudos-per-hour! Just a quick shout-out to an invaluable collaborator!
A Lot more workers
I don’t know how it’s happening but the AI Horde is nearing 100 dreamers! I am getting the fireworks ready for the first time we hit this threshold!
Funding and Support
My current infrastructure has been sufficiently stable since the last migration to a dedicated host which I think you have experienced with the low amount of downtimes and interruptions since.
This is my usual opportunity to point out that running, improving and maintaining the the AI Horde is basically a full-time job for me so please if you see the open commons potential of this service, consider subscribing as my patron where I reward people with monthly priority for their support.
While development slowed significantly in June, we’re still doing significant work for the open commons. I have just not had the mental capacity to build up hype as much as I used to, and to make it worse, the social media landscape is completely in the air at the moment.
I am really hoping more people can step up and help promote the AI Horde and what it represents as my workload is just through the roof and to be perfectly honest, I am at the limit of my “plate-spinning” capabilities.
Please talk about the AI Horde and the tools in its ecosystem. The more people who know about it, the more valuable it becomes for the benefit of everyone!
We have plenty of ways one can help and we shower with kudos everyone doing so. From people sharing images and helping others in the community, to developers bug-fixing my terrible code, to community managers on discord and admins on lemmy. If you want to help out, let us know!
Lucid Creations has gotten a few more updates, with the most recent being the ability to load your generation settings from a previous generation you have saved. Plus a few other quality of life fixes
More Styles
The AI Horde stylelist has received Lora support and about 40 new ones have been added using the awesome work by konyconi. Likewise a new category called “stylepunks” has been added which will generate using random styles from those new ones.
Mastodon & Reddit Bots
The Mastodon and Reddit bots have been updated to support LoRa styles
Discord Bot
The official AI Horde discord bot has also been updated by ZeldaFan to support the new LoRa styles, but no categories support yet. Hopefully soon.
Other Clients
A ton of other clients are receiving flood of updates. Particularly ArtBot and aislingeach are both seeing very rapid development. Remember to check their individual discord channels if you haven’t checked them out yet
As I mentioned last time, we received a DMCA take-down for a couple of missing attributions on AGPL3 files. We have since added those back, but as is the case with DMCAs, once you start them, they don’t stop on their own.
I remind that hlky had made no attempt to request those attributions informally, just went straight to DMCA, which tracks since this person is always acting in bad faith. It just in this case it’s a convenient way to attack the AI Horde project once more.
Demand for AI Horde to support LoRas was high ever since they were discovered and unleashed into the Stable Diffusion ecosystem. About 2 months ago we were really close to releasing support for them but alas “stuff happened” which culminated in us having to completely rewrite out inference backend.
Fortunately, ComfyUI already supported LoRas, so once we switched to using it as our inference library, we just needed to hook into its LoRa capabilities. Myself along with the extremely skilled hordelib devs, Jug and Tazlin have been working constantly on this for the past half a month or so and today I’m proud to announce that the Lora capability has been released to all AI Horde Workers!
You might have seen earlier that some Client like Artbot, the Krita plugin and Lucid Creations were already supported LoRas. This was however only using the few beta testing workers we had. Now those clients will start seeing a lot more speed as worker with the power to do so start running them.
This also signifies our tightest integration with CivitAI yet! We we were already utilizing them, but mostly to just pull stable diffusion checkpoint files. The LoRa integration however goes far far further than that, using them as an integral part of our discovery and control process.
Through the very useful REST API provided by CivitAI, we have developed a method where we can discover, download and utilize any LoRa that is hosted there. No more do you need to search, validate compatibility, download etc. The AI Horde and its Worker will handle that automatically!
To showcase this point, I’ve made a small video showing off this capability in Lucid Creations which I’ve tightly integrated into CivitAI, so that it can look for any LoRa based on ID or partial name, display all relevant information and even warns you of compatibility issues.
Sorry about the sound quality. I don’t have a professional streamer setup, just a shitty webcam 😀
I’ve put a lot of effort into making finding and using LoRas as painless as possible, because there’s literally dozens of thousands of the things. Manually searching through the website and copy-pasting IDs is just a complete PITA. Never mind downloading and trying them out one-by-one.
Through the AI Horde, you can simply type the partial name of what you want, and keep mix and matching them until you achieve the desired outcome! Changing and trying new LoRas now takes literally a few seconds!
As this has just been released, currently only a few UIs support this. Lucid Creations which has full discovery, and ArtBot which requires you to know the ID. The Krita plugin is also supporting them. I expect the other UIs to soon follow in the coming days.
Hopefully this is another step in unleashing your creativity, without requiring a powerful GPU of your own!
Stay tuned for the next update adding access to all Textual Inversions as well.
In case you don’t know what LoRa are, the short version is nothing less than a breakthrough in Generative AI technology, by allowing to condense the time and power a training (or “fine-tuning”) needs!
This kind of breakthrough, along with the achievements done by new models such Stable Diffusion and Llama is what is causing the big tech companies to scramble, to the point that OpenAI had to run to the government to ask them to regulate future breakthroughs (but please, please don’t regulate what they’re currently doing, OK?)
But I digress. I haven’t yet officially announced LoRa support on the AI Horde yet, as we’re still trying to squash all bugs on the workers, but I hope the addition to Lucid Creations will help other integrators see how it can be done
Close
Ad-blocker not detected
Consider installing a browser extension that blocks ads and other malicious scripts in your browser to protect your privacy and security. Learn more.
