It’s high time I wrote one more of these posts to keep everyone up to date. It’s been generally a fairly slow month as far as the Horde is concerned. That’s not tot to mean that we produced less content, but rather that there hasn’t been a lot of progress on features as all the developers appear to have been busy with other projects a lot.

(In case you’re new here and do not know what is the AI Horde: It is a crowdsourced free/libre software Open API which connects Generative AI clients to volunteer workers which provide the inference.)

LoRas in the mix

Since the last State of the AI Horde, we saw the introduction of LoRas into payloads allowing everyone access to all LoRas in CivitAI at the click of a button. I have likewise been recording the amount of times a lora is being used, storing it by its CivitAI ID. Below you can see the top25 LoRas used since we started recording them. You can check which one it is by adding the number to this URL https://civitai.com/models/<LORA ID>. Using this method we can see that the Add More Details LoRa is clearly one of the most valuable one following closely by Details Tweaker. People do love adding more details!

All in all, a total 801.326 LoRas have been recorded being used successfully in the AI Horde!

 lora  |     count 
-------+-----------
 82098 |     46095
 58390 |     36222
 48139 |     18042
 60724 |     16980
 13941 |      9245
 32827 |      9191
 87245 |      9083
 43814 |      8999
 12820 |      8148
 28742 |      8009
 9652  |      7960
 82946 |      7922
 25995 |      7671
 28511 |      7225
 87080 |      5966
 16928 |      5950
 6693  |      5431
 24583 |      5121
 48299 |      4905
 42214 |      4838
 63278 |      4821
 9025  |      4105
 9651  |      3828
 10816 |      3786
 37006 |      3759

Image Generation Stats

In image news, our usage remains fairly stable, which is fairly impressive if one considers just how much extra slowdown is added by all these LoRa. We are stable at ~300K images generated per day and ~1M images per month. Worth noting that since it started the AI Horde has generated close to 1 whole PETApixestep for free and ~70 million images!

On the model side, Deliberate solidifies itself furher as the best generalist model, while Stable Diffusion drops down to 3rd place as the Anime takes the 2nd spot. Our own special Kiwi, ResidenChief’s model seems to have been a massive success as well, coming out of nowhere to grab a solid 4th place. And this time the Furries are also in forth, capturing the 6th position! Pretty cool stuff!

1. Deliberate 31.6% (3127771)
2. Anything Diffusion 11.4% (1133058)
3. stable_diffusion 9.6% (951705)
4. ICBINP – I Can’t Believe It’s Not Photography 6.7% (662205)
5. Dreamshaper 5.6% (550335)
6. BB95 Furry Mix 3.7% (370853)
7. Hentai Diffusion 3.1% (303351)
8. Counterfeit 2.6% (254197)
9. ChilloutMix 1.9% (186525)
10. Pretty 2.5D 1.8% (178713)

Text Generation Stats

On the text side, not much has changed since May, with out generation staying similat at ~3M requests fulfilled and 377 Megatokens.

Likewise on the model top 10 Pygmalion 6B is still leading the pack with Mr. Seeker’s Erebus and Nerybus still in heavy use.

1. PygmalionAI/pygmalion-6b 36.4% (1118193)
2. KoboldAI/OPT-13B-Erebus 9.2% (284531)
3. KoboldAI/OPT-13B-Nerybus-Mix 6.4% (197689)
4. Pygmalion-7b-4bit-GPTQ-Safetensors 4.8% (148631)
5. VicUnlocked-alpaca-65b-4bit 3.8% (116034)
6. chronos-hermes-13B-GPTQ 3.4% (105727)
7. KoboldAI/OPT-30B-Erebus 3.2% (99782)
8. manticore-13b-chat-pyg-GPTQ 2.7% (82276)
9. 13B-HyperMantis/GPTQ_4bit-128g 2.3% (69403)
10. asmodeus 2.0% (61863)

Hordelib is back!

A month ago, I mentioned that hlky sent a bogus DMCA against hordelib which we fought against. I am glad to announce that this process finally completed and the hordelib is once again visible in Github and all contributions by hlky have been purged. I honestly hope that’s the last I’ll hear about this person…

Lemmy and Reddit

The main reason for being otherwise busy is that I’ve been furiously transferring my Reddit presence to my own self-hosted Lemmy instance, because Reddit is speed-running enshittification. I won’t bore you with the details but you can read up some of my work and see some of my development in the relevant blog tag.

However I do want to say that the instance I’ve fired up, the Divisions by zero has been more successful than I ever could have imagined! With ~10K users registered and thousands of subscribers to the communities, and some of the best admin team I could hope for.

Unfortunately there’s also been reddit drama which has been mightily distracting to me, but things are slowly settling down and I am putting my reddit days well behind me.

In relevance to the AI Horde however, we do have some cool communities you should subscribe to

• Stable Diffusion
• Stable Diffusion Art
• Bot Art

I am likewise already planning more events and automation to more closely tie the AI Horde into the Lemmy instance for cool art stuff! Stay tuned and/or throw me your ideas!

Prompt Challenges

R from our mod team has started running some cool prompt challenges in the discord server which you are all more than welcome to join! Winner gets a nice bundle of kudos, not to mention the amount you get by simply posting. It’s just fun all around, and the winners are featured in the Lemmy communities as well!

Worker Updates

Tazlin has been hard at work improving the AI Horde Worker with bugfixes (not to menton the huge amounts of tech support given in discord). The AI Horde Worker as a result has become much much more stable which should have a good impact on your kudos-per-hour! Just a quick shout-out to an invaluable collaborator!

A Lot more workers

I don’t know how it’s happening but the AI Horde is nearing 100 dreamers! I am getting the fireworks ready for the first time we hit this threshold!

Funding and Support

My current infrastructure has been sufficiently stable since the last migration to a dedicated host which I think you have experienced with the low amount of downtimes and interruptions since.

This is my usual opportunity to point out that running, improving and maintaining the the AI Horde is basically a full-time job for me so please if you see the open commons potential of this service, consider subscribing as my patron where I reward people with monthly priority for their support.

If you prefer, you can instead sponsor me on github.

Final Word

While development slowed significantly in June, we’re still doing significant work for the open commons. I have just not had the mental capacity to build up hype as much as I used to, and to make it worse, the social media landscape is completely in the air at the moment.

I am really hoping more people can step up and help promote the AI Horde and what it represents as my workload is just through the roof and to be perfectly honest, I am at the limit of my “plate-spinning” capabilities.

Please talk about the AI Horde and the tools in its ecosystem. The more people who know about it, the more valuable it becomes for the benefit of everyone!

We have plenty of ways one can help and we shower with kudos everyone doing so. From people sharing images and helping others in the community, to developers bug-fixing my terrible code, to community managers on discord and admins on lemmy. If you want to help out, let us know!

I’ve been meaning to write one of these posts every month, but the events since I wrote the last piece have been fairly disruptive. With the loss of our maintainers of nataili, we’ve been forced to put our heads down to alleviate the backend issue, and I didn’t want to write another “State of the AI Horde” until that business was completed. More about that later. First, let’s look at the basics.

(In case you’re new here and do not know what is the AI Horde: It is a crowdsourced free/libre software Open API which connects Generative AI clients to volunteer workers which provide the inference.)

More Requests, More Power!

The total amount of images generated has stayed relatively stable since the last month, with only ~12M images generated. However the total amount of Terapixelsteps is up to 4.2TPS compared to 3.7TPS last month. This shows that people are looking for more details at higher resolution instead of just grabbing more images. Unfortunately we can’t capture the impact of ControlNet as easily, but suffice to say, its demand is pretty significant.

On the LLM front, we’ve generated 3.5M texts, for a total of 374 Megatokens. We have effectively tripled the LLM output! This makes sense as we see plenty of traction in the LLM communities since Google Colab started banning Pygmalion.

Also, did you know Cubox from our discord community has gone and setup a whole public Grafana server to track our stats? Head over and check it out yourselves.

Top 10 Stable Diffusion models

No significant changes since last month in this chart, with Deliberate still proving its worth and further solidifying its position with a 25% use rate throughout the entire month. Stable Diffusion 1.5 sees plenty of use too.

Anime models seem to be losing some popularity across the board, and Dreamshaper kicked Abyss OrangeMix off the board so as to settle on a decent 5%. Good showing!

1. Deliberate 25.8% (3067246)
2. stable_diffusion 14.4% (1707729)
3. Anything Diffusion 9.0% (1067386)
4. Dreamshaper 5.0% (596651)
5. Realistic Vision 4.7% (563079)
6. URPM 3.6% (428415)
7. Hentai Diffusion 2.9% (348985)
8. ChilloutMix 2.6% (314301)
9. Project Unreal Engine 5 2.6% (307907)
10. Counterfeit 2.2% (260739)

Top 10 Text models

As is to be expected, Pygmalion 6b is still leading the charts with a significant margin, but its big brother, the recently released Pygmalion 7b, which is based on the Llama models, has already secured 3rd place and is only set to further cannibalize 3b’s position.

Erebus and Nerybus continue to fill up the rest of the board with a few new 4bit models starting to finally come into the fray.

1. PygmalionAI/pygmalion-6b 44.5% (1571109)
2. KoboldAI/OPT-13B-Erebus 9.8% (345983)
3. PygmalionAI/pygmalion-7b 5.2% (183413)
4. KoboldAI/OPT-2.7B-Nerybus-Mix 3.8% (135223)
5. KoboldAI/OPT-13B-Nerybus-Mix 3.8% (132564)
6. bittensor/subnet1 2.5% (86859)
7. Pygmalion-7b-4bit-32g-GPTQ-Safetensors 2.3% (79577)
8. gpt4-x-alpaca-13b-native-4bit-128g 2.0% (72358)
9. pygmalion-6b-gptq-4bit 1.8% (64750)
10. OPT-6.7B-Nerybus-Mix 1.8% (62300)

Image Ratings keep flowing in

Adding new ratings continue unabated but we’re still finding people trying to bypass our countermeasures and just automatically rate images to poison the dataset. Please don’t do that. If you want kudos, you can easily get more by simply asking in our discord instead of wasting our time 🙁

One interesting note is that the Stable UI has finally bypassed the Artbot on amount of images rated

 count  |        client        
--------+----------------------
   6270 | AAAI UI
 110826 | ArtBot
 279112 | Stable UI
   3197 | Unknown
   2700 | ZeldaFan Discord Bot

A total of 400K ratings over the past month. Very impressive.

I have also finally on-boarded the full DiffusionDB dataset of 12M images, into the ratings DB, we we’ll have enough images to rate for the foreseeable future.

A whole new backend

As I mentioned at the start, the big reason this state of the AI horde was delayed, was the need to switch to a completely new backend. It’s a big story, so if you want to read up on some details on that, do check out the devlog about it.

New Features and tweaks

Not much to announce since all our effort was in the backend switch, but a few things nevertheless

• We have added support for Shared Keys, so that you can share your account priority, without risk.
• A new kudos calculation model has been put live, instead of the manual way I was calculating kudos for each request based on magic numbers. The new kudos model is a Neural Network trained by Jug, and it takes into account the whole payload and figures out the time it would take to generate empirically.
• Kudos consumption has been removed for weights, and a tiny kudos tax of 1 kudos has been added per request to discourage splitting the same prompt into multiple separate requests.
• You can now change your worker whitelist into a worker blacklist so that you avoid specific workers, instead of requesting explicitly which ones you want.
• A new “Customizer” role has been added to allow you to host custom Stable Diffusion models. This is not possible on the Worker yet, but once it is, these people will be able to do it. Getting this role is a ping to the AI Horde moderators.
• We had to fight back another raid, on the LLM-side this time, which forced me to implement some more countermeasures. Scribes (i.e. LLM workers) cannot connect through VPN unless trusted, untrusted users can only onboard 3 workers, and there’s an worker-limit per IP now.

Discord Updates

I’ve been doing some work to improve the social aspects in our discord server, one of them is onboarding 3 new volunteer community managers through and hoping we can start doing more events and interactions with the community.

Another is the addition of a new channel per Integration I know of. I give the developers/maintainers of these services admin access to that channel so they can use it to provide support to their users, or redirect to their own discord servers if they prefer.

If you have a service integrating into the AI Horde API, please do let me know and I’ll be happy to open a new channel for you and give you admin access to it.

Infrastructure Improvements

Our demand is increasing significantly, as we have way more and more concurrent web sessions every day. Unfortunately a few database downtimes this month convinced me that hosting it in a Contabo VM with 30% CPU steal is not feasible anymore.

So I finally finalized switching the Database to a dedicated server. It’s a lot more expensive but massively worth it as our response times have increased 5-fold! A worker payload pop that used to take 1-3 seconds, now can take 0.2 – 1 second! As a result the whole AI horde should feel way snappier now.

I have also added our first dedicated box API front-end. Initially it didn’t look to be doing too good, having some of the worst performance, but once I switched to the dedicated database server, it suddenly became the fastest by far, making very obvious just how much impact latency has.

I have also finally deployed a central log server based on Graylog, which should help me track issues across all servers and look them up historically as well.

Funding and Support

All of the above now means the AI horde is significantly more expensive to run and it’s almost a second full-time job for me at this point. Currently all of this is funded via my patreon subscribers only but that is not scaling quite as fast as my infrastructure costs :-/

To add to this, the stability.ai support seems to have run dry and the GPUs they were providing to the AI Horde have been removed and I haven’t been able to arrange to bring them up again.

So I think I need to more consistently promote the way to help me sustain the AI Horde as a truly free and open API for everyone.

So, please if you see the potential of this service, consider subscribing as my patron where I reward people with monthly priority for their support.

If you prefer, you can instead sponsor me on github.

I am also looking for business sponsors who either see the value of a Crowdsourced Generative AI Inference REST API or might want to use it for themselves. I basically have no contacts in the AI world so I would appreciate people forwarding this info to whoever might be interested.

Final Word

The last month has been very difficult for the AI Horde but fortunately I’ve had the help of some really valuable contributors from both the AI Horde community as well as the KoboldAI community and we managed to pull through.

Now with our new backend I expect new features to come much faster and of more quality. We’re already working hard on LoRa support horde-wide for one, so stay tuned!