Tag: NLNet

LCM and multiple versions of LoRas on the AI Horde

Db01 Comment

Finally 2024 is here and this allowed me a bit of free time to work on some of my NLNet tasks. The first thing on my list to tackle was adding LCM support on the AI Horde as it provides massively reduced steps, which for a crowdsourced service like ours, it makes all the difference in how much we can deliver.

For those who don’t know, LCM is a new breakthrough in Stable Diffusion that allows to “finetune” the model in such a way where an image can be generated using 10% of the steps previously required. So an image which would require 30 steps to converge, now needs just 3! That is a massive boost for lower-range GPUs. For high range GPUs, it starts avenues such as video generation as an image can happen at millisecond speeds!

Given the benefits, I wanted to work on this as soon as possible, and given the flexibility of the FOSS GenAI technology enthusiasts, we already had a great way to use LCMs, by using LoRas to turn any SD model into an LCM version.

However there was a snag. You see while the AI Horde already supports all LoRas on CivitAI, we never supported different versions of each, as we never expected anyone would want to use more than the latest. Unfortunately people on CivitAI started using the versioning system as “alternative” versions. And the LCM LoRa was using the same approach, where there was a version for each different sampler.

So the first order of business had to be to allow the AI horde to understand and support all LoRa versions of each LoRa! This took the better part of a full work-week of development and debugging, and then another week of troubleshooting and fixing in beta.

The good news is that this lead to us also identifying and squashing a very frustrating long-running bug where workers would rarely return previous images they’d generated instead of the ones requested. Getting someone else’s image is something we definitely don’t want to ever happen so we’re very happy we figured it out.

With that out of the way, I simply had to update the AI Horde itself to be able to handle the payload for specific LoRa versions, and then add support for the LCM sampler and then some ways to urge users to switch to it.

If you’re an AI Horde integrator, we strongly suggest you change your default settings to utilize LCM LoRas in your generations. You can get them from the same API you receive the model details, under the modelVersions key. To use them, you need to send the exact version ID as a string (found in modelVersions[#]['id]) this won’t accept a version name. You will also need to set is_version: true for the LoRa payload. This will tell the worker to look for a version instead of a LoRa ID.

Sending the LoRa name or ID will continue working as usual, grabbing the latest version (modelVersions[0]) from that list, so you existing implementations should continue working as usual.

Also we recently added AlbedoXL in our model list, to provide a better baseline for SDXL generations than basic SDXL 1.0 which requires a refiner to work. Using Albedo you can get generations that do not require a refiner in your workflow at all and get much less “fuzzy” generations in the process!

AI Horde to receive NLNet grant!

Db02 Comments

Back in July I first discovered NLNet and decided to apply for their NGI Zero Core grant to help me help me continue developing the AI Horde. Today I’m excited to announce that the AI Horde has officially been greenlit as one of the projects which will receive the August 2023 grant!

You can see the entry for AI Horde on NLNet here: https://nlnet.nl/project/AI-Horde/

AI Horde has been a passion project from the start but it’s difficult to maintain the level of intensity I had for such a length of time. With my patreon funding significantly dwindling month-to-month, multiple of our backend developers dropping, and the need for me to also keep up with some of my real life duties as well as my other FOSS projects, development of the AI Horde has sadly slowed in recent months. While It rmains extraordinarily stable, it has had little to announce, and some new features are slower to “cook” and reliant in the work of valuable backend volunteers like Tazlin.

I am hoping that the addition of NLnet funding should help reintroduce some of that momentum as the release of the grant is contingent on specific milestones being reached.

I have plans for 5 roughly grouped areas of development, for which I have thought of various tasks to receive “bounties” for in the scope of this grant: AI Horde, Dreamer, Scribe, Alchemist and Godot Engine

The AI Horde tasks will focus on improving the middleware itself. Such as extending the shared key functionalities, improving the API documentation etc.

The Dreamer, Scribes and Alchemists tasks will focus on adding new functionalities to the the official workers, such as more generation workflows, batch processing etc.

Finally the Godot Engine tasks will improve the existing toolset of the AI Horde to support game development, such as improving my AI Horde Client, migrating it and Lucid Creations it to Godot Engine 4, etc.

The good news is that due to the way NLNet works, I had to submit stuff to work on, but I couldn’t work on them before the AI Horde was officially accepted (or rather, I could, but I couldn’t receive a “bounty” for them). Now that this is locked-in, I can start working on some of these with the added incentive of getting a reward at the end which can alleviate some of my ADHD executive disfunction. So if all goes well, you should start seeing more activity from me soon.

As always, if you want to support the AI Horde and my work in FOSS and the open commons, please do consider funding me at:

These funds go towards paying for the existing infrastructure first, and motivation continuous development second.

PS: Interestingly enough, it’s my birthday today too 😀