Tag: LAION

The image ratings are flooding in!

Db0

It’s been about 2 weeks since we deployed the ratings system to gather data for LAION and once the main UIs on-boarded them, they’ve been coming in at an extraordinary pace!

So I thought I’d share some numbers.

Amount of ratings per client

 count  |     client
--------+-----------------
 175060 | ArtBot
    461 | Discord Bot
    159 | Droom
   4124 | Lucid Creations
   2545 | Stable UI
   9430 | UnknownCode language: SQL (Structured Query Language) (sql)

As you can see, the Artbot ratings are crushing everything else, but to also be fair, Artbot was the first to integrate into the ratings system and is also a very popular UI. The Lucid Creations has added post-generation ratings, but not yet generic ratings. Stable UI and Discord Bot only added ratings a couple days ago. Things are about to pick up even more!

Artifacts per client

 count |     client
-------+-----------------
 15308 | ArtBot
     0 | Discord Bot
     0 | Droom
  1073 | Lucid Creations
  2550 | Stable UI
     2 | Unknown
(6 rows)

And we also can see how many artifact ratings each client has done. Artifacts were added only a couple days ago as well. Still ArtBot dominates, but only by a single order of magnitude instead of two 😀

Rated images count

 count | ratings_count
-------+---------------
 71197 |             1
  7860 |             2
 30140 |             3
  3644 |             4
    71 |             5
    11 |             6
     2 |             7
(7 rows)

 count
--------
 112928
(1 row)

The amount of images which have received at least 1 ratings is very heartwarming. We have 112K images rated with at least 1 rating, of which 30K have 3 rating each!

Total ratings count

 count
--------
 191914
(1 row)

This is just the raw amount of ratings submitted. Almost 200K between generic ratings and post-generation ratings in ~14 days. That is ~13K ratings per day! For free! Just because people want to support something good for the commons!

This is the power of building a community around mutual aid! People love telling others what they think of their work (just see how many ratings midjourney has received) and people also love supporting an endeavour which through FOSS and transparency, ensures they are not being taken advantage to enrich a few at the top!

This is exactly what my dream was for this project. To unite a community around helping each other, instead of benefiting a corporation at the top. And it is working better than I ever expected in this short amount of time!

Y’all are awesome!

Stable Horde receives stability.ai processing power!

Db0

A week ago I mentioned that we had begun a collaboration with LAION to provide them with ratings on images. The amount of ratings we have received since then has blown away all our expectations! In just a week, you’ve all rated close to 130.000 individual images! As a comparison, the LAION-aesthetics v2, which was instrumental for training Stable Diffusion v1.x, used less than 600K rated images. We’ve reached 1/4 of that amount in a week!

Needless to say, these amounts seemed to turn some heads to the power of mutual aid provided by the stable horde, and some gears were set in motion.

LAION spoke with stability.ai directly and arranged that it would likewise benefit them to support the health of the stable horde itself. Since stability.ai is set to be the most direct beneficiaries of a better trained the laion-aesthetics v3 it makes perfect sense.

I was not privy to the discussions that happened, but I was happy to learn that Tom, the CTO of stability.ai arranged to provide us with some sponsored resources in the form of 4 VMs with RTX4000s Nvidia GPUs!

Quite surprisingly I had to deploy the VMs myself, so I crafted the most optimal setup for taking advantage of those 8Gb of VRAM through my experience with my own RTX2070. Each of them has been loaded with standard stable_diffusion 1.5 and 2.1 and each of them then has 8-10 other finetuned models to help cover the versatility provided by the Stable Horde. Granted, we are serving close to 100 different models currently, but the fact that those workers will remain running consistently 24/7, should help provide cover and allow other workers to switch to less supported models as well.

I hope this is the start of a fruitful collaboration between the stability.ai and the Stable Horde. The way I see it, the current scenario is a win-win for everyone. We get a more consistent service which allows more people to use it and makes them more likely to rate images to give back, which are then fed back to LAION and by extension stability.ai.

A collaboration begins between Stable Horde and LAION!

Db01 Comment

last week I wrote how we started creating a new dataset of stable horde images to provide to LAION. Today I am proud to announce that we have further deepened our collaboration by setting up a mechanism which will allow the Stable Horde community to contribute dataset aesthetic ratings for LAION datasets!

Me along with hlky from Sygil.dev have used the last weekend to deploy a new service which allows us to aesthetically rate images from LAION’s multiple datasets. We deployed an API and thus allowed any client to interact with it. You can read the details of how it works on the blog I linked above, so I’m not going to repeat everything.

This is exciting for me because the Stable Horde has suffered from a distinct lack of visibility. None of the major AI-focused media (newsletters, YouTubers etc) have mentioned us to date. The very first coverage we got was from a PC magazine!

All that is to say that it’s been an uphill struggle to get the Stable Horde noticed in a way that will lead to more workers which will allow us to democratize access to AI for everyone. So I am very happy to pivot the amazing stable horde community in such a positive work which will bring more attention to what we’re trying to achieve.

We are still hard at work tweaking the information we store for each rating. For example we store the amount of images they had generated at the time of the rating, which will allow researches to filter out potentially spammy users.

We are also adding more and more countermeasures, as there’s always the fear that someone will just script random ratings to get kudos. Even though the Stable Horde is free to use without kudos and even though kudos has no value, people do strange things to see “numba go up”. Now I don’t particularly care if people harvest kudos like this, but I very well do care about our ratings being poisoned by garbage.

So if you’re someone who wants to make an exploit script to harvest kudos via ratings, please just join our discord instead. The kudos flow like candy when you’re active! And you will also not be harming the AI community itself.

Already our exported dataset has grown to 80K shared images. We have 20K ratings on the LAION datasets within 2 days. For comparison some of the biggest rated datasets have just 175K ratings which were done by paid workers (and we all know how motivated they are to be accurate). Our kudos incentives and community passion to improve AI is surprising even my wildest expectations to be honest!

Here’s to making the best damn dataset that exists!

Sharing is Caring

Db01 Comment

For a while now I’ve been discussing with LAION on a way to use the power of the horde to help them in some fashion. After coordination with hlky from the Sygil.dev crew, I decided to provide an opt-in mechanism for people to store their text2img stable diffusion generations in an alternative storage bucket. This bucket in turn will be provided to LAION so that they can use it for aesthetic training, or for other similar purposes.

So today I finally released this new mode. For clients it’s a simple flag during the payload. Set “share” to True, and your request will be uploaded to the specific storage bucket. This will also save me infrastructure costs as I will not have to pay for the storage out of my own pocket for these.

To give a further incentive for people to turn this on, and because I wanted a way to show the cost of running the horde, I have also implemented a “horde tax” kudos burn. Every time you generate an image, the overall kudos cost is then increased by 3. This signifies the overall resource cost of passing through the horde, such as bandwidth, i/o and storage. However, if you opt to turn on the sharing switch, the overall “tax” is just +1 kudos.

You might ask, why not make the cost proportional to the overall kudos cost. Something like +30%/+10%. The reason is that the overall kudos cost is dependent on how difficult it is for the workers to generate that image. From the perspective of the horde, a 512x512x50 image is not much different from a 1024x1024x300 image, even though the latter would take an order of magnitude more time to generate.

In fact, many small requests are technically worse for the horde infrastructure costs, than a few small ones. It’s not that I want to discourage the small ones though, because they are actually good for the horde workers (and thus the overall generation speed). Therefore the “tax” is fairly trivial in the grand scheme of things. Just a bit of extra “burn”.

One important thing to note however, is that anonymous accounts image generations are always shared. This is part of my general strategy where I want to discourage anonymous use of the horde. It just more difficult to manage the load when people are using it like that. This is why anonymous has the lowest priority and the most restrictions. And now they will always help provide data to LAION as well.

Finally, img2img and inpainting requests are never shared. This is because those are based on existing images and I cannot know if someone used some personal photo at low strength or something. So I prefer to err on the side of caution.

This is not the last support the AI Horde plans to give to LAION either. We are already working on new features like an aesthetic rating trainer and so on. I hope this sort of assistance can be put to good use for the benefit of all humanity!