Tag: fediverse

How did we move from forums to Reddit, Facebook groups, and Discord?

Db01 Comment

From the first moment I first went online in 1996, forums were the main place to hang out. In fact the very first thing I did was join an online forum run by the Greek magazine “PC Master” so I could directly to my favourite game reviewers (for me it was Tsourinakis, for those old enough to remember).

Whoever didn’t like the real-time nature of the IRC livechat, forums were all the rage and I admit they had a wonderful charm for the upcoming teenager who wanted to express themselves with fancy signatures and some name recognition for their antics. Each forum was a wonderful microcosm, a little community of people with a similar hobby and/or mind-frame.

BBcode-style forums took the web 1.0 internet by storm and I remember I had to juggle dozens of accounts, one for for each one I was interacting with. Basically, one for each video game (or video game publisher) I was playing, plus some Linux distros, hobbies, politics and the like. It was a wonderful mess.

But a mess it was, and if the dozens of accounts and constant context switching barely enough to handle for an PC nerd like myself, I can only imagine how impenetrable it was for the less tech-savvy. Of course, for people like me this was an added benefit, since it kept the “normies” out and avoided the “Eternal September” in our little communities.

However the demand for places accessible for everyone to discuss was not missing, it was just unfulfilled. So as soon as Web 2.0 took over with the massive walled gardens of MySpace, Facebook, Twitter and so on, that demand manifested and the ability for anyone to create and run a forum within those spaces regardless of technical competency or BBcode knowledge, spawned thousands of little communities.

Soon after Digg and then Reddit came out, and after the self-inflicted implosion of Digg, Reddit along with Facebook became the de-facto spot to create and nurture new async-discussion communities, once they added the functionality for everyone to create one and run it as they wanted.

But the previously existing BBcode forums still existed and were very well established. Places like Something Awful had such strong communities that they resisted the pull of these corporate walled gardens for a long time. But eventually, they all more or less succumbed to the pressure and their members had an exodus. What happened?

I’m not a researcher, but I was there from the start and I saw the same process play out multiple times in the old forums I used to be in. Accessibility and convenience won.

There’s a few things I attribute this to.

  1. The executive costs to create a new forum account is very high. Every time you want to join one, you need to go through making a username (often trying to find one that’s not taken, so now you have to juggle multiple usernames as well), new password, captchas, email verifications, application forms, review periods, lurker wait times and so on. It’s a whole thing and it’s frustrating to do every time. Even for someone like me who has gone through this process multiple times, I would internally groan for having to do it all over again.
  2. Keeping up to date was a lot of work. Every time I wanted to keep up to date with all my topics, I had to open new tabs for each of my forums and look at what’s new is going on. The fact that most of the forums didn’t have threaded discussions and just floated old discussions with new replies to the top didn’t help at all (“thread necromancy” was a big netiquette faux-pas). Eventually most forums added RSS feeds, but not only were most people not technical enough to utilize RSS efficiently (even I struggled), but often the RSS was not implemented in a way that was efficient to use.
  3. Discoverability was too onerous. Because of (1) Many people preferred to just hang out in one massive forum, and just beg or demand new forum topics to be added for their interests so they wouldn’t have to register, or learn other forum software and interact with foreign communities. This is how massive “anything goes” forums like Something Awful started, and this also started impacting other massive forums like RPGnet who slowly but surely expanded to many more topics. Hell almost every forum I remember has politics and/or “out of topic” sections for people to talk without disrupting the main topics because people couldn’t stop themselves.
    And where the forum admins didn’t open new subject areas, the bottom-up pressure demanded that solutions be invented in the current paradigm. This is how you ended up with immortal threads, thousands of pages deep for one subject, or regular mega-threads and so on. Internet life found a way.
  4. Forum admins and staff were the same petty dictators they always were and always will be. Personality cults and good ole boys clubs abounded. People were established and woe to anyone who didn’t know enough to respect it, goddammit! I run into such situations more than once, even blogged about it back in the day. But it was an expected part of the setup, so people tolerated it because, well what else will you do? Run your own forum? Who has the time and knowledge for that? And even if you did, would anyone even join you?

And so, this was the paradigm we all lived in. People just declared this is how it had to be and never considered any proper interactivity between forums as worth the effort. In fact, one would be heavily ridiculed and shunned for even suggesting such blasphemous concepts

That is, until Facebook and Reddit made it possible for everyone to run their own little fief and upended everything we knew. By adding forum functionality into a central location, and then allowing everyone to create one for any topic, they immediately solved so many of these issues.

  1. The executive cost to join a new topic is very low. One already has an account on Reddit and/or Facebook. All they have to do is press a button on the subreddit, group they want to join. At worst they might need to pass an approval, but they get to keep the same account, password and so on. Sure you might need to juggle 1-3 accounts for your main spaces (Reddit, Facebook, Discord), but that’s so much easier than 12 or more.
  2. Keeping up to date is built-in. Reddit subscriptions allows one a personalized homepage, Facebook just gives you your own feed, discord shows you where there’s activity and so on. Of course the corporate enshittification of those services means that you’re getting more and more ads along masquerading as actual content and invisible algorithms are feeding you ragebait and fearbait to get you to keep interacting at the cost of your mental and social health, but that is invisible for most users so it doesn’t turn them off.
  3. Discoverability is easy. Facebook randomly might show you content from groups you’re not in, shared by others. Reddit’s /r/all feed showed posts from topics you might not even know existed and people are quick to link to relevant subreddits. Every project has its own discord server link and so on.

The fourth forum problem of course was and can never be solved. There will always be sad little kings of small sad little hills. However solving 1-3 meant that the power of those abusing their power as moderators was massively diminished as one could just set up a new forum in a couple of minutes and if there was enough power abuse, whole communities would abandon the old space and move to the new one. This wasn’t perfect of course, as in Reddit, only one person could squat one specific subreddit, but as seen with successful transitions from /r/marijuana to /r/trees, given enough blow-back, it can certainly be achieved.

And the final cherry on top is that places like Reddit and discord are just…easier to use. Ain’t nobody who likes learning or using BBcode on 20-year-old software. Markdown became the norm for a reason due to how natural it is to use. Add to that less restrictions on uploads (file size, image size etc) and fancier interfaces with threaded discussions, emoji reactions and so on, and you get a lot of people using the service instead of trying to use the service. There are of course newer and better forum software like the excellent Discourse, but sadly that came in a bit too late to change momentum.

So while forums never went away, people just stopped using them, first slowly but accelerating as time passed. People banned just wouldn’t bother to create new accounts all over again when they already had a Facebook account. People who wanted to discuss a new topic wouldn’t bother with immortal mega-threads when they could just join or make a subreddit instead. It was a slow-burn that was impossible to stop once started.

10-15 years after Reddit started, it was all but over for forums. Now when someone wants to discuss a new topic, they don’t bother to even google for an appropriate forum (not that terminally enshittified search engines would find one anyway). They just search Reddit or Facebook, or ask in their discord servers for a link.

I admit, I was an immediate convert since Reddit added custom communities. I created and/or run some big ones back in the day, because I was naive about the corporate nature of Reddit and thought it was “one of the good ones”, even though I had already abandoned Facebook much earlier. It was just so much easier to use one Reddit account and have it as my internet homepage, especially once gReader was killed by Google.

But of course, as these things go, the big corporate gardens couldn’t avoid their nature and eventually once the old web forums were abandoned for good and people had no real alternatives, they started squeezing. What are you gonna do? Set up your own Reddit? Who has the time and knowledge for that? And even if you did, would anyone even join you?

Nowadays, I hear a lot of people say that the alternative to these massive services is to go back to old-school forums. My peeps, that is absurd. Nobody wants to go back to that clusterfuck I just described. The grognards who suggest this are either some of the lucky ones who used to be in the “in-crowd” in some big forums and miss the community and power they had, or they are so scarred by having to work in that paradigm, that they practically feel more comfortable in it.

No the answer is not anymore an archipelago of little fiefdoms. 1-3 forbid it! If we want to escape the greedy little fingers of u/spez and Zuckeberg, the only reasonable solution is moving forward is activitypub federated software.

We have already lemmy, piefed, and mbin, who already fulfill the role of forums, where everyone can run their own community, while at the same time solving for 1-3 above! Even Discourse understood this and started adding apub integration (although I think they should be focusing on threadiverse interoperability rather than not microblogging.)

Imagine a massive old-school forum like RPGnet migrating to a federated software and immediately allow their massive community access to the rest of the threadiverse without having to go through new accounts and so on, while everyone else gets access to the treasure trove of discussions and reviews they have. It’s a win-win for everyone and a loss for the profiteers of our social media presence.

Not only do federated forums solve for the pain points I described above, but they add a lot of other advantages as well. For example we now have way less single points of failure, as the abandonment of a federated instance doesn’t lose its content which continues living in the caches of the others who knew about it and makes it much easier for people to migrate from one lemmy instance to another due to common software and import/export functionalities. There’s a lot of other benefits, like common sysadmin support channels, support services like fediseer and so on.

These days, I see federated forums as the only way forward and I’m optimistic of the path forward. I think Reddit is a dead site running and the only way they have to go is down. I know we have our own challenges to face, but I place far more trust in the FOSS commons than I do in corporate overlords.

Everything Haidra touches

Db0

The second fediverse canvas event just concluded and I’m very happy how this turned out. In case you don’t know what this is. Check out this post and then take your time to go and explore the second canvas in depth before it’s taken down, and look for all the interesting and sometimes even hidden pieces of pixel art.

This time I had a more interesting idea to participate. I decided to draw the Haidra Org logo. I didn’t expect a massive support, but was pleasantly surprised with how many people joined in to help create it after my initial post about it and my announcement on the AI Horde discord server. Some frontends like horde-ng even linked to it with an announcement.

Almost as soon as it started, we ended up conflicting in our placement with someone who was drawing a little forest on just below and to our left. I decided that they can have the foreground since we had plenty of space available which avoided any fighting over pixels. All in all, we managed to complete it within half a day or so which is pretty cool I like to think and we even got a small “garden” so to speak.

The final form of the Haidra drawing, including the little forest below and two Stus

Afterwards I thought it would be interesting to have the Haidra tendrils “touch” various points of importance or sprites that I like. I decided to extend out as if we’re made of water and a lot of other “canvaseers” joined in to help which I found really sweet.

First we extended towards the (then) center of the canvas (top left on the featured image above), passing next to the Godot logo, below OSU and finally reached the explosion of the beams. That took most of the first day but people were still pretty active, even though the infrastructure of the event had already started buckling under its own success.

Fortunately as we could “flow” like water and even “go under” other pixelart, we didn’t encounter any resistance in our journey, and a lot of people gave us a helping hand as well.

Once this was achieved on a whim, I decided to double down on the “river” similaity, and drew a little 17px pirate ship to show our roots and went to bed. When I woke up next morning, I was surprised to discover a Kraken was attacking it making a really cool little display of collaborative minimalistic art.

Haidra pirate ship fighting a Kraken

This kind of thing is why I love events like these. I love emergent stuff like these and seeing people putting the own little touches on what other started is awesome!

The next day the canvas had extended to be double in size and so a whole new area to the right was available, I had already noticed someone had created a little pirate banner towards the new canvas center, but it was alone and sad. So I decided we should try to give it a little bit of that Haidra embrace. So a long journey started with a new tendril to reach it. I had a rough idea of the path to follow as the direct route was blocked, but as soon as other started adding to it, it almost took a life of its own on its journey.

Eventually, towards the middle of the second day we reached it, passing under Belgium, through some letters and crossing the big under-construction trans flag before going over piracy, before I spawned yet another pirate ship before waterfalling down onto the mushroom house.

The path to piracy

At this point, the whole event took a dramatic turn as the performance problems had become so severe, that the admin decided to take the whole thing down to fix them, rather than let people get frustrated. This took half a dozen hours or so, and even though the event was extended by 24 hours to make up for it, the event momentum was kneecapped as well.

Once the canvas was back up for the third day, the next objective I had was a much longer journey to try and touch The Void that was extending from the top right. When I started, the path was still mostly empty, but as we moved towards it, the canvas became more more congested, forcing us to take some creative detours to avoid messing with other art.

All in all, we flowed over the Factorio cog, creating a little lake and spawning a rubber duckie in the process. Then through the second half of the trans flag, which caused a minor edit war, as the canvaseers thought we were vandalizing. Then the way up and over the massive English flag was sorta blocked, so we had to take a detour and slither between the Pokemon to its left first.

Until finally we reached the top of the English flag, where I took a little creative detour to draw a little naval battle. My plan was to have an English brigantine fighting with two pirate sloops, but as soon as I finished it, other jumped in with their own plans. First one of my pirate ships revealed itself as a Spanish privateer instead (which I suspect was a reference to the recent football events). And then over the course of the next two days, the three ships kept changing allegiances every couple of hours. Quite the little mini-story to see unfold.

Finally we were almost at our final objective, only to discover that our final objective was not there anymore. The Void had been thoroughly contained and blocked by a massive cat butler (catler?). The only thing left to touch, was a single solitary void tendril on the top. Surprisingly, as soon as we reached it, it livened and flourished into life, which was certainly not my original idea, but I went with it happily.

Having achieved all I wanted to do, and with the event (and the day) drawing to a close, I decided there’s no point setting any more goals and just left those interested start extending Haidra on a whim. You can see my final post here, which also links to all my previous posts, which also contain some historic canvas images, showing the actual state of the board at the time of the posting.

All in all, I had a lot of fun, and enjoyed this way more than Reddit /r/place which is botted to hell and back, making contributions by individual humans practically meaningless. Due to the lack of significant botting, not only was one’s own pixels more impactful, but humans tended to mostly collaborate instead of having scripts mindlessly enforcing a template. This ended with a much more creative canvas, as people worked off others ideas and themes, and where there was conflict, a lot of the time a compromise solution was discovered where both pieces of art could co-exist.

The conflict points tended to be political, as it so often happens. For example the Hexbears constantly trying to make the Nato flag into a swastika, or some effectively people rehashing the conflict around the Israel colonization of Palestine in pixel conflict form.

Some other things of interest:

  • I mentioned that the Spanish seem to have boarded and overtaken my pirate ship, and someone drew a little vertical ship coming up the stream for reinforcements. ❤️
  • Stus and AmongUs everywhere, sometimes in negative space, or only visible in the heatmap. Can you find them all?
  • The Void getting absolutely bodied when it tried to be destructive, but being allowed to extend a lot more when they actually played nice with other creations.
  • The amount of My Little Pony art is too damn high!
  • Pleasantly little national flag jingoism on display!
  • A very healthy amount of anarchist art and concepts and symbols. Well done mates! Ⓐ

See you next year!

Can we improve the Fediverse Allow-List Model?

Db06 Comments

Looks like someone really kicked the hornet’s nest recently on mastodon by announcing (not even deploying) a Mastodon-BlueSky bridge. Just take a look at the github comments here to get an idea of how this was received.

Plenty of people way more experienced than myself have weighted on this issue so I don’t feel the need to leave my two cents. However I wanted to talk about a very common counter-argument made towards those who do not want such bridges to exist. Namely, that Fediverse already provides the tools towards not having such a bridge be an issue: The allow-list model.

The idea being that if your ActivityPub server by default rejects all federation except towards trusted instances, then such bridges pose no problems whatsoever. The bridge (and any potential undercover APub scrappers) would not be able to get to your instance anyway.

Naturally, the counterargument is that this is way too limiting to one’s reach, and they shouldn’t be forced into isolation like this. Unfortunately the alternative here appears to try and scold others into submission, and this is unlikely to be long term solution. Eventually the Eternal September will come to the Fediverse. If you spent the past few years relying on peer pressure to enforce social norms, then the influx of people who do not share your values is going to make that tactic moot.

In fact, we can already see the pushback to the scolding tactics unfolding right now.

The solution then has to be a way to improve the way we handle such scenarios. Improve the tooling and our tactics so that such bridges and scrappers cannot be an issue.

A lot of the frustration I feel also comes down to the limited set of tools provided by Mastodon and other Fediverse services. A lot of the time, the improvement of tooling is stubbornly refused by the privileged core developers who don’t feel the need to support the needs of the marginalized communities. But that doesn’t mean the tooling couldn’t be expanded to be more flexible.

So let’s think about the Allow-List model for a moment. The biggest issue of an Allow-List is not necessarily that the origin server restricts themselves from the discussion. In fact they’re probably perfectly happy with that. The problem is that if this became the norm, it massively restricts the biggest strength of the Fediverse, which is for anyone to create and run their own server.

If I make a new server and most of everyone I want to interact with is in Allow-List mode, how do I even get in? We then have to start creating informal communication channels where one has to apply to join the allow-circle. Such processes have way too many drawbacks to list, such as naturally marginalizing Neurodivergent people with Rejection Sensitivity Dysphoria, balkanizing the Fediverse, empowering whisper networks and so on.

I want to instead suggest an alternative hybrid approach: The Feeler network. (provisional name)

The idea is thus: You have your well protected servers in Allow-List mode. These are the servers which require protection from constant harassment when their posts are spread publicly. These servers have a few “Feeler” instances they trust on their allow-list. Those servers in turn do not have an allow-mode turned on, but rely on blocklist like usual. Their users would be those privileged enough to be able to handle the occasional abuse or troll coming their way before blocking them.

So far so good. Nothing changes here. However what if those Feeler servers could also use the wider reach to see which instances are cool and announce that to their trusted servers? So a new instance appears in your federation. You, as a Feeler server, interact with them for a bit and nothing suspicious happens, and their users seem all to be ideologically aligned enough. You then add them into a public “endorsed list”. Now all the servers in your trust circle who are in allow-mode see this endorsement and automatically add them to their allow-lists. Bam! Problem solved. New servers have a way to be seen and eventually come into reach with Allow-List instances through a sort of organic probation period, and allow-listed servers can keep expanding their reach without private communications, and arduous application processes.

Now you might argue: “Hey Db0, yes my feelers can see my allow-list server posts, but if they boost them, now anyone can see them, and now they will be bridged to bluesky and I’m back in a bad spot!”

Yes this is possible, but also technically solvable. All we need to do is to make the Feeler servers only federate boosted posts from servers in allow-mode, to the servers that the ones in the allow-list already allow. So let’s say Server T1 and T2 are instances in allow-list mode which trust each other. Server F1 is a Feeler server trusted by T1 and T2. Server S1 is an external instance that is not blocked by F1, but not yet endorsed either. User in F1 boosts a post from T1. Normally a user in S1 would see that post by following that user. All we need to do is to change the software so that if F1 boosts a post from T1, the boost would only federate towards T2 and other instances in T1’s allow-list, instead of everyone. Sure this would require a bit more boost complexity, but it’s nothing impossible. Let’s call this “protected boost”.

Of course, this would require all Apub software to expose an “Endorsement” list for this to work. This is where the big difficulty comes from, as you now have to herd the cats that are the multitude of APub developers to add new functionality. Fortunately, this is where tools like the Fediseer can cover for the lack of development, or outright rejection by your software developer. The Fediseer already provides endorsement functionality along with a full REST API, so you can already implement this Feeler functionality by a few simple scripts!

The “protected boost” mode would require mastodon developers to do some work of course, as that relies in the software internals which cannot be easily hacked by server admins. But this too can potentially just be a patch to the software that only Feeler-admins would need to run.

The best part of this approach is that it doesn’t require any communication whatsoever. All it needs is for the “Feeler” admins to be actively curating their endorsements (either on the Fediseer, or locally if it’s ever added to the SW). Then all allow-list server has to do is choose which Feelers they trust and “subscribe” to their endorsement list for their own allow-list. And of course, they can synchronize or expand their allow-list further as they wish. This approach naturally makes the distributed nature of the Fediverse into a strength, instead of a weakness!

Now personally, I’m a big proponent of the “human touch” in social networks, so I feel that endorsement lists should be a manual mechanism. But if you want to take this to the next level, you could also easily set up a mechanism where newly discovered instances would automatically pass into your endorsement list after X weeks/months of interaction with your user without reports and X-amount of likes or whatever. Assuming admins on-point, this could make widely Feeler servers as a trusted gateway into a well protected space on the fedi, where bad actors would find it extraordinarily difficult to infiltrate, regardless of how many instances they spawn. And it this network would still keep increasing each reach constantly, without adding an extraordinary amount of load to its admins.

Barring the “protected boost” mode, this concept is already possible through the Fediseer. The scripts to do this work already exist as well. All it requires is for people to attempt to use it and see how it functions!

Do point out pitfalls you foresee in this approach and we can discuss how to potentially address them.

Rebuttals on the Fediseer

Db02 Comments

I noticed recently that a few instances are just issuing counter-censures on the Fediseer just to be able to reply to the censures they received from others. While I get the need to say your piece, I didn’t like this utilization. So I wanted to provide an official way for instances to reply to censures and hesitations they received.

So now we’ve deployed Rebuttals. A Rebuttal is a “reply” to a censure or hesitation from another instance. If you have received both, the same rebuttal applies. This is set up this way so that rebuttals are not tied to any specific censure/hesitation and therefore being deleted when those are. If someone deletes and re-adds their censure against your instance, the same rebuttal will re-appear.

As always, remember there’s no hate speech allowed on the Fediseer, so make sure your rebuttals stay cool.

Also keep in mind the Fediseer is not a forum. There won’t be multi-threaded discussions going on. The expectation is that you can use the Rebuttals to explain why a censure is bogus and whatnot. Not to maintain flamewars.

I know there’s plenty of beefs on the Fediverse. I’m hoping with this feature won’t become fuel for them, but rather a way for everyone to feel they have a say in a neutral ground. I hope this can serve as a de-escalation as well. Sometimes an instance might receive a hesitation or censure from someone they don’t dislike, due to a misunderstanding. This will allow them to try and counter that, without having to counter-censure.

Let me know what you think in the comments below, by replying to this post on mastodon, or in the lemmy community.

Btw, since I have you here, did you know that you can support the hosting and development of the Fediseer? Currently I’m paying the hosting costs out of pocket. It’s not a lot but it would be nice if the infrastructure costs were self-sustaining. So if you find value in this service, feel free to throw some money at the development team.

Announcing: Pythonseer – A Fediseer SDK for Python

Db0

With the recent updates to the Fediseer, it is now possible to use it more efficiently for programmatically adjusing one’s blocklist using the built-in censures, so I wanted to add this capability to the Divisions by zero by updating my update_blacklist.py script to utilize this method.

While I could have done this by using simple http requests, I thought it was enough features there that creating a python SDK package around the fediseer API would be worthwhile. So I’m excited to announce the release of pythonseer!

It is built very similar to Pythörhead, so if you’re already using that in your scripts, it should be trivial to add support for the Fediseer via pythonseer as well.

I have already updated the provided script to automatically maintain your blocklists to allow it to make use of the fediseer censures as well. Simply censure the domains you would like to defederate from, and the script will automatically add them to your list along with the suspicious instances. If you have the script to run on a schedule, you can then simply add new censures on the fediseer API and your blocklist will be automatically kept up to date.

Likewise, if you don’t want to maintain all the blocklist yourself, find other like-minded instances and combine their censures with yours. This way whenever they censure someone, you also benefit.

Enterprising fediverse admins could also do other fancy things with the pythonseer, such as creating polls to democratically add instances to their endorsements or censures etc. So do let me know if you come up with new ideas!

And don’t forget to subscribe to the fediseer community on lemmy!

Fediseer multi-domain Endorsement and Censures

Db01 Comment

Fediseer has now been updated with 2 new features

Multi-domain endorsement lists

One can now request all endorsements from multiple domains:

This allows instances to quickly discover a “list of friends” based on other instances. Use cases for this might include scripts which auto-approve comments in moderation, or automatically update a fediverse instance’s whitelist based on common endorsements.

Censures

The current fediseer guarantees are meant to apply only to consideration of spam. As such we do not have a way to mark instance that many would consider terrible in all other ways except spam (e.g. pro-nazi instance, or an instance allowing loli content) as such.

To solve this a new feature has been added: Censures

An instance can now apply a censure to any other instance domain (whether it’s guaranteed already or not) for any reason. This extreme disapproval can come from any subjective reason, but like endorsements it doesn’t, by itself, have any mechanical impact.

In fact, because endorsements and censures are explicitly subjective, I have taken the decision to not display censure counts on instance details to avoid people using them for “objectively” rating instances which is not their intended purpose. This is because one’s instance could easily be censured by a lot of, say, fascist instances, but this should have no overall impact on how non-fascists percive that instance.

Instead, one can see combined censure lists from multiple instances, like one can see combined endorsements. You simply pass a comma-separated list of domains to the /api/v1/censures_given endpoint and you get a list of all the instances they have censured collectively.

This endpoint can then be consumed to make collective blocklists among instances that trust each other, without one having to manually discover and parse different instance federation blocklists all the time.

Likewise, by not being explicitly tied to a blocklist, the censure list could also be used to enforce a softer approach. For example by having an auto-moderator script which flags comments from censured instances for manual review, etc. This could allow an instance to retain a less-restrictive blocklist, but still allow for more rapid response to users coming from potentially problematic instances.

As always, the point of the fediseer is to allow a way to provide the information easily, rather than to dictate how to use it. I excited to see in which ways people will utilize the new abilities.

Fediseer Streamlining

Db0

Just one day ago I released my initial release of the Overseer and I was annoyed by the implementation almost immediately. The requirement to register on another Lemmy instance using a custom username and wait for manual approval (which could also lead to someone sniping that username, and forcing me to manually delete it), and THEN register your instance on the Overseer was just too clunky.

While a few people did register, I realized almost immediately it’s very likely this would never take off. So I started rethinking how I could streamline this process so that it would require far less steps.

My initial plan was to simply register all available instances on the fediverse by default, and allow admins to claim them later. That would require me somehow being able to contact those admins directly. This led me to try to figure out the best way to do that where I discovered I could theoretically talk to any fediverse instance directly, and not have to rely on the a specific lemmy instance with all its limitations.

So I spent many many hours figuring out how to do that. The documentation is frustratingly sparse and incomplete on this point, with my best guide was this blogpost, from half a decade ago for a different language than python. Fortunately this day and age, I had access to ChatGPT, so I could ask it to translate the code on that page to python and then with some trial and error and digging in the official documentation, I had a working DM system for mastodon!

Then I set out to do the same for lemmy which is where the big frustration was waiting, as the documentation is practically non-existent, the messages are cryptic and Rust and the way it’s coded was completely impenetrable to me. Lots and lots of digging in the code, trial and error and asking around in desperation, and I managed to figure out that the main blocking point was a “type” key in my activitypub instance, instead of a fault in my payload.

Unfortunately figuring out how to “speak activitypub” took me the better part of my day. But the good news is that once I had that down, the rest was just a matter of time. I just had to refactor everything. And hell, while I am doing that, why not change the name and domain as well

And that’s how Fediseer.com is born!

I have already updated my previous post with the new workflow, but the big difference is that it completely removes the need for an extra lemmy instance one has to manually register. Instead one just has to “claim” their instance and they will get a PM directly in their mailbox!

Likewise, you can guarantee for a different instance even if their admins haven’t claimed it first and they will get a helpful PM informing them about it. If the instance doesn’t exist yet, all you need to do is search for it in the whitelist and it will be automatically added, and then can be guaranteed. My next step is to automatically import all known instances by pulling them from the federation, which should avoid the manual step of searching for them first.

So let’s see how it goes. Now with the ability to talk to fediverse instances directly, it also opens up some really fascinating doors for automation! Stay tuned!

Fediseer: A Fediverse Chain of Trust

Db01 Comment

Recently I’ve started running my own lemmy instance, as part of my decoupling from Reddit, due to them speed-running enshittification.

The instance has been growing nicely and holding up very well indeed. but there’s dark clouds forming on the horizon, as more of more of the early adopters and people with principles are leaving that service and are looking for alternatives.

The first signs of trouble appeared when I noticed that the top instances in the Fediverse Observer were growing by thousands of users per day, but had very little activity to speak of, with few threads and barely any comments. This was a clear sign of botted accounts being generated by the thousands.

My initial reaction was to cook up a quick REST API and a complementing script which would allow instance admins to quickly de-federate from instances with this amount of botted accounts, as it would point to an instance with insufficient protection, and those account could easily be used in the future to spam others. A small pre-emptive measure. It’s not particularly sophisticated, but I wanted to get something out before trouble occurs.

The Fediverse Fediseer was born, but I wanted something more. I feel like a big issue with the Fediverse as it stands right now is the same as email. It’s trivial for someone to set up hundreds or thousands fake fediverse domains and start spamming other instances. All it requires is that any account on those instances to follow their spam domains, to open the door. Sure, instances with manual user approval are somewhat more protected, but this approach does not scale well, and will lose us the opportunity to capture the people looking for alternatives outside of corporate control. Likewise any place with open registrations is available to botters to inject their fake accounts in order to follow their spam instances. In general, it’s a big problem to solve through de-federation alone.

My thoughts then was to find a way utilize the “whitelisting” capabilities of lemmy and the fediverse to ensure that only trusted instances are federated. But this has it’s own share of problems. Particularly, it’s also difficult to scale, and it easily excludes people running their own instances.

However the main benefit of whitelists, is that one does not have to be constantly vigilant. A dedicated bot network can spawn thousands of new domains and sleeper accounts to flood the fediverse and de-federation lists would easily grow exponentially to try and fruitlessly fight against this. A whitelist should theoretically remain relatively short and there’s natively protects against domain flooding.

So I wanted to come up with something that would both make it easy to compile and maintain whitelists, but also not lock out people from individual accounts.

After a day of work, I’m excited to unveil the new Fediseer functionality which works around a Chain of Trust!

This all works via a REST API (which I hope some enterprising people will make a fancy UI for it)

The first step is to use the Fediseer API to register your instance’s domain. You can do it from the provided interface, or use a curl call, But I hope in the future the community will develop way fancier UIs to handle this. You will need to provide a username which is an admin on the instance you want to claim (preferably yours).

If all goes well, the user you provided will then receive an API key via Private Message on their own instance, which they can then use to guarantee or endorse other instances.

A new account on the Fediseer however is not visible on the whitelist API by default. This is because only accounts which are “Guaranteed” by other instances are visible. The Fediseer starts with an core instance, the fediverse.com, which functions as the root for the Chain of Trust. Either this account, or another account previously guaranteed, will have to guarantee your instance, before it is available for the whitelist.

Using the Guarantee API endpoint you can now guarantee other instances with your own instance.

Once your instance is guaranteed, it also get its first endorsement from its guarantor, and now will be displayed on the whitelist using the default settings

This list can be exported also in csv format, for easy injection into lemmy whitelists

This naturally forms a network of trust with instances guaranteeing each other down the chain. The purpose of this guarantee is to prevent bad actors from sneaking in. So let’s pretend that a spam instance sneaks in, and starts causing trouble. What do we do?

Very easy, instead of having to waste time going around asking everyone to defederate that instance, and reminding new users to do likewise, we simply withdraw our guarantee for that instance. Once a guarantee is withdrawn, that instance is not anymore visible in the whitelist. Which means any servers which automatically pull and deploy the whitelist from the Fediseer, will automatically reject such instances.

But this goes even further. Let’s say someone pretend to be nice in order to start letting spam instances into the whitelist. When people shout about it, they say the right words and withdraw their guarantees, but keep adding new ones in. Eventually, we’re going to notice that all guarantees for spam seem to be coming from the same instance. When that happens, we just withdraw our guarantee for that instance which does 3 things. A) It prevents that instance from being in the whitelist. B) It prevents that instance from guaranteeing or endorsing others C) It removes the instances lower in the chain branch of that instance as well! So if instance A faked being nice, then guaranteed for 20 spam instances once in, all it would take for those 20 spam instances to be removed from thew whitelist, for the guarantor of instance A, to withdraw their guarantee! If they don’t want to do that, then whoever guaranteed them might withdraw it, until we find someone who does.

Now, I won’t claim that this system is perfect. Human nature being what it is, I expect power groups will form which might not agree with who else is guaranteed. This is where the fediseer being FOSS helps. If there’s a core disagreement between big groups of fediverse projects about who should be guaranteed in the first place, I expect other Fediseers to spawn with their own Chains of Trust which are more or less strict than other. An instance could very well be registered to multiple Overseers and thus be part of different whitelists. I am perfectly aware that I will not be able to satisfy everyone limits but I hope to provide a tool that can!

The main point here is to create the system which can start building Chains of Trust, which have a manual human control but are easy to adjust as the environment changes.

There’s more here that I haven’t mentioned yet, such as the endorsements, where guaranteed instances can endorse others, and anyone can set their whitelist to require more of less endorsements. Or being able to whitelist only instances with endorsements from another instance. Or how the Fediseer will PM you with changes to endorsements and guarantees etc

There’s plenty of ideas to add. This is merely the first step. And I’d love you all to help me take more of them!