I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“. (The italic bits are where I’ve redacted the ideology they’re seeking).
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
I think this is just more of the ongoing controversy being spun up against db0.
This weeks flavor appears to be more data driven, a ‘just asking questions’ phase. I guess in hope the whole ‘falsifying evidence to make db0 users look like neo-nazis’ thing blows over.
Like it’s clear there’s an effort to rid db0 from the fediverse, and it’s just the pretext hasn’t been sorted out yet.
I don’t doubt it even for a second.
I’m lowkey kind of fascinated this morning with what feels like a moment of real panic among western liberal-democratic institutions (projecting a little from my morning news and coffee). That an anarchist instance is getting this much targeted harassment feels like a microscopic extension of that (if I allow myself to be so bold)
As far as I can tell, dbzer0 isnt even being explicitly called out here, but it has an undeniable bdzer0 flavor to it. If it doesnt come out that this was one of our mods at this point, I’d almost be disappointed.
Yep and it doesn’t help that Rimu himself is a very questionable dude. He says he defederated from lemy.lol over their pepe frog logo but honestly I don’t believe it is the reason/ He hates memes. He said as much to me himself. A while back, someone realized that piefed was hard coded to give negative reputation to certain people, regardless of what settings the admins had made. Piefed is built off of Rimu’s opinions and he puts the majority there as an opt-out, not an opt-in. When I made a jokingly apology for all the memes I post, he said “It’s not your fault that the lemmy devs didn’t put guardrails in.”
If he’s that opinionated about memes…
Combine that with the fact that this post has literally no information other than “Trust me bro”? I don’t trust Rimu as far as I can throw his garbage platform and I can’t even throw the thing because its digital code. Hell, Mr. Kaplan even had to do a ton of un-fucking of Piefed to get piefed.world to work and was talking with me about it at the time. Just LITTERED with Rimu’s stances.
Not to mention that Rimu routinely comes in with preconcieved notions that others have to point out the bullshit of and he quietly then steps back from it or refuses to engage entirely and dismisses it out of hand.
Like… the Lemmy devs suck, absolutely. But the only facet in which Rimu is better is he’s not a transphobe.
Edit: See below for screenshots. I’m tired of this shit. Yes. He’s wildly opinionated.
Edit 2: Lol this comment is not visible on piefed.social. Rimu out here screaming about fiefdoms and hidin things critical of him.
Please don’t spread old mis-info or at least back this up with actual links to the source-code (and if we are talking about the same thing, this was clearly debunked).
As for the OP post, this is factually correct and I have seen the evidence. Although maybe Rimu should have been more clear in pointing out that this seems to be not an official instance tool, but rather something some moderators have cobbled together themselves.
I mean, you just did it. The OP post is not factually correct as he stated that it is at an instance level. It is not. It is at a moderator level.
Gotta say though, getting real tired of people telling me that I didn’t understand conversations I was a part of. No. It was not ‘debunked’. It was added under an opt-in toggle after everyone noticed it and called out his bullshit. You might be thinking of something else, but this is what I was talking about.
Piefed is infected with Rimu’s extreme opinionated garbage and he only backs down and puts them into a toggle after someone notices it. That’s not the behavior of a developer that I find personally trustworthy. So when he’s out here making a post that is outright false, claiming that instances are doing something that moderators are doing, I don’t trust him. Not to mention the extremely long conversations I had with Mr Kaplan about how Piefed.world needed to be un-rimu’d in order to work as LW wanted it to.
I personally prefer Rimu taking feedback into account and then making changes than Lemmy devs bluntly closing feature requests widely upvoted by the community because “reasons”
Let’s be honest here, it’s not like there are so many options we can choose from.
Also, the example you gave about shows more than anything else that Rimu did change his mind.
“All the settings in this screenshot are off by default”
He only changed his mind after someone discovered it. The thing I don’t like is his obfuscating his opinions that he’s injected into Lemmy. Having to manually discover them and then bug him about adding an opt out toggle is not exactly trustworthy behavior nor is it something I’m going to applaud. He still tried to sneak it in regardless and only backed down when called out. I’m not giving him props for that.
The ironic thing is that people claim the Lemmy devs will do that to Lemmy. And here it’s Rimu doing it with Piefed.
He also has silenced my account from piefed.social. None of my comments are federating to that instance yet there is nothing in the modlog about me. So, Rimu is out here complaining about the modlog problem but he’s actively sidestepping it. He’s silencing users without any reason. So… he’s shadowbanning people.
Are we SURE he isn’t Spez?
That’s worthy of a meme in and of itself.