I wonder what effect #AI generated text like #GPT will have on internet spam. Until now, our anti-spam filters were trained heuristically to try and recognize typical spam patterns, but now spammers could turn into AI models to generate unique text to mask their spammy messages on emails, social media and blogs. Imagine spam links wrapped in elaborate text about any subject you can conceive of. And if the site link behind it is novel each time, it will be increasingly unfeasible for administrators to figure it out.

Not only that, but a lot of sites use minimum active account life to figure out which accounts could be spam. Such accounts are marked based on site interaction and ratings which either forced spammers to buy account, or set up spam farms with people doing nothing but interacting natively on social media until their account was live long enough so that it could bypass spam filter.

Now spammers can simply plug an text AI behind each account and have it run on autopilot for a few months, until they flip on its spam switch, and it transparently starts peppering spam links in its normal posts, which would make the switch almost indistinguishable unless it posts a well known spam site or something. They could even target specific subreddits and train finetuned generative models on the typical posts of users in there, which will even make it fits the typical subreddit pattern extremely well.

I also expect anti-spam to turn into their own AI models to catch them, but I wonder how much this countermeasures can help. It could be that a model could be trained to figure spam patterns a human cannot distinguish, but I’m curious to see how effective it could be. I suspect an easier solution would be to have an AI check the site behind each link and try to figure out if it’s a spam site or not, based on heuristics a human might take too long to figure out.

I have the feeling however that ultimately this is a battle that countermeasure AI is bound to lose. And if that happens, it will start leading to what I suspect will be a internet-wide dissolution of anonymous trust. But I will write about that in a latter post.