When the Mann Co. update was released 6 months ago (September 30, 2010) introducing trading, the store and five class sets, we did not predict it would be a problem for us. We have not previously interfered with updates, for instance we have opted not to introduce any class limits even when everyone played engineer. Everyone wants to try the new features and it will only take a week or so for everyone to switch classes again anyway. But this was different. Our chat quickly turned into market but we figured it was just part of the game now and once players have traded the items they want it would calm down.

We were wrong. While more and more players were indeed not interested in trading anymore, the few that were sounded like a broken record, constantly spamming the chat with their offers. This became annoying and when we realized the Mann-Conomy was out of control, we had to start warning, kicking and in rare cases even ban spammers. But it's not easy to keep track of spammers. We went from a few incidents per day to a constant stream of troublemakers. Most were not regulars. They would come from everywhere to find someone to buy their stuff at outrageous prices. Warning spammers was all we did and the workload was too heavy.

If your short-term memory works alright then you should remember that the spammers sounded like broken records. This could easily be exploited to shut those records off. Simply by keeping a separate list of each players previously sent chat messages and comparing any new messages to the old ones we would be able to ignore any repeated messages. During December 2010 s3 built a working prototype with these specifications and we now had a simple server-side anti-spam program for TF2.

This had spammers very confused and astonishingly it actually stopped most of them! Of course many figured out how to bypass it; simply add, delete or change a single character and it would not detect any spam since it would be considered unique. Still, it made keeping up with the wrongdoers a lot easier. Rather comically the Medieval server helped spammers bypass it without them even noticing since the game mode rewrites chat messages as they are written. Spammers who had not figured out how to bind or copy and paste text also had no trouble with our spam filter since even though they wrote the exact some phrases, few could spell.


Around the turn of the year Procyon had rewritten and upgraded our in-house anti-spam program. This more advanced, "final" version has been running on all our servers for 3 months now. Hopefully none of you have had the pleasure to notice it and never would have if not for this post, which is our goal with most of our work.
The more notable features are:

* Pattern recognition
By calculating the Levenshtein distance (google it!) between messages, bypassing the chat filter by simply adding a single character is no longer possible. Since the introduction of this feature only a single player has had to be banned for spamming. For example, "STOUT SHAKO FOR 2 REFINED" matches "SELLING STOUT SHAKO FOR 2 REFINED!!".

* Dynamic rate limiter
As opposed to simply blocking every repeated message a timer is activated after a message has been repeated once (that's two identical or similar chat messages). If a message is repeated once more during this period it will be ignored and not shown to anyone. At first this timer will be short, but if the spammer keep it up the spam filter will respond by progressively getting more aggressive. The reason for this is to minimize the risk of false positives as well as allowing players to still use the chat for trading as long as it's used in moderation.

* Forgiveness
If you chat a lot the phrases you cannot use again will build up until you risk being falsely marked as a spammer because you repeated something you said hours ago. By giving the spam filter a mild case of amnesia messages written long ago cannot match recent messages, since they're not saved. Oh, and it also saves a bunch memory and some CPU time. :)