Comments on: Cloudmark Fingerprinting Algorithm

By: ครีมมะขามพะเยา

ครีมมะขามพะเยา — Sat, 24 Sep 2011 04:42:27 +0000

I’d guess from the description and that little diagram that they take certain HTML nodes in the message content (links, headers, images, maybe even paragraphs) and run them through a hashing function to get a numeric code describing the content. Then they see if that same code has been entered into their database of naughty email pieces. If enough of them match (which means your email has a number of the same parts as known spam, for example a link to mypennystockscam.com), you are likely to be spam too.

This would be distinct from a rule-based system which checks the message for many specific conditions like {Contains the phrase “penny stocks”} and would be faster and less susceptible to spammer trickery like misspelling words and moving pieces around randomly.

By: Jrad5221

Jrad5221 — Wed, 10 Aug 2011 18:33:34 +0000

Does Cloudmark actually “block” emails from reaching their destination or do they filter emails to SPAM if its fingerprinted? Also, if the better part of your email deploy is fingerprinted (most hit the SPAM box) can that result in a domain block or do consumers actually have to mark it as a SPAM complaint?

By: Tiago

Tiago — Sat, 07 Mar 2009 02:25:17 +0000

I also would like to thank Christopher and Ryan for the explanations.

By: Ben

Ben — Sat, 07 Mar 2009 00:57:56 +0000

@Christopher and @Ryan – Nerd alert! Can’t believe you guys fell for this.

Seriously, thanks very much for the scientific explanations.

By: Ben

Ben — Sat, 07 Mar 2009 00:20:12 +0000

@Ryan – that sounds pretty good to me!

By: Ben

Ben — Sat, 07 Mar 2009 00:19:21 +0000

When I wrote this post, I was worried that customers would think that they were gonna have access to our Cloudmark system.

Because this is strictly for our abuse desk.

I just received word from ReturnPath that they’re adding Cloudmark and Barracuda to our Inbox Inspector tool. Woo-hoo!

We’re gonna look into getting these up and running. Super exciting news!

By: Christopher

Christopher — Sat, 07 Mar 2009 00:18:08 +0000

The simplest kind of “fingerprinting” you can do on a piece of data is to compute its hash using an algorithm such as SHA-1. This reduces a piece of text, of any length, into a 20-byte “fingerprint”.

Because changing the input to a hash algorithm even by one letter completely alters the output, the fingerprints generated are – for any given piece of text – essentially unique and unpredictable.

This is probably why Cloudmark extract numerous chunks from the message rather than computing the fingerprint from the entire message body. Otherwise your merge tags would make each persons’ email within a single campaign ever-so-slightly different, radically altering the “fingerprint” for each mail.

This is why spammers insert random garbage text into the body or subject line of spam email: they’re trying to evade tools that blacklist emails by only generating a single fingerprint against the entire message body.

With numerous fingerprints calculated, CloudMark can then see how many of them are spammy according to their database.
So even if your mail does have some text/fingerprints that happen to coincide with some spam (e.g. common phrases like “You are receiving this email because…”), hopefully you’ll have a much higher concentration of good, unique, non-spammy content.

Probably much over-simplified from how CloudMark works, and possibly a bit too geeky, but there you are..

By: Ryan

Ryan — Fri, 06 Mar 2009 23:42:44 +0000