July 13th, 2007, 10:07 PM
Partner List and Notphish tag
I'm the co-inventor of the following antiphishing method. This is a request for serious technical feedback from readers. Hope it's ok to post here.
A bank publishes a set of domains that will be in outgoing messages, before it sends out any of those messages. Call this a Partner List. It goes to a central website ("Agg Center") that gets such lists from banks.
A browser modification is made. When a user gets an email claiming to be from the bank, the browser finds the domains in links in the message. It contacts the Agg Center and gets the Partner List for that bank. If a domain is not in the Partner List, then the message is considered phishing. The browser turns a "Notphish" button red. It can also disable all links or just the suspect link. But if all the message's domains are in the Partner List, then the button turns green.
How does the browser decide whether to contact the Agg Center? A real message from the bank will have a Notphish tag, eg <notphish a="bank.com" >. Where bank.com is replaced by the domain of the actual bank. Most messages are not from the bank or phishing, and they won't have the tag. So the mod will just leave the button neutral.
All that a user has to be trained to do, is expect that real messages from a bank will turn the Notphish button green. If a phisher omits the tag, then the phishing message will not validate (button=neutral). If she puts in a tag for a real bank, and she has a link to her phishing website, then this will not be in the bank's Partner List. The browser will discover this and turn the button red.
The use of the Notphish tag avoids a problem with some methods that have heuristics and expect the user to manually push a button to run those tests against a suspect message. Since most messages are ok, the user might tire about doing the tests. And, by definition, she won't do those against a message that fools her. The tag also avoids an automated approach that checks all messages against some central website. Very wasteful of bandwidth.
A simple extension is that the bank can also publish hashes of its future outgoing messages to the Agg Center.
The method avoids the user having to memorise multiple passwords [that are text or image] for websites at which she has an account. It is objective in that it does not use subjective heuristics. Lightweight, for there is no advanced cryptography.
The method also avoids the drawback of blacklists used against phishing. These are susceptible to a zero day attack. Which is the time interval between when a phisher sends out messages, pointing to a new phishing website that she has, and when those messages are detected by various antiphishing groups, and decisions made to put the website's domain into a blacklist, and the promulgation of the blacklist. Whereas here, the bank disseminates its Partner List before the messages go out.
A user does not have to use a fob to generate one time passwords for a website. Fobs are expensive. And do not scale when a user has accounts at several websites, each with its own fob. Cost and useability issues. Also, our method lets a user get a verified message from a bank at which she does not have an account. Where the message might be to try to sign her up. There is no prospect of her having a fob at a bank at which she is not a customer.
The method can also be used when a user is surfing the web. Websites associated with a bank can have a Notphish tag in their pages. The bank can have another Partner List, that gives the domains of associated websites. So the tag lets the method treat messages and websites in the same way.
The biggest problem with most current antiphishing methods is that they do not involve the banks, in the manner described above. Hence, when a method gets a message or webpage, it has a hard AI problem, trying to decide if that item is phishing or not. An open loop problem. Our method closes the loop by involving the banks.
You can read the full text of the method at the following link, to the World Intellectual Property Organisation website -
Last edited by wombon; July 14th, 2007 at 12:29 AM.
July 14th, 2007, 10:19 PM
I know that blacklists are considered pretty effective against general spam that has links in it to the spammer websites. Are you really trying to say that because blacklists arent so good against phish sites, that phishing is a special case of spam? Different from most other spam?
July 15th, 2007, 12:56 AM
yes, phishing is distinct from generic spam
Yes. Some antispam vendors think of phishing as a special type of spam. We suggest that it is more useful to consider phishing as distinct from spam. This is not a semantic quibble. While a spam domain might be offensive, so long as it's a legal business in the region where it's physically located, then it can continue to operate on the net. But this also means that when the domain sends out messages with links to itself, that a blacklist is useful. Because the domain is persistent.
However phishing domains are essentially illegal everywhere. So a blacklist of phishing mostly contains obsolete entries, that are no longer operating. Phishers make their money in a zero day attack.
There is also another way that phishers respond to blacklists. They subvert many computers, that act as phishing domains or pharms. Instead of a phisher just using 1 domain at a time, she might have hundreds [or more]. So the value of detecting 1 phishing domain that is currently running and then blacklisting it, is small.
Essentially, phishing pushes the concept of a blacklist to destruction.
July 16th, 2007, 10:48 AM
One problem, what's to stop a phiser creating a message and using your tag?
July 16th, 2007, 01:16 PM
Read my mind there WolfeTone, exactly what I was thinking.
So instead of "<notphish a="bank.com" >" I put "<notphish a="otherbank.com" >"
However, I presume we are working from the assumption that your central server is a trusted server and so otherbank.com will not be registered.
Indeed, however this idea is the other way round. This is a whitelist of approved sites with all others being blocked rather than a blacklist of blocked sites with all others allowed.
Originally Posted by wombon
The big flaw is keeping your central server secure. You need to be 100% sure that hackers haven't added entries to the database which...is reasonable with good logging and firewalls. What is harder is making sure that some pissed off ex-employee of (for example) HSBC hasn't just added a couple URLs on his last day so that he can get revenge.
Oh, and heres something to consider - Some nasty virus that inserts itself as a local proxy/filter/sniffer and overrides your verification software/plugin. I presume some form of SSL authentication would help with this, but then again maybe not.
If the world doesn't stop annoying me I will name my kids ";DROP DATABASE;" and get revenge.
July 16th, 2007, 10:15 PM
Reply to Aardpsymon
Yes, you're right. If the phisher writes a tag, <notphish a="otherbank.com" />, where otherbank is a website controlled by the phisher, then it will be detected. Since otherbank.com won't be in the Agg's list of client banks.
As far as having a central server - it's not really a flaw. Think of the large websites, eg. ebay, citibank, bofa. These already have redundant machines, multiple sites, and competent IT staff. Occasionally, we read about some big website going offline. But the reason that's news is because it is relatively rare. Same with the method's Agg Center.
Now, regarding some ex-employee of a bank that sabotages the bank's list that goes to the Agg. Or perhaps, an employee who got subverted/bribed by a phisher, to change the bank's list to put in the phisher's domain. But this sort of thing is what banks are already defending against, long before the web rose. Most/all banks have deliberate separations of duties, and multiple signoffs on key actions. The making, approval and submission of a Partner List from the bank to the Agg is something that would require multiple signoffs, at a fairly high level.
Of course, if enough crucial employees of a bank got together in a conspiracy, they could subvert the Partner List. But the more people in a conspiracy, the harder it is to put together and avoid detection. That's the point about multiple approvals. Plus, in this case of a conspiracy, they might not even bother with phishing. The latter is just a means to loot a bank, via many customers' accounts. But typical accounts only have a few $k in each. A highly placed conspiracy might instead hit the wire transfers, and move much larger amounts.
Here's another angle. Right now, no bank is using the method.This is equivalent to every bank having a Partner List that is every domain on the net. When a phisher sends out messages right now that attack bank0.com, she is effectively adding her domain to bank0's Partner List. So that if a browser mod existed that did our method, and if the Agg exists, it would be moot, because of that open ended [=infinite] list. The bank cannot currently control that list. Our method gives the bank the control.
On the last point - about a virus subverting the browser mod. Yes, it is theoretically possible. One rejoinder is that our method that involves the mod can also be done at an ISP/message provider. When its mail servers get an incoming message with a Notphish tag, it can check that message's links against the Agg. So nothing to do with any browser. And the ISP has a professional IT staff to defend its center against subversion.
But going back to the browser mod being subverted. The technical barrier is nontrivial. Currently, phishing obeys all the rules of SMTP, http and HTML. Which is why it's so easy to do phishing. No technical barrier.
Suppose now we have a browser, IE running on a Microsoft OS on a desktop. There are ways to lock IE and any mods, at the Ring 0 level of the OS. Possibly with crypto signing of any extra future mods. Similar could be said for other OSs.
July 19th, 2007, 02:01 AM
SMTP is root of the problem. It needs to be replaced with something secure, or we need to bolt-on something better.
RFC 4406 talks about trying a bolt on process for making sure the sender is the real sender, with spf settings in dns. http://www.ietf.org/rfc/rfc4406.txt?number=4406
But even a good bolt-on solution will take a long time to get everybody involved to follow the recommended practice. A replacement for SMTP will require an act of God to approved and implemented easily.
So my answer is: Wait until Microsoft tries to fix the problem, the rest of the products will follow Microsoft's lead.
The real problem is no matter what you implement, the users will complain, the hackers/phishers will find another way around it. Man-in-the-middle SSL attacks, spoofing/poisoning dns, phone phishing, etc. The reason they phish is because its easier and humans are stupid. They will always find a way to out smart the people that seem to constantly prove Darwin wrong.
July 19th, 2007, 02:21 PM
Sadly that is all true. Heres how we fix phishing:
1) Obtain a large bat
2) Engrave it with "DO NOT GIVE YOUR PASSWORDS TO ANYONE"
3) Beat people round the head with it.
4) Repeat until the message is recieved.
If the world doesn't stop annoying me I will name my kids ";DROP DATABASE;" and get revenge.
July 19th, 2007, 04:38 PM
Aardpsymon has a good method. Listen to him haha
July 21st, 2007, 09:47 AM
In the 1980s, the toothpaste got out of the tube, with mail senders being able to forge essentially anything in the header. There have been innumerable suggestions about trying to improve SMTP. The problems revolve around the very success of the web and email. No one can persuade most of the SMTP mailers around the world to change overnight.
Originally Posted by caveman8fb
As far as Microsoft is concerned, it implements STMP in some of its products, and in its OS. But it's not the most important player in this subject, let alone the key player.
There have been many proposals to prevent sender forging in SMTP. The only major proposals to take off so far have been Domain Keys and SPF.
Our method takes a different tack. We eschew the public key encryption in the latter, as being too heavyweight and susceptible to DoS attacks. Instead, we offer authentication via hashing. A crucial difference, if you know cryptography. Most email is transmitted via mail relays as plaintext. "Postcard level of privacy". For the vast majority of email, we suggest that there is no need for encryption. The above methods use public key, for authenticating the message, anyway. Not for encrypting it. But a strong hash function gives you that at a level of confidence adequate for most users.
Plus, our method also authenticates websites, via our tag and Partner Lists. Which even in principle Domain Keys and SPF can't do. Their scope is email. Not browsing.