Introduction
Monty Python did a skit where they said the word "SPAM" so many times
you wanted to run screaming from the room. SPAM is either junk e-mail or junk
postings in a newsgroup. Typically it is an advertisement for some product, or
scam totally unrelated to the newsgroup, e.g. pornography in the comp.lang.java.programmer
newsgroup. People try various tactics to avoid getting on the spammer’s
hit lists. For the most part they just annoy or block legitimate correspondents.
Eventually we will invent legal or technical countermeasures, but for now it is
just a fact of life like mosquitos on a camping trip.
Spam is usually an advertisement for something, but it can be any sort of junk
mail sent without any regard for whether it would be of interest to the
recipient, such as chain letters or Kristian
prosletysing.
Spam is beginning to cripple the entire email system. The number of spam message
has increased 8 fold between December 2000 and May 2002. This is a compounding
rate of 13% a month, even faster than MasterCard interest mounts up.
Spammers commandeer mail sites and make the broadcast spam email. Going through
a commandeered mail server helps mask the spammer’s identity.
There are three things can do, report abuse, secure
your mailserver and block spam.
“Spammer” as Epithet
People often use the word spammer as a general insult
word in place of “shithead”. It is used to chastise someone for a
lame post, an irrelevant post, an unhelpful post, a post that another disagrees
with, a slightly-off-topic post, an erroneous post, a post that mentions a
commerical product favourably, a post with a link to one’s own website, a
post that answers the wrong question…
It can drive you mad trying to defend yourself against the charge of “spammer”
if you take the insult literally since those using it have no idea of its
original meaning.
Reporting Abuse
For how to report newsgroup spam see net abuse.
Spam Cop provides an
unsolicited email complaint system with access via both email and the web. They
try to figure out the responsible party or parties and send an (somewhat)
anonymised complaint form to them. They also have a local newsserver with
several discussion groups at news.spamcop.net.
Note, this a newsserver, not a web page.
To do the complaint yourself, use the Eudora Blah Blah icon, or equivalent in
your mail program, to display all the message headers. In there, are clues to
the possible culprit.
X-Persona: <Shaw>
Return-path: <someone@mindprod.com>
Received: from pd2mr1so.prod.shaw.ca
(pd2mr1so-qfe3.prod.shaw.ca [10.0.141.110]) by l-daemon
with ESMTP id <0HWA001A9NPPLM@l-daemon> for someone@shaw.ca; Fri, Received:
from pd5mi2so.prod.shaw.ca ([10.0.121.83])
by pd2mr1so.prod.shaw.ca (Sun ONE Messaging Server 6.0 HotFix 1.01 (built Mar
15 2004)) with ESMTP id <0HWA00962NPJ0ZC0@pd2mr1so.prod.shaw.ca> for
someone@shaw.ca (ORCPT someone@shaw.ca); Fri, 16 Apr 2004 20:46:31 -0600 (MDT)
Received: from vega.servlets.net (vega.servlets.net [209.162.192.248])
with ESMTP id <0HWA00B2YNPO47@l-daemon> for someone@shaw.ca; Fri,
Received: from mail.inter-corporate.com ([24.87.56.254])
by vega.servlets.net (8.9.3/8.9.3) with ESMTP id TAA07545 for
<java@immuexa.com>; Fri, 16 Apr 2004 19:46:41 -0700
ID MO0006B1; Fri, 16 Apr 2004 19:46:33 -0700
Received: from spooler by mail.inter-corporate.com (Mercury/32 v3.32); Fri,
Received: from someone.mindprod.com (24.68.232.84) by mail.inter-corporate.com
Date: Fri, 16 Apr 2004 19:42:55 -0700
From: Roedy Green <someone@mindprod.com>
X-Sender: someone.mindprod.com@mail.mindprod.com
Message-id: <6.1.0.6.0.20040416193649.02f391f8@mail.mindprod.com> Original-recipient:
rfc822;someone@shaw.ca
Buried in that gibberish there, especially X-Complaints-To,
are many domain names you can look up with whois, and
IPs that you can look up who owns them at arin.net
or whois.sc. From that you
can track down some email addresses to complain to a telephone numbers to call,
in the same manner as for newsgroup net abuse.
When you make your complaints, make sure you include the complete text of the
email including the full header.
If the email contains a virus, there is no point is reporting net abuse. The
person it purports to be from was very unlikely the actual sender, and the
person who did send it did not do so intentionally. Don’t blame the FROM:
person. He is nearly always innocent! His machine is not necessarily infected.
The machine of someone with his email address in the Outlook address book is
infected.
Fraud
The American FTC is still interested in email scams such as 419 (Nigerian,
advance fee) spam mail. The usual scam involves someone wanting to launder
millions of dollars through your bank account. Report the such scamming emails
to the FTC,
or email them at uce@ftc.gov. See the FTC
Spam Website for more info.
Securing Your Mailserver
This only applies if you run your own mailserver. Most people let their ISP do
that for them. There are three ways to fight back to stop spammers from using
your mail server.
- Refuse to forward mail unless the FROM: field is your domain. This is fairly
easy to spoof so is not very secure.
- Keep a list of valid IPs from which your mail server is
prepared to accept outgoing mail.
- Use POP3 authentication. Make people login with a user id and password if they
want to use your mail server.
Blocking Spam
If people would stop using Microsoft Email programs Outlook and Outlook Explorer,
it would stop most virus-based spam in its tracks. These two programs are
criminally negligent in the way they deliberately aid viruses to spread. Use
something else e.g. Eudora, or some other mail
reader.
To stop email viruses and worms, you need a virus checker such as Norton
Antivirus or Panda Antivirus. You are protecting
not only yourself, but also your reputation. If you are don’t take
precautions you will infect everyone you send mail to.
There are 5 types of spam-blocking software:
- an add-in or feature of your email client.
- a program than runs on the client that gets between your email program and the
mailserver.
- a program that runs on the client, that runs in parallel with your email program.
It takes a first peek at the mail and classifies or deletes spam, then your mail
program fetches what is left from the server.
- software you run in conjunction with a mailserver.
- a service you sign up for to provide spam-fee mailboxes, usually not with your
domain name.
Spam Blocking Software
Spam blocking software has two problems, recognising spam based on word patterns,
without accidentally blocking real mail. It needs fairly sophisticated logic to
make those decisions.
- BogoFilter:
with C source for Linux, FreeBSD, Solaris, OS X, HP-UX, AIX. Uses a Bayesian
filtering technique.
- ChiaraMail
requires iMap mail server, not POP3
- Em Tec Spam Detective a
spam filter that works with MAPI, POP3 and SMTP3.
- HashCash:
the idea of this is to force the sender to invest some time and money in getting
through to you, by forcing him to spend CPU time to compute a key to get through.
This expense should deter spammers. Unfortunately, it will deter legit callers
too.
- iHateSpam
, works only with Outlook and Outlook Express. Server based. No software in
client at all.
- K9: free with request for
donations. Gradually learns what is spam. It acts as a proxy mailserver. Your
mail program goes to it for mail and it goes to your ISP. This makes it a little
more complicated to set up. It does not delete any mail, just tags it with [spam]
so your email filter program can easily identify it. Unfortunately it does not
seem to handle SMTP proxy as well, so it requires an email program, e.g. not
Eudora 6, that than configure the passwords and servers independently for SMTP
and POP3. The manual is on the web. The program has not even rudimentary
tooltips. It is not a program you can figure out easily without reading the
documentation. It won’t delete the junk off the server for you. You must
still download it into your mail program and dispose of it there.
- MailBlocks: similar to Zaep,
now owned by AOL, but server based so you don’t have to tunnel challenge
messages through a firewall. You sign up with new email accounts at MailBlocks.
Then you can do three things:
- Get people to send you mail directly to your new MailBlocks accounts.
- On bended knee, ask your ISP to forward your mail to your existing email
accounts to the new MailBlocks accounts.
- Ask MailBlocks to periodically pick up your mail from your old accounts.
Everyone in your address book is whitelisted. Everyone else gets a challenge the
first time they send you email. If ignore the challenge, the email is treated as
spam. If they answer, they get put on the white list. Basic service is free.
Premium service (more space to store mail, more rules for filtering) is
per year. This sounds fairly fool proof compared with Zaep. The disadvantage is
legit callers will be offended and will refuse to answer the challenge, or the
challenge will be lost and treated as spam itself.
- Mailinator:
Free disposable email accounts. You are on the web, at a party, or talking to
your favorite insurance salesman. Wherever you are, someone (or some webpage)
asks for your email address. You know if you give it, you’ll be on their
spam list. On the other hand, you do want at least one message from that person.
The answer is to give them a Mailinator address. You don’t need to sign-up.
You just make it up on the spot. Pick jonesy@mailinator.com
or bipster@mailinator.com - pick anything you want (up
to 15 characters before the @ sign). Obviously, these
are not secure. There are no passwords. Anyone can pick up your mail who knows
the account. Use these whenever a someone demands an email address to download
software or activate an account when you want no further mail from them after
that. sneakermail is a
similar service.
- Use Tagged email addresses. This requires no special software. Use a return
address like this localpart+tag@example.com that will
deliver to localpart@example.com and allow you to see
where the address came from. For instance, if you end up getting spam from localpart+amazon.com@example.com,
and you only gave that address to amazon.com, you
know where the leak occurred. Of course clever spammers will strip the tag.
- MailWasher: free with
request for donations. Previews mail, similarly to SpamDetective and deletes it.
Lets you mark all mail as deletable or bounceable, but not the reverse. Accesses
databases of blacklisted ISPs. I found it froze up frequently when confronted
with 1500+ pieces of Sven Worm-created junk mail.
- NewsReader/MailReader
student project
- Nucem
, not a spam filter but a tool to track down the source of spam and to manage
complaints to the offending ISP.
- Popfile too
often mistakes legitimate mail for spam. It sits between your email program and
the mail server. It works with Windows, or with any platform that supports perl.
It is free. It works by identifying spammish words from a dictionary you
maintain.
- SaProxy
uses 25 to 80 MB of RAM.
- Spam Assassin free. Uses Vipul’s Razor to
collaboratively evaluate spam.
- Spam Butcher.
one time charge. Uses a POP3 fuzzy logic filter than runs on your own computer.
- Spam Filter student project
- Spam Inspector
one time charge. Integrates with Eudora or Outlook. Free trial.
- Spam slicer
per year.
- SpamArrest
per year for a spam-free mailbox. They look after detecting and removing spam.
The nice thing about this service is you don’t need to install any
software on your machine and you don’t need to change your email address.
What happens is you change your email program to pick up mail from SpamArrest,
and SpamArrest picks up the mail from your ISP.
- Spambayes.
Its IMAP proxy is buggy, though the POP3 proxy seems OK though.
- spamcop.net
per year. Sell spam-free email accounts, and lists of spammers to feed into
blocking software.
- SpamCure
to
per month. You use a special email address in their domain.
- Spam
Remedy
- Spam
Repellent
per month for a spam-free mailbox. They look after detecting and removing spam.
When you run the software on your own server, they call it Spam Cure.
SpamNix:
a Baynesian filter than integrates with Eudora.
This is what I use myself. It took about a year before it got good at
discriminating spam from gold. Free trial with nagging to purchase every time
you start Eudora. Persistent nagging is only appropriate after the advertised
trial has ended. You train it by letting it sniff mailboxes that contain either
pure spam or pure gold. This initial training process is quite slow and gobbles
up all your CPU. It must be done with freshly compacted mailboxes. Thereafter it
just does it on individual messages it errs in categorising. CNet rates it
highly. SpamNix uses some of the SpamAssassin code. Use the junk/not
junk to move spam that gets through you manually and train in one step.
All that happens if you click accept/reject is it
trains itself for the future or lets you set up an explicit filter. The nice
thing about it is it quickly gets spam out of the in folder, which is delicate
and is corrupted if the Panda antivirus program deletes a message. I still end
up reviewing every piece of spam before finally deteting it since it sometimes
make mistakes. Oddly by default it does nothing with spam but categorise it. You
can to configure it to throw spam into the junk mailbox or trash mailboxes based
on some cutoff level of confidence. It stores its list of explicit allow/rejects
in F:\Program Files\Eudora\plugins\spamnix.ini. n
Windows, copy the file mailfolder\Plugins\Spamnix.ini and the
directory mailfolder\Plugins\Spamnix to the new computer, where mailfolderis
the location of your mail files. It stores its Baynesian training information in F:\Program Files\Eudora\plugins\Spamnix\*.db.
The file F:\Program Files\Eudora\plugins\0Spamnix.dll
is supposed to be there despite its peculiar name. If you move Spamnix to a new
computer, move F:\Program Files\Eudora\plugins\spamnix.ini
and everything in F:\Program Files\Eudora\plugins\Spamnix\.
- SpamWatch:
This is a built-in no-extra-cost feature of the Eudora mail program. Every time
you transfer a message to the junk mailbox, it learns its characteristics so it
can automatically detect similar spam in future. You can put junk and unjunk
icons on your tool bar for marking junk, and rescuing good stuff from the junk
folder.
- Vipul’s Razor free. Perl geeks solution to
collaboratively evaluating spam.
- Zaep from Rhinosoft the
makers of FTP Voyager. This works a quite
different way. The first time anyone sends you mail, they get an automatically
generated response asking them to click an url taking them to Zaep’s
webserver to confirm they intended to send you mail. After they have done that,
that mail and all subsequent mail gets through unimpeded. You don’t need
to set up a mailserver. At the client site, Zaep stands between the client email
software and any of their mailservers, local or at ISPs, as a miniature proxy
mailserver.
Hint: when you first install the default userid/password is admin/admin.
You have to dig in the knowledge base to discover this. After you change it, it
is registered on the Zaep server, so it does not revert back, even if you
uninstall/reinstall.
Zaep does not currently support IMAP.
You need to configure it with a domain name or permanent IP. If you have a
dynamic IP, you can get a free domain name that tracks it from DynDNS
or DNS4ME. The spam harvesters may at some point learn to defeat this thing, but
for now it has a good chance of getting rid of all spam.
The big problem is you may miss mail from legitimate customers who can’t
be bothered to respond to the challenge, or whose own spam blocking software
throws the challenges away thinking them spam. This is a solution for someone
inundated with spam with legitimate correspondents trying hard to get through. I
am working get it going on my own machine. I have discovered it does not work
with the Opera browser for administration, and does not work with IE, on my
machine, unless I manually modify the URLs it uses from 127.0.0.1
to localhost. It appears to support only one
mailserver, but many email accounts, possibly coming from different machines on
the LAN. It is fairly complicated. You require two internal proxy ports, one
external port for accepting confirmation requests and a fourth port used for
doing configuration changes, either locally or remotely.
You must configure your firewall and router to let the confirmation port through.
You must also configure your router as a virtual server to pass through incoming
messages on the confirmation port to the particular machine you have set up as
the Zaep server. You also must be sure Windows filtering is letting the messages
through. Check out Start ⇒ Settings Control Panel ⇒
Network ⇒ LAN ⇒ Properties ⇒ Advanced. Eudora
6.1 no longer lets you configure the SMTP and POP3 ports. unless you copy extrastuff\esoteric.epi
to the main Eudora directory. Unfortunately, that does not give you the ability
to individually configure each of your personalities. It effectively limits you
to one email server. To do that, you must manually edit the eudora.ini
file.
In version 3.0 you have the option of ignoring the notifications from the Zaep
server tunneling through your firewall, and just automatically generate the
email challenges yourself when you go on-line to fetch mail. Even with this
simplification, I could not get it to work.
Spam Blocking Hardware
Tyrnstone Systems Deep
Six is a box that protects an entire network from spam. It claims to be much
better at detecting spam and avoiding false positives than the competition. It
claims to allow only 0.8% of spam through with 0.002% false positives. It uses
blacklists (bad guys) and whitelists (friends). It costs
so it can be justified only for corporate use. Tynstone keep updating the
appliance automatically, though it is not clear if they are maintaining
blacklists for you or just fine-tuning their detection algorithms. Spam costs
corporations huge amounts in employee time, so even modest increases in spam-detecting
efficiency are worth pursuing. WARRANTY: 30 day device performance assurance. 90
days appliance malfunction. Extended warranty and upgrade assurance is available.
Blacklisting
There are dozens of databases that track known spammers. Many mail programs
refuse to transport mail from or to this bad guys. People who leave open relays
allowing spammers to highjack their mail servers can also get on this list.
Sometimes people put you on such lists out of spite. To get off, you first need
to check your status, then contact the various databases to plead you case.
Insert the IP of the site you want test after ip=, or you can key it once you
get to the dnssnuff site. Use ping to get the IP.
Junk Mail
You can block junk snail mail (aka hard copy spam) in Canada by writing to:
Canadian Direct Marketing Association
Do Not Mail Service
1 Concorde Gate Suite 607
Don Mills ON M3C 3N6
CANADA
Tel: (416) 391 2362
fax: (416) 441 4062
or in the United States:
Direct Marketing Association
Mail Preference Service
P.O. Box 9008
Farmingdale NY 11735
9008
U.S.A.
Tel: (212) 768 7277
You can request telemarketers and junk mailers leave you alone at iOptOut.ca.
Spam Motivation
There are at least eight classes of spammer:
- Vendors trying sell you something, usually pornography.
- Con artists fishing for suckers.
- L’enfant
provocateurs just trying to annoy you out of simple childish malice.
- Fanatics trying to sell you religious ideas. They believe the importance of
their divine message overrides the normal rules of courtesy.
- Propagandists with a desperate political message. They may even consider what
they are doing a form of electronic warfare.
- Control freaks who want to shut you up and censor your ideas by clogging your
email system and thus preventing you from communicating with others.
- Bigots who seek revenge on you for holding a divergent opinion from them,
usually on matters political, religious or sexual. These types have taken to
sending larger and larger messages, so that even if you automatically identify
them as spam, they have still managed to tie up your Internet connection.
- Viruses that generate gibberish mail just to annoy people, but not to persuade
them to act in any particular way. It is sort of competition to see how much
havoc the virus creator can stir up.
The Future Of Spam
I had a bit of a fright in 2004-06. I thought for a while I was under another
email DOS Denial Of Service attack. I wondered if I would
be able to publicly post even my munged public email address ever again. During
the Serbian war, I received 80,000 letter bombs a day from people who objected
to my pro-US stance. Pretty well anyone, even marginally more famous or
controversial than I am, can no longer maintain a public email address. The
proportion of people being cut off totally from public email access is gradually
increasing.
In like manner, I can see how spammers with political, religious, pornographic,
malicious, or commercial interests will gradually make the newsgroups and
standard email totally unusable. As my Dad you used say all the time, "watch
the derivative" eXibitionsoftware.com
is selling software to the technopeasant fanatics to spam tens of thousands of
newsgroups at a pop.
We can’t wait like frogs in hot water until the email and newsgroups are
completely gridlocked before taking action.
I see a multi-pronged approach will be necessary:
legal means
Spamming needs to be made criminal and spammers prosecuted, preferably by
hanging, drawing and quartering. Was there ever a better case for the death
penalty? Was there a less provoked crime? However, spammers will always find
some country to harbour them. Surely some third world country will always foster
the spam industry just as the Cayman Islands harbours crooked companies, and
Nigeria harbours tramp ships. With the net, they can set up shop in
SomethingIstan and effective maintain virtual storefronts in every country.
boycotts
We must educate people to ensure spammers don’t get whatever it is
they want from spamming, be it sales, web hits, censorship, notoriety, sense of
power, malice, revenge denial of service or attention. Refuse all mail from ISPs
that harbour spammers and let them know why you are doing that. Make sure they
are truly guilty, not just the victims of virus counterfeit spam.
The Boulder Pledge
“Under no circumstances will I ever purchase anything offered to me as the
result of an unsolicited e-mail message. Nor will I forward chain letters,
petitions, mass mailings, or virus warnings to large numbers of others. This is
my contribution to the survival of the on-line community.”
~ Roger Ebert
Future Technology
I see a new email delivery system evolving to completely replace POP3/SMTP. It
will have a number of features.
- Automatic encryption, compression and digital signing. The degree of encryption
has to be automatically decided based on the laws governing sender and receiver.
The basic idea is no one can send you mail without your permission. With digital
signatures, it is practically impossible to forge email. Basically, nothing gets
transported any leg of the way without a preclearance permission.
- Automatic tracking, much the way you can track what has happened to a Fedex
parcel as it wends its way. You should potentially be able to know if a message
was not delivered or not noticed.
- Forwarding standard with mechanisms to inform all your legit correspondents
automatically of your new address and keep them up to date on whatever vCard
style information you want them to know.
- Full efficient use of the 8-bit transparent channels. The current email system
wastes much of the bandwidth with voluminous human-readable headers, 7-bit
characters, and no default compression.
- Sender-pays-receiver system so any spam that does leak through still costs the
spammer. If it costs the sender
to send an email, and the receiver gets
of that, most people will break even or make money. As soon as spammers have to
pay costs comparable to junk snail mail, they will drastically cut back. As it
is now, we subsidise the spammers to pester us.
- The best anti-spam thinking is built-in, suitable for technopeasants —
technology along the line of Vipul’s Razor with
the geeky edges shaved off. Spam detection has to move to the server where it
can be quickly headed off even before the entire message has been delivered.
- Suitable for exchanging large files, and common files, similar to BitTorrent.
- Ways to protect against denial of service attacks by presenting a united front
against the spammer, rather than leaving an individual to fend for himself.
- Designed from the ground up for technopeasants. Everything is automatic and
transparent.
- Anti-spam clubs that police their members. Members get time-limited digital
certificates. You can accept or reject mail based on the reputation of the self-policing
club. You can then be anonymous, uniquely identifiable, but still have a public
reputation. Spam club members either police themselves or destroy their own
reputations.
- The original email system was cooked up overnight as a demo. The author surely
never dreamed his system would be used almost unmodified for planetary email
scheme. It needs a major overhaul.
- There needs to be a separate system for public newsgroups like the Group
Lens where posters of useful material are rewarded finanically and those
posting spam are fined.
- Dealing with spam is a challenging technical problem, and I don’t think we
will make much progress without an overhaul of the basic mail system. This means
we can’t wait for total gridlock before acting. The solution is difficult
both technically and politically and will take substantial time to solve.