Announcing a new beta of Deploying Rails, or why e-mail is dead

Yesterday we released the second beta of the Deploying Rails Applications book. It has a new chapter on Windows-based deployment, and includes updates now that Capistrano 2.0 is out.

Our new online system automatically sent e-mails to all existing owners, telling them about the beta.

And all but a few hundred of these e-mails were deleted by our commercial STMP service—the vast majority never reached their intended recipients. Apparently all the commercial e-mail hosting companies are now afraid of being tagged as "bulk-mailers," and will detect folks sending large numbers of emails in a given period.

All of which sucks.

Something our customers have asked for is better notification of updates to PDFs. The new site delivers. When you log in, it tells you if any of your PDFs have been updated. You can subscribe to your personal RSS feed and receive a notification that way. But most people seem to want to receive e-mail notifications. And now, just when we get into a position to be able to do that automatically, we seem to be blocked.

So, a question. We're not a big company. We don't have the resources to be able to manage the intricacies of mass emailing in-house. And we want something automated—we need to be able to send potentially many thousands of slightly-customized emails out when a PDF gets updated. And we need to be able to do it programmatically. Does anyone know of a decent solution?

Thanks.

Comments

[Khushil Dep]

Hi Dave,

Why not just set your own mail servers up and send from them? It's a trivial task to setup these servers and only slightly less trivial to write an app to deliver these e-mails spaced out within a time-frame to avoid 'blacklisting' by hosts which don't like mass mails. If you want to be really clever just get a nice virtualised solution whith multiple mail servers with IP's from different IP ranges. I'm surprised you use a 'commercial' smtp solution when owning your own is not a great oulay - technically nor financially.

Khushil

[slothbear]

The personal RSS feed seems like overkill. To me. How about just one public feed for all updates? Might work until you're putting out hundreds of books a month (next year??). I hope you can find a method that doesn't include account creation. I (fanboy) have an account, but it's just too much "work" for an average transaction.

[Dave Thomas]

Khushil: We did just that. They got blacklisted :)

[Chui Tey]

I use aweber. I have been with them for exactly a year. Can't beat it for about $15 a month. Does canspam, and automatic unsubscribe. Good deliverability, including aol.com addresses.

Here's a link (aff - goes towards my renewal fees) http://www.aweber.com/?216527

[Romulo A. Ceccon]

I think Keith's idea, of a mailing list, is the best option.

But you could, perhaps, setup the SMTP server for DomainKeys (http://antispam.yahoo.com/domainkeys). Although not bullet proof, both Yahoo and Google use it to help identify spammers and, unless your customers tag your messages as spam, they will always be delivered to the customers' inbox. I'm not sure, though.

Just my 2 cents.

[Victor Cosby]

Eric Hodel's ARMailer allows you to queue and send mail in specified batch sizes and delay intervals.

http://seattlerb.rubyforge.org/ar_mailer/

Usage: ar_sendmail [options]

ar_sendmail scans the email table for new messages and sends them to the
website's configured SMTP host.

ar_sendmail must be run from a Rails application's root.

Sendmail options:
-b, --batch-size BATCH_SIZE Maximum number of emails to send per delay
Default: Deliver all available emails
--delay DELAY Delay between checks for new mail
in the database
Default: 60

[Dave Thomas]

Victor:

We're already using ar_mailer, but thanks for the tip.

Dave

[Victor Cosby]

Dave:

Ack. Now I'm worried. :(

We're not sending out as many emails as you are, probably at most 400 at a time, but sounds like we're also at risk of getting blacklisted.

Would decreasing your batch size and/or increasing your delay help? If your mail provider let through 100 is that their limit?

I have to say email has been one of the most frustrating aspects of development for me personally. I hope there is a special ring in Dante's Inferno reserved for spammers because they make our job a living hell sometimes.

Victor

[Tim Morton]

There are companies that specialize in this problem. For a price, you can piggyback on their excellent reputation as non-spam bulk emailers.

The one that I know of is MailerMailer: http://www.mailermailer.com/

[Jim James]

Hi Dave,

We run into this problem with sending only 50 emails ourselves each day. We are expecting to send maybe 2000 per day by March. And we've not found any good solution to the problem.

When you _do_ find a solution you like, would you please post it back to your blog or to this posting? There are a LOT of us who are facing this problem now or soon.

Many thanks!! Jim.

[John B.]

Why not get into the "Web 2.0" world and partner with someone like Pownce or Twitter. (I prefer Pwnce personally)

Offer an account to all your subscribers and just have them add the "Prag Programmer account" as a friend.

Instance... er.. instant update.

No mail, no nothing to maintain.

Have fun.

[Hamza Khan-Cheema]

Hi,

We had the same problem and tried many solutions. We are sending out a mailing to about 50,000 emails twice a week.

We thought the solution would be to use Lyris List Manager. It seemed to be an excellent solution at the time. However to cut a long story short, we had problems with the software from day one, even with the support package. We have seen a number of improvements in the software, but none of the updates really solved our problems. It constantly crashed and was very buggy, to the extent that even if you marked a user as unsubscribed, it still delivered the email anyway. Not to mention the process power required to go through the postgreSQL database.

To move away from Lyris, we tried to use phpList, on a smaller project. Lets just say that this is not a very good piece of software and we are still having problems with it.

So we finally moved our entire mailing list over to feedblitz. Feedblitz is great, I would highly recommend it. If you also opt for the upgrade, it will allow you to control when the mail is sent out and also the template of the email. The only downside is that you would need to have a RSS feed, unless you opt for the Newsletter edition.

Campaign Monitor is good, but it costs to much.

Hamza