I built the handcrafted version of an automation failure mode back in 2016.
One of my early ML experiments.
At the time, I was using a popular lead generation / classified site for consulting leads, and they were still publishing raw email addresses.
This struck me as an easy opportunity to drive some business globally!
So I built a system that used a statistical prediction graph to reach out to people who seemed to be looking for the kind of consulting work I wanted to do.
Pretty simple idea.
Build a graph of interests from the post. Infer company size and budget from the category, the description, and how articulate the poster was. Then generate a lead email tailored to those interests and invariants, something that could plausibly pass as a legitimate response from a thoughtful template.
Easy, right?
I tested it on my own account across about 7,000 global ads. Good sample size. Everything looked pristine.
Better than pristine, honestly. The messages looked expensive. Thoughtful. Credible. Exactly the right vibe for what I was doing at the time.
I don’t remember exactly how long it took. I want to say it was a large-medium t-shirt sized project, so by my personal metrics, less than 13 story points. Probably about ten days after training the graph.
But there was a flaw in my method.
It was there then. It is there now.
When I’m excited about a project, I don’t sleep. I’ll go days, sometimes weeks, sleeping only a couple hours at a time because I’m so locked in.
So, naturally, at 4am, I hit deploy.
The tool went up, started spidering want ads, and started sending emails.
Then I went to sleep.
I woke up three hours later to a five alarm fire in my inbox.
People were furious.
Not confused. Actually. Fucking. Furious.
There was also a message from my web hosting provider asking if I had been hacked.
I had no idea why.
So I shut the system off and checked the logs.
This was bad.
The system had sent individual leads hundreds of unique, personalized, totally different cold emails.
They looked amazing.
But holy shit.
There were two key problems.
First, the part of the system that was supposed to check whether someone had already been contacted had an SQL error I didn’t catch. In practice, that meant one cold lead could receive an unlimited number of emails.
Second, because I deployed tired at 4am, there was a typo in the cron job.
Instead of sending once every four hours, it was sending once every four seconds.
I repaired the system, apologized to as many people as I could, and then spent the next two weeks dealing with bureaucracy. I had been reported to Spamhaus and friends, and I had to work pretty hard to get myself removed from those lists.
I think that is the one time in my career I made the spam lists and actually deserved it.
Not because I intended to spam anyone.
But rather because I had built a machine that could do my incompetence at scale.
The funny thing is, after all that, the system worked great for about a year.
No major issues.
But I learned a few things from the experience:
1. Never give any system unbounded email sending capabilities.
2. Never deploy when tired.
3. Always unit test your code.