Content Marketing Experiment: Update on our Newsletter
Using Math to Show Customer Engagement in Content Marketing
Earlier today I exchanged thoughts with Mike Volpe, Hubspot’s VP of Marketing, in regards to this post on customer engagement within our newsletter. Mike is a self-described “stickler for math in marketing” so I wanted to share some additional information to see if the difference in our open rates, is in fact, statistically significant. Here goes:
I took a look at the past 9 newsletters:


Headlines for the 9 newsletters:
- 12/10: 7 tips for finding prized inside sales talent
- 12/3: Learn how to let go of your baby
- 11/26: Top 3 corporate blogging excuses
- 11/19: Should you syndicate an investment?
- 11/12: Attention CEOs, would you hire yourself?
- 11/5: It’s war, are your generals ready?
- 10/29: Q&A with a Sales Expert
- 10/22: How to progress through the expansion stage
- 10/15: What a CEO should be doing NOW
As Mike pointed out, the difference between all of the open rates is not statistically significant. Our small jumps of 1% are not a significant amount to prove that the subject lines played a role in our test. However, when comparing last week’s issue to the 10/15 issue, the data shows a difference with a 100% confidence level.

Even though this test confirms a difference in our open rates, it doesn’t necessarily indicate that the difference is based solely on the subject lines of the emails. The topic of focus on the email may have played a large role as well.
What tests are you conducting within your email program?
Be a part of the experiment! Sign up to receive weekly tips and tricks for senior managers of growing tech companies.
Subscribe with RSS or Email to get the best new ideas for
building great technology companies delivered to you.

More from Twitter…