Split Testing This Phrase Increased Our Signups By 484%

In November we wrote a post with 16 growth hacking tips that have resulted in some sort of positive metric growth for us.

One of the tips talked about how our widget is one of our biggest drivers of new signups (over 30%), I also mentioned that I really wanted to see if we could squeeze this number higher. Since our widget reaches over 1.3M eyeballs a month currently, this is a very scalable growth hack.

By default we include a Powered by Gleam link at the bottom of our widget. The clickthrough rate on this is approximately 0.006% which means that roughly for every 1M unique users we’ll get ~6000 users click through to our competitions page.

Here’s the footer link in action:

  Learn More   Use This Template

Of the users that clickthrough this page normally converts at 3.88% which means for every 1M uniques we drive roughly 233 new signups. This isn’t a bad marketing channel for us now that we’re pushing upwards of 1M sessions a month.

This also means that there’s also 233 opportunities per 1M users for our existing customers to earn referral credit.

We knew we could do better, so we decided to come up with 12 variations which we put into a Javascript array to cycle evenly, we then tracked the clickthroughs & conversion rates in Google Analytics.

These variations only ran on customers that were using the Free version of our product:

• Powered by Gleam (control)
• Run a Free Contest Like This
• Create a Free Competition
• Run Free Giveaways
• Run A Free Giveaway
• Run Free Giveaways with Gleam
• Created With Gleam Competitions
• Run Your Own Competition
• Run Your Own Competition Like This
• Start Using Gleam For Your Business
• Gleam Rocks For Competitions
• Not Using Gleam Yet?

We let the experiment run for 60 days to collect data.

After 60 days I checked in on the experiment & was very surprised at some of the data that was coming back. Changing the phrase has a noticeable impact on 2 particular metrics.

This metric was really what we initially intended to try & improve. We had a good idea of the conversion rate of our landing pages, so we simply thought if we send more traffic they will convert at roughly the same rate (but boy were we wrong).

This is something that I was not expecting at all, as you’ll see in the data below. The phrase used in the widget had a considerable impact on whether or not a user converted on the landing page (we made no changes to the competitions landing page at all).

Phase Variation	Clicks	Signups	Conv Rate
Gleam Rocks For Competitions	351	15	4.27%
Run a Free Contest Like This	350	23	6.57%
Create a Free Competition	328	18	5.49%
Not Using Gleam Yet?	268	46	17.16%
Run Free Giveaways	268	15	5.60%
Run Free Giveaways with Gleam	231	15	6.49%
Run A Free Giveaway	216	11	5.09%
Powered by Gleam	206	8	3.88%
Created With Gleam Competitions	206	12	5.83%
Start Using Gleam For Your Business	188	18	9.57%
Run Your Own Competition	144	13	9.03%
Run Your Own Competition Like This	144	10	6.94%

What this data tells us is that users are anywhere up to 70% more likely to clickthrough depending on the phrase used at the bottom of the widget. But they are anywhere up to 442% more likely to convert depending on what phrase is used.

Another datapoint that was interesting is that our original “Powered by Gleam” phrase that we’ve had in place for almost a year gets the poorest results from any variation. This is a typical phrase that a lot of businesses will throw in their widgets by default.

This means that if we took our original calculation:

( 1M impressions x 0.006 CTR ) x 3.88% = 233 signups per 1M impressions

Now lets do the same equation taking into consideration the increased CTR & conversion rate:

( 1M impressions x 0.0078 CTR) x 17.16% = 1338 signups per 1M impressions

That’s a total increase of 474% just by changing one footer phrase on our widget. Pretty incredible.