Split Testing This Phrase Increased Our Signups By 484%

Split Testing This Phrase Increased Our Signups By 484%


Stuart McKeown’s Picture

Stuart McKeown

One simple phrase "Powered by" is used by startups everywhere. Can it be improved?

In November we wrote a post with 16 growth hacking tips that have resulted in some sort of positive metric growth for us.

One of the tips talked about how our widget is one of our biggest drivers of new signups (over 30%), I also mentioned that I really wanted to see if we could squeeze this number higher. Since our widget reaches over 1.3M eyeballs a month currently, this is a very scalable growth hack.

By default we include a Powered by Gleam link at the bottom of our widget. The clickthrough rate on this is approximately 0.006% which means that roughly for every 1M unique users we’ll get ~6000 users click through to our competitions page.

Here’s the footer link in action:

Of the users that clickthrough this page normally converts at 3.88% which means for every 1M uniques we drive roughly 233 new signups. This isn’t a bad marketing channel for us now that we’re pushing upwards of 1M sessions a month.

This also means that there’s also 233 opportunities per 1M users for our existing customers to earn referral credit.

The Experiment

We knew we could do better, so we decided to come up with 12 variations which we put into a Javascript array to cycle evenly, we then tracked the clickthroughs & conversion rates in Google Analytics.

These variations only ran on customers that were using the Free version of our product:

  • Powered by Gleam (control)
  • Run a Free Contest Like This
  • Create a Free Competition
  • Run Free Giveaways
  • Run A Free Giveaway
  • Run Free Giveaways with Gleam
  • Created With Gleam Competitions
  • Run Your Own Competition
  • Run Your Own Competition Like This
  • Start Using Gleam For Your Business
  • Gleam Rocks For Competitions
  • Not Using Gleam Yet?

We let the experiment run for 60 days to collect data.

Astonishing Results

After 60 days I checked in on the experiment & was very surprised at some of the data that was coming back. Changing the phrase has a noticable impact on 2 particular metrics.

Number of clicks on the footer message

This metric was really what we initially intended to try & improve. We had a good idea of the conversion rate of our landing pages, so we simply thought if we send more traffic they will convert at roughly the same rate (but boy were we wrong).

Number of conversions post click

This is something that I was not expecting at all, as you’ll see in the data below. The phrase used in the widget had a considerable impact on whether or not a user converted on the landing page (we made no changes to the competitions landing page at all).

Phrase VariationClicksSignupsConv Rate
Gleam Rocks For Competitions351154.27%
Run a Free Contest Like This350236.57%
Create a Free Competition328185.49%
Not Using Gleam Yet?26846**17.16%**
Run Free Giveaways268155.60%
Run Free Giveaways with Gleam231156.49%
Run A Free Giveaway216115.09%
Powered by Gleam20683.88%
Created With Gleam Competitions206125.83%
Start Using Gleam For Your Business188189.57%
Run Your Own Competition144139.03%
Run Your Own Competition Like This144106.94%

What this data tells us is that users are anywhere up to 70% more likely to clickthrough depending on the phrase used at the bottom of the widget. But they are anywhere up to 442% more likely to convert depending on what phrase is used.

Another datapoint that was interesting is that our original “Powered by Gleam” phrase that we’ve had in place for almost a year gets the poorest results from any variation. This is a typical phrase that a lot of businesses will throw in their widgets by default.

What Increases Did We See?

This means that if we took our original calculation:

( 1M impressions x 0.006 CTR ) x 3.88% = 233 signups per 1M impressions

Now lets do the same equation taking into consideration the increased CTR & conversion rate:

( 1M impressions x 0.0078 CTR) x 17.16% = 1338 signups per 1M impressions

That’s a total increase of 474% just by changing one footer phrase on our widget. Pretty incredible.

Want Tips On How You Can Grow Your Own Business?