Split Testing This Phrase Increased Our Signups By 484%

One simple phrase "Powered by" is used by startups everywhere. Can it be improved?

In November we wrote a post with 16 growth hacking tips that have resulted in some sort of positive metric growth for us.

One of the tips talked about how our widget is one of our biggest drivers of new signups (over 30%), I also mentioned that I really wanted to see if we could squeeze this number higher. Since our widget reaches over 1.3M eyeballs a month currently, this is a very scalable growth hack.

By default we include a Powered by Gleam link at the bottom of our widget. The clickthrough rate on this is approximately 0.006% which means that roughly for every 1M unique users we’ll get ~6000 users click through to our competitions page.

Here’s the footer link in action:

  Learn More   Use This Template

Of the users that clickthrough this page normally converts at 3.88% which means for every 1M uniques we drive roughly 233 new signups. This isn’t a bad marketing channel for us now that we’re pushing upwards of 1M sessions a month.

This also means that there’s also 233 opportunities per 1M users for our existing customers to earn referral credit.

The Experiment

We knew we could do better, so we decided to come up with 12 variations which we put into a Javascript array to cycle evenly, we then tracked the clickthroughs & conversion rates in Google Analytics.

These variations only ran on customers that were using the Free version of our product:

  • Powered by Gleam (control)
  • Run a Free Contest Like This
  • Create a Free Competition
  • Run Free Giveaways
  • Run A Free Giveaway
  • Run Free Giveaways with Gleam
  • Created With Gleam Competitions
  • Run Your Own Competition
  • Run Your Own Competition Like This
  • Start Using Gleam For Your Business
  • Gleam Rocks For Competitions
  • Not Using Gleam Yet?

We let the experiment run for 60 days to collect data.

Astonishing Results

After 60 days I checked in on the experiment & was very surprised at some of the data that was coming back. Changing the phrase has a noticeable impact on 2 particular metrics.

This metric was really what we initially intended to try & improve. We had a good idea of the conversion rate of our landing pages, so we simply thought if we send more traffic they will convert at roughly the same rate (but boy were we wrong).

Number of conversions post click

This is something that I was not expecting at all, as you’ll see in the data below. The phrase used in the widget had a considerable impact on whether or not a user converted on the landing page (we made no changes to the competitions landing page at all).

Phase Variation Clicks Signups Conv Rate
Gleam Rocks For Competitions 351 15 4.27%
Run a Free Contest Like This 350 23 6.57%
Create a Free Competition 328 18 5.49%
Not Using Gleam Yet? 268 46 17.16%
Run Free Giveaways 268 15 5.60%
Run Free Giveaways with Gleam 231 15 6.49%
Run A Free Giveaway 216 11 5.09%
Powered by Gleam 206 8 3.88%
Created With Gleam Competitions 206 12 5.83%
Start Using Gleam For Your Business 188 18 9.57%
Run Your Own Competition 144 13 9.03%
Run Your Own Competition Like This 144 10 6.94%

What this data tells us is that users are anywhere up to 70% more likely to clickthrough depending on the phrase used at the bottom of the widget. But they are anywhere up to 442% more likely to convert depending on what phrase is used.

Gleam Widget Split Tested Footer

Another datapoint that was interesting is that our original “Powered by Gleam” phrase that we’ve had in place for almost a year gets the poorest results from any variation. This is a typical phrase that a lot of businesses will throw in their widgets by default.

What Increases Did We See?

This means that if we took our original calculation:

( 1M impressions x 0.006 CTR ) x 3.88% = 233 signups per 1M impressions

Now lets do the same equation taking into consideration the increased CTR & conversion rate:

( 1M impressions x 0.0078 CTR) x 17.16% = 1338 signups per 1M impressions

That’s a total increase of 474% just by changing one footer phrase on our widget. Pretty incredible.

Author

Stuart McKeown

Stuart McKeown is one of the Co-founders at Gleam. Aside from writing and helping businesses grow, he also enjoys sound design and drinking tea ☕️