How to Do A/B Testing in Push Notification?

How to increase conversion? This is the most popular question among all affiliates. If somebody finds an easy way, there will be a really high demand for it.

To be honest, there is a solution, but it’s not as easy as it seems: do a/b testing in push notification.

It’s quite a complicated scheme and requires time but it’s worth it. If you work hard and find an appropriate strategy, your efforts will be deservedly rewarded: conversions will increase and CTR will be high as well.

What is A/B push testing of creatives?

A/B test (or split-test) is an experiment, during which the user is shown different variants of ad materials (for example, push notifications) and then, according to their reactions, the successful elements that bring conversion are estimated.

A/B tests aim to identify the most appropriate variant for conversion, then scale it and get profit. It can be applied to find a profitable lander-offer combination at the start and for its optimization during campaign running. It’s necessary to avoid burning out of successful creatives.

So, you’ve made up a creative: have chosen an icon and the main image, have written a headline and description. Here is variant A from A/B term. If you change just one element in this creative, for example, the main image, you will get variant B.

At RichAds you can run creatives’ split test within usual campaign launching. You won’t need any additional instruments. Add to your campaign creatives and lander-offer combinations you want to compare. You can test simultaneously up to 10 creatives. 

All creatives are displayed on the campaign page: you can see the full version of their image, icon, and text in one place. The most important indicator of push notification’s success – CTR or click-through rate – will be shown here too. Pay attention to the creatives with the highest CTR. They will become your favorites.

Learn more about 30 Ad Strategies to Increase CTR here.

How to start an A/B test?

As we’ve just looked into the definition of push notification testing, it’s high time to figure out how to conduct the test to our advantage. Let’s discuss the plan step by step!

Step 1: Formulate a hypothesis
Step 2: Run the test
Step 3: Document the results

Step 1: Formulate a hypothesis

It is highly important to make your own analysis of the market and competitors. Describe your target audience as accurately as possible (age, gender, education, possible interests) and guess what kind of messages can capture their attention. Shall you use humor or is it better to focus on the benefits of the product? What is the expected result of your test?

Focus on meeting their needs, choose the right tone of voice for your message, work on the creatives and the necessity of using emojis.

Step 2: Run the test

Choose a few options that seem the most suitable to you and start your push notification test. You’d better think about your targeting, best bid and settings in advance. It’ll also be great to specify what the target action is for you and the approximate desired CPA.

Important: Do not forget that it is best to test one variable at a time (creatives, texts, landing pages) in order to be able to track which element worked and brought conversions.

Step 3: Document the results

Which variable worked more effectively? Which elements have you tested? Do not rely only on yourself and your memory: write down the outcome in a separate spreadsheet to observe the results of your efforts.

Make sure that you have marked the beginning and end of AB testing of notifications, the effectiveness of each elements group, as well as their impact on CR and CTR.

Most effective A/B testing elements

Now that we know the main stages of the push notifications split testing, we’ll try to understand what can be tested within one campaign and how to do it wisely.

Let’s list the most basic elements of the A/B test, then focus on some of them and provide detailed examples.

The main elements of A/B testing:

1. creatives
2. titles and texts
3. landing pages (with and without prelanding)
4. GEO
5. devices
6. additional targeting elements (browsers, carriers, OS)

Why should an A/B test begin with creatives?

Our brain processes information in quite an interesting way. For example, when you see a person for the first time, it evaluates his appearance and the first five seconds define whether we like this person or not. We’ve not listened to the person yet, but the first impression has already been made. 

Visual messages influence a person stronger than verbal signals or texts. 

The same rule also works for creatives: first people notice and evaluate pictures and icons, then the text. This will be our starting point when conducting an A/B test. First of all, you need to clarify that it is important to look at your offer from different sides and from different angles when testing creatives, as your target audience cannot and should not be identical.

When we talk about push testing, creatives are the first and most effective way to segment the audience. They allow you to target different people depending on their tastes and needs.

For example, if you launch a dating offer, you can use different approaches with the help of creatives:

• Blondes vs Brunettes
• Curvy vs Skinny
• Young vs Mature
• Selfie vs Full-length photo

 How to conduct an A/B test with creatives?

There are plenty of various approaches when it comes to creatives. Let’s have a look at variations of audience segmentation with the help of creatives. We’ll take a gambling offer as an example and try to conduct comprehensive split testing of push notifications.

Men- and women-oriented creatives

Segment your audience by gender to understand which users are more active and willing to perform your target action. You should not limit your targeting to one gender, even if it seems that the choice is obvious — everything is individual depending on your offer.

1. Women-oriented

👈 vs 👉

2. Men-oriented

• You can also focus on the user’s victory to inspire him and make the banners stand out by using fonts or bright colors.

• For some players it’s crucial to see the pleasure of playing and winning and feel the entertainment

There are a bunch of ideas that can be used to test creatives, so the more you use, the better!

Download the pack of 300 best icons for push notifications for free.

How to A/B test texts?

If you want to have a high CR, you need to attract the user not only with a banner, but also choose the right tone of your message to be heard.

Using the example of gambling offers, we can highlight several approaches:

• Value vs. Urgency

A) Urgency: Register Today and Get Your Bonus 💵 💵 💵

B) Value: Get a 100$ Bonus + 150 Free Spins 🎰

• Call to Action vs. Intrigue

A) Call to Action: Register and Win Big Prizes! 🤩

B) Intrigue: Are you our next instant millionaire?

• Uniqueness vs. Opportunity

A) Uniqueness: Welcome Bonus for New Users ✅

B) Opportunity: Get a chance to win $1M for just $10 💵

Bonus: To save your time on research, we’ll provide you useful articles on creating effective creatives for different verticals:

• Nutra
• Betting
• Antivirus
• Dating

How to A/B test landing pages?

Next step after testing the main elements is the prelanding test. It can be effective when promoting your offer, as it will allow you to highlight the useful properties of the product, show real users reviews or warn your audience about the spicy content in advance.

Perhaps the usage of a prelanding will not be suitable for your offer, on the contrary it may distract your audience from a quick purchase or deposit, while the direct link will work better. So, we’d advise to test both approaches.

As a rule, prelanding funnel works better for push notifications. The success of such campaigns is 80%. But there is always the remaining 20% which work better with direct links.

Basic recommendations for A/B tests

By this point, you have already conducted your analysis and research, formulated a hypothesis, made fresh creatives and texts, selected the best settings. It’s time to start a test, right?

Before you step on this fascinating path of A/B testing, we have a few more recommendations to simplify this process as much as possible and help you interpret the results correctly.

Test one variable at a time

As mentioned earlier, testing more than one variable at a time is not recommended. Why? It may seem that this way you’ll save time and money, but in fact the results will be opposite.

If you test a lot of variables, there is a high probability that you will not be able to track the results correctly and understand what exactly brought you conversions — creatives, texts or landing pages? Or maybe the new targeting? We advise you not to make the process complicated and test everything in turns.

Interpret the results correctly

First of all, these are the target indicators – CR and CTR. 

Remember that even if the CTR is high enough, it does not mean that it will bring you a lot of conversions. For this reason it is better to consider both indicators together to get the full picture.

Consider additional factors, perhaps, your target audience was more active at a certain time of the day or time of the year. It is also likely that the CR is too low or high due to holidays or the seasonality of the offer, these factors shall also be considered.

Measure the outcome

Is it possible to make a time limit for the test? It would be so useful: to know that in a few hours you will get relevant results. Sometimes a few hours is enough, but sometimes you need a few days. 

It’s better to measure each test not by time but by data obtained.

To measure success, check one lander-offer combination using a budget equal to 3 conversion’s prices if one conversion costs more than $10. The sum of money you spend on the test of each variant of the creative should be no less than the cost of 1 conversion.

One more variant is to rely on the sums you are ready to spend. For the test prepare 10-20% of the budget you are going to spend on your campaign, split it into a few parts for each step, and start testing. 

Split-testing is a quite time-consuming activity that can last not one day. You will have to be creative, and patient as well as be ready to spend the budget, but it’s worth it. 

Don’t stop on one working creative

If you have found a converting creative, you should not rely only on it, as creatives tend to burn out. It is vital to analyze the combinations, constantly update them with the most effective variants, and not stop at one combination. 

Change banners, icons, rephrase texts but stick to the working approach. We recommend using 5-10 creatives per campaign to achieve results. If you don’t have enough time or imagination to create new combinations frequently, you can order creatives from your manager at RichAds for free.

Conclusion

Repetition is the key to success when we talk about A/B testing of push notifications. As soon as you have completed your first test, chosen the best strategy for converting creatives, texts, landing pages and settings — start the next one! This will help you to find a win-win formula and the combination that will convert well and work for your result.

Don’t give up split-tests and get profitable lander-offer combinations with RichAds.

What is RichAds?
🔝 Push and pop ads,
🔼 Domain redirect traffic,
🔝 CPM for domain ads starts from $ 1.5,
⏫ CPC for push ads starts from $ 0.003,
🔼 CPM for pops — from $0.3 in Tier 3, $1 in Tier 1,
⏫ large volumes of available traffic in more than 200 goes.

Learn more about the five stages of testing push campaigns here