To test different subject lines and email body copies for all of your cold email campaigns, find the best performers, and make data-driven decisions, you can use the A/Z testing feature to create variants.
This feature is available on Hyper Growth or above email outreach plans.
Basic A/Z Testing
To test different variants of your email copy and subject lines, please follow these steps to create variants:
Create a new campaign or open the target campaign
Click on "Add variant" to create a new version of the sequence(same procedure to add variants for other follow-up steps). You can test as many variants as you want in a campaign.
Use the toggle to enable or pause variants.
When the toggle is blue, the variant is enabled.
When the toggle is grey, the variant is paused.
Here are the steps to check the analytics for each variant:
Go to the Analytics section of the campaign
Scroll down to find the Step Analytics
View the performance of each variant and select the one with the best results for your campaign. It’s recommended to select a longer time range(Last 4 weeks, etc) to see the complete results.
You can use the toggle to easily turn off variants
Note: Keep in mind, if you want to keep the emails threaded, the subject line of the second email should remain the same as in the previously sent email (Step 1). You can create multiple variations for the email body.
If you change the subject line in the Step 2 variants or any other steps, the email from that variant will not be threaded.
Auto optimize A/Z testing
When you enable the auto optimize A/Z test feature, our algorithm analyzes variants automatically (A, B, C...Z) to determine which version performs the best based on a defined winning metric - reply rate, click rate, or open rate.
Once the system identifies the highest-performing variant, it automatically deactivates the other, less effective versions. This process ensures that the sent-out emails are the ones with the highest likelihood of achieving your campaign's goals.
Go to Campaign Options -> Advanced Options -> Auto optimize A/Z testing
Select the winning metric - reply rate/click rate/open rate
Don't forget to click Save.
You can watch the details about this new feature in Instantly by clicking the link here.
A/Z variants not being distributed evenly
By default, our system balances A/Z testing variant usage over the entire lifetime of a campaign, which requires looking at the overall email volume, not just recent data. This means that if you add new variants to a campaign that has been running for a while, only the new variants will be used until their usage is balanced with the older variants.
However, variants were actually sent evenly when you view the analytics over a longer time range.
You can also enable the "Reset A/Z variant usage tracking daily" option on the Preference page to reset variant usage tracking daily if needed, which will override the default behavior and send out all variants equally each day.
FAQ
Q: Will Step 2 A follow Step 1 A? Or is it random?
A: There is no option to match variants. It will be random because you can have multiple variants of each step. (ex. 3 variants for Step 1 and 5 variants for Step 2).
The assignment of variants in each step is random, and there is no option to match variants across steps. For example, if you have 3 variants in Step 1 and 5 variants in Step 2, a lead who received Step 1 variant A could receive any of the Step 2 variants (A, B, C, etc.) in the next step.