Analytics & performance

Run an A/B Test on a VLVTN Smart Link

Updated 2026-07-18
The short answer
A clean smart-link A/B test changes one variable, splits comparable traffic between a control and variant, and measures a predefined result such as DSP click-outs per page view. Plan the sample before launch, run both versions concurrently, and accept that the result may be inconclusive. A/B testing is a Pro feature.
Note
A/B testing is a Pro feature. On a non-Pro plan the feature is not available.

Set up an A/B test

  1. Write one hypothesis and one metric
    Name the single change and the result it should affect. For a streaming landing page, use click-through rate when the outcome is DSP click-outs divided by page views.
  2. Plan the sample before launch
    Use the current conversion rate, the smallest lift worth detecting, a confidence level, and statistical power to estimate visits per group. Do not pick a stopping point after looking at early results.
  3. Create one variant from the control
    Start the test, edit only the variable named in the hypothesis, and leave artwork, copy, links, and order unchanged unless one of those is the variable being tested.
  4. Split comparable traffic
    Use equal weights for a straightforward two-group test. Run both versions at the same time, from the same campaign and audience. Keep auto-optimize off for a fixed allocation.
  5. Run to the planned sample
    Exclude your own visits, watch for broken destinations or tracking gaps, and avoid ending the test because one variant leads early.
  6. Read the result honestly
    Compare the completed groups, check the observed lift against the test plan, and keep the control when the result is inconclusive.

Choose the metric that matches the job

VLVTN offers Click-Through Rate, Email Signups, and Total Clicks as success metrics. For a music smart link whose job is to move a fan from the landing page to a DSP, Click-Through Rate is the cleanest of those options because it divides DSP clicks by page views. Total Clicks can reward a higher-traffic variant even when its rate is no better.

The dashboard leader is not proof

The results table shows views, clicks, CTR, lift, and the variant with the highest observed value. It does not calculate a confidence interval or statistical significance. A small early lead can be noise. Use the sample target you set before launch and preserve the raw counts when recording the decision.

Watch out
The built-in minimum of 100 views per variant is a product threshold, not a universal sample-size guarantee. Required traffic changes with the baseline rate, minimum detectable lift, confidence, and power.

NIST's sample-size guidance for proportions formalizes those inputs and uses a normal approximation. Read the NIST methodology before treating a planned traffic number as a guarantee.

Frequently asked

Is A/B testing a Pro feature?

Yes. A/B testing is Pro-gated. On a non-Pro plan the feature is not available.

How does A/B testing work?

VLVTN duplicates the control into variants, splits visitors by the weights you choose, and records page views, DSP clicks, click-through rate, and email signups for each version. You can create up to three total variants, including the control.

How do I know which variant won?

The results table identifies the current observed leader, but that label is not a statistical-significance test. Compare the result with the sample plan you set before launch, check data quality, and accept an inconclusive outcome when the evidence is weak.

Should I turn on auto-optimize?

Not for a clean fixed-split experiment. Auto-optimize can shift traffic toward the current leader after the product's minimum sample threshold, which changes the allocation. Keep it off when you need a simple concurrent comparison with planned equal groups.