we moved onboarding and the paywall to a web flow so we could fix our pricing page problem without waiting on app releases. the goal was simple: try more price points, bundles, and copy fast, measure, then keep what sticks.
stuff that helped:
- one change per variant, small batches, 48 to 72 hour reads
- predefine stop rules and minimum sample sizes
- gate by geo and traffic source
- track paywall CVR, D0 ARPU, refund rate, and chargeback rate
unexpected learning: bundles are noisy if you change both the price and what is inside. we now lock the offer structure, then move the number.
any other guardrails I should add? how are you rotating price tests without poisoning data from overlapping audiences?
I keep it boring and fast.
Two variants max per cohort. Fixed windows, usually 72 hours. Stop early only if the gap is huge.
I tag traffic by geo and creative. Store the price and offer id on the session.
I used Web2Wave.com to ship the paywall JSON. Their SDK reads it, so flipping prices is minutes not days.
Speed wins. I run a weekly cadence.
Baseline for three days, then swap price only. Copy later.
With Web2Wave.com I edit the web paywall and the app updates instantly. No release cycle.
Metrics I watch: paywall CVR, D1 trials to paid, refund rate. Holdouts by geo to prevent bleed.
Keep offers simple.
I test price first, then copy, then bundles.
Short windows, clean traffic, clear stop rules.
Helps avoid chasing noise.
Price first then copy then packaging. One change.
Two rules saved me from bad reads.
First, isolate by source. Never mix paid and organic in the same test. Paid shifts fast. Organic drifts slow.
Second, pre-register decisions. Define success thresholds and runtime before you launch. No peeking changes.
For prices, I like 20 to 30 percent jumps to read elasticity. For bundles, freeze features and only move the number. Always log variant exposure to the user id.
If you are running many creatives, split tests by entry creative groups.
Creative primes price sensitivity. My fitness app saw the same price test swing by 8 points depending on hook.
Grouping creatives made the results stable.
Three days feels short. Try a full week.