Been running tests for a while now and noticed some patterns emerging in how teams approach experimentation.
Seeing more focus on micro-conversions rather than just final purchase rates. Also noticing shifts toward longer test durations to account for user behavior cycles.
What changes have you spotted in how ab testing is evolving lately?
Most people overthink it now. Just run simple tests.
Sequential testing is getting more attention from what I’ve seen. Instead of running everything at once, teams test one element, let it run, then build the next test based on those results.
Made this shift on a travel app recently. We tested the signup flow first, found what worked, then moved to onboarding. Got cleaner data and better results than when we tried testing multiple parts simultaneously.
Also noticing more teams are finally tracking business metrics alongside conversion rates. Revenue per user tells a different story than just signup numbers.
We track retention more now instead of just installs. App store optimization changed how we measure success too.
Testing is now more focused on specific user segments instead of just broad groups. Teams understand that experienced users and newcomers act differently, which can lead to unclear results if tested together.
I also notice more experimentation across user flows over multiple sessions instead of just one. Most users do not convert right away, so this approach makes sense.
The timing aspect you mentioned is definitely important.