Despite the challenges of A/B testing, there are multiple solutions that have emerged to help businesses conduct effective experiments.
A/B Testing Options
Existing solutions for A/B testing run the gamut. There are bare-bones apps exposing APIs for creating and retrieving experiments from the command line. In addition, there are many What You See Is What You Get (WYSIWYG) options for performing A/B testing on client-side-rendered applications that can be used with limited developer expertise (e.g., VWO). At the other end of the spectrum are full featured A/B testing solutions that offer exposed APIs as well as SDKs for both server-side and client-side rendered applications that require significant developer expertise.
Options for A/B testing solutions that are open source with self-hosting options are relatively few – applications that look promising either have little to no development or engagement over time or they become fully-featured enterprise options that offer only expensive, managed solutions (e.g., LaunchDarkly, Optimizely).
Selected A/B Testing Providers and Features
|Self-Hosted Option||None||None||Docker||Docker, Node||None|
|Analytics Data Storage||No||No||No||No||Yes|
|Client Data Store Required||No||No||Yes||No||No|
|Developer Expertise Required||Yes||No||Yes||Yes||Yes|
|Client- or Server-Side||Client||Client||Both||Both||Both|
|GUI for Test Creation||Yes||Yes||Yes||Yes||Pro or Enterprise Plan|
|GUI for Data Visualization||Yes||Yes||Yes||No||Pro or Enterprise Plan|
Within the open-source, self-hosted category, two key options are Unleash and GrowthBook, both of which offer rich documentation and API access, with open-source deployment options. These solutions inspired some of the key features and functionality of Test Lab.