Part
01
of one
Part
01
I need an overview/comparison of A/B site testing tools that can be used on sitecore CMS comparing features, costs, and ease of use. Major players would include: VWO, Optimizely, Sitespect, and Monetate. Baked within this question is what additional benefits adding these services have versus basic sitecore CMS
Of the major companies that offer solutions with A/B testing features, Optimizely scored the best in terms of critical response and customer satisfaction. However, unless there is a specific feature not offered by Sitecore, it does not appear that any of these products would be a beneficial add-on to Sitecore's own testing platform. Sitecore offers A/B testing and the other major solutions are either redundant, direct competitors or not recommended as an add-on product.
In order to reach this conclusion, we compared four products with A/B testing features: Optimizely, SiteSpect, Monetate and Visual Website Optimizer (also known as VWO). Details about pricing and main features, as well as general critical and customer responses were compiled for these products.
We then specifically compared these products with the features already available within Sitecore.
Overview Of Major A/B Testing Solutions
FinancesOnline, a website dedicated to evaluations of B2B and finance software, conducted a series of reviews of the four products in question. It gave Optimizely the best overall score, marking it a 9.0 out of 10. VWO came in second with an 8.3, followed by Monetate with a 7.8. SiteSpect received the lowest grade by far, coming in at just 6.8.
When comparing customer satisfaction, though, the order came in slightly different. Optimizely still scored the best, with a 100% customer satisfaction score, as compiled by FinancesOnline. However, second on the list was SiteSpect, which had the lowest critical grade. It had a customer satisfaction rating of 95%. Again, VWO and Monetate were roughly comparable with customer satisfaction scores of 88% and 92%, respectively.
Breakdown Of Major Providers
FinancesOnline Score: 8.3/10
Customer Satisfaction: 88%
Price:
$49/month standard plan
$129/month business plan
Main Features:
Personalization
Analytics
Reports
FinancesOnline Score: 9.0/10
Customer Satisfaction: 100%
Price: Quote Based (Free Trial)
Main Features:
Predictive Analytics
Mobile Compatible
Multi Domain Support
FinancesOnline Score: 6.8/10
Customer Satisfaction: 95%
Price: Quote Based (Free Trial)
Main Features:
Automation
Personalizaton
Single Page Development
FinancesOnline Score: 7.8/10
Customer Satisfaction: 92%
Price: Quote Based (No Free Trial)
Main Features:
Data Collection
Personalization
Integration Analysis With Sitecore
When specifically analyzing the possibility of using the major A/B testing solutions with a Sitecore CMS, it becomes clear that the other major competitors are mostly redundant.
Optimizely scored the highest rating for both critical response and customer satisfaction. However, in the forum on the Optimizely website, a customer specifically asked about the possibility of using Optimizely and Sitecore together. The only response on the forum is from a company representative from Optimizely, who explicitly stated that he did not see a good reason to integrate the two.
"As I understand by visiting Sitecore website, they also provide A/B and multivariate testing. This is why I would not recommend using them both or at least not on the same page. This could create conflicts as they would both make changes on the same page."
Meanwhile, there are a number of reviews expressly comparing Monetate and Sitecore. They are shown as being roughly equivalent. TrustRadius, compiling more than 90 reviews and ratings, gave Monetate a 7.3 out of 10 versus a 7.5 for Sitecore. The overall impression is that there wouldn't be much value in using the two services in parallel.
There appear to be no explicit discussions about using SiteSpect in tandem with Sitecore. This suggests that integration of the two is not common. Given its relatively low critical score, it does not seem likely that SiteSpect would offer any tangible benefit compared with Sitecore's built-in features.
Similarly, there are no specific discussions of an integration between VWO and Sitecore. This suggests that such integrations are rare and likely more cumbersome than they are worth.
Conclusion
Most of the popular options for A/B testing solutions offer comparable products, both in terms of features and pricing structures. Optimizely scores better than the other three included in this report on both critical and customer ratings. However, it seems rare that these solutions are used as add-ons for Sitecore's own A/B features, to the extent that Optimizely explicitly states that it sees no reason to use the two together. As such, there appears to be little value of spending anything additional on a second solution.