A/B testing adaptations based on possibilistic reward methods for checkout processes: a numerical analysis

Abstract

A/B Testing can be used in digital contexts to optimize the e-commerce purchasing process so as to reduce customer effort during online purchasing and assure that the largest possible number of customers place their order. In this paper we focus on the checkout process. Most of the companies are very interested in agilice this process in order to reduce the customer abandon rate during the purchase sequence and to increase the customer satisfaction. In this paper, we use an adaptation of A/B testing based on multi-armed bandit algorithms, which also includes the definition of alternative stopping criteria. In real contexts, where the family to which the reward distribution belongs is unknown, the possibilistic reward (PR) methods become a powerful alternative. In PR methods, the probability distribution of the expected rewards is approximately modeled and only the minimum and maximum reward bounds have to be known. A comparative numerical analysis based on the simulation of real checkout process scenarios is used to analyze the performance of the proposed A/B testing adaptations in non-Bernoulli environments. The conclusion is that the PR3 method can be efficiently used in such environments in combination with the PR3-based stopping criteria.