Optimize payment flow while reducing code complexity with Stripe's A/B Testing

In ecommerce, the payment process is the final step in converting a potential sale into revenue. However, many businesses struggle with optimizing their payment methods, often due to concerns about potential negative impacts or the complexity of implementing changes. Stripe's Payment Method A/B testing can offer a powerful solution to this common challenge.

Stripe's A/B testing feature allows businesses to experiment with different payment methods and configurations without the need for complex coding or risky full-scale deployments. By enabling easy testing and optimization of payment flows, businesses can significantly improve their conversion rates and overall customer experience.

This post explores how Stripe's Payment Method A/B testing works, its benefits, and how you can use it to boost online sales. It discusses the importance of offering the right payment methods, how to conduct effective A/B tests, and how to use Stripe's Workbench for in-depth analysis of your payment flows. insights can help you make data-driven decisions to optimize your payment process and drive business growth.

Optimizing the payment methods can improve conversion rates

Providing the appropriate payment methods tailored to your customer base can significantly impact your online payment conversion rates (CVR). A 2022 survey revealed that 81% of customers frequently abandon their carts if their preferred payment method isn't available. This allows customers to choose their familiar payment method to complete the payment flow. For example, credit cards are popular in North America, while European customers often prefer options like SEPA Direct Debit or iDEAL.

Payment preferences vary not only by country or region, but also by demographic. For example, a younger audience may not be able to apply for credit cards so you need to support digital wallets such as Apple Pay/Google Pay and BNPL payments such as Klarna. If you sell high-value items to seniors in Japan, consider also supporting bank transfers.

Displaying multiple payment methods may seem like it would require complex code with various conditional branches. The following sample code implements changing the payment methods presented depending on three currencies and the purchase amount.

app.post("/create-payment-intent", async (req, res) => {
  const { items } = req.body;
  const orderAmount = calculateOrderAmount(items)
  const orderCurrency = calculateOrderCurrency(items)
 
  let paymentMethodTypes = ["card"];
  // Add specific payment methods based on currency and amount
  switch(orderCurrency) {
  case "eur":
    paymentMethodTypes.push("giropay");
    break;
  case "gbp":
    paymentMethodTypes.push("klarna");
    if (orderAmount >= 100 && orderAmount <= 100000) {
      paymentMethodTypes.push("afterpay_clearpay");
    }
    break;
  case "usd":
    paymentMethodTypes.push("paypal");
    break;
  default:
  }

 // Create PaymentIntent with payment_method_types
  const paymentIntent = await stripe.paymentIntents.create({
    amount: orderAmount,
    currency: orderCurrency,
    payment_method_types: paymentMethodTypes,
  });

  res.send({
    clientSecret: paymentIntent.client_secret,
  });
});

There are challenges with hard-coding such payment method conditions. The first is that the amount of code to be maintained grows, which increases the number of pre-release test items and maintenance targets. The second is that changing the conditions requires deployment, making it difficult to quickly trial and error to improve conversion rates.

You can solve these problems using Stripe Elements and the Dashboard. By using Payment Elements and enabling the dynamic payment method function, you can control the country/region and amount by simply changing the settings on the Dashboard.

This is all you need to write in the API code to create a Payment Intent:

app.post("/create-payment-intent", async (req, res) => {
  const { items } = req.body;
  const orderAmount = calculateOrderAmount(items)
  const orderCurrency = calculateOrderCurrency(items)
 
 // Create PaymentIntent with payment_method_types
  const paymentIntent = await stripe.paymentIntents.create({
    amount: orderAmount,
    currency: orderCurrency,

  });

  res.send({
    clientSecret: paymentIntent.client_secret,
  });
});

The functions of Stripe Elements allow you to customize payment methods to increase conversion rates more easily.

Using A/B testing to conduct optimization experiments

While adjusting payment methods and their usage conditions can significantly increase conversion rates, these changes may also carry risks. Adjustments might decrease sales if not implemented correctly. To mitigate this risk, A/B testing is a technique that allows you to experiment with various configurations, ensuring data-driven decisions that align with your business goals.

Let's look at how to experimentally add the BNPL payment method Klarna to your application. On the Payments Settings page of your Stripe dashboard, in the Payment methods tab, you find a 'Create an experiment' button to start a new A/B test. Click this to start setting up display rules for the payment methods you want to experiment with.

On the experiment creation screen, you can turn each payment method on and off and set custom rules. If you want to introduce Klarna as an experiment, click the toggle to enable it. Click [...] to the right of the toggle to customize the conditions for displaying Klarna by amount, region, etc.

You can also add multiple payment methods at the same time and customize the rules. However, if you set multiple conditions, it can be difficult to measure the effectiveness of the experiment and investigate causal relationships. Was the improvement in conversion rate due to supporting Klarna, or was it because Amazon Pay was also supported at the same time? The purpose of an A/B test experiment is to verify a hypothesis that you have thought up in advance. Therefore, when starting an experiment, set it up in a way that makes it easier to determine causal relationships from the experiment results.

Once you have decided on the payment methods you want to experiment with, the last step is to set the percentage of traffic to test. By default, 50% of payment transactions are processed using the experimental settings. To experiment with less, set it to a lower percentage (such as 30%). Once you have reviewed the experiment content, you can start the experiment. The experiment begins at the percentage you set as soon as you click Start Experiment.

Now you can start experiments on a small scale to verify the effects of customizing how payment methods are provided.

Use the Workbench to investigate individual payment flows

For developers, investigating and debugging where an A/B test is taking place can seem difficult. This is because you need to determine whether the behavior you want to check is in A or B of the A/B test and intentionally reproduce it. When it comes to payment form experiments, while you can set up the experiment without any development resources, you may be asked to test and investigate what kind of UI and payment methods are displayed for the customer during the payment process.

Stripe makes it easier to conduct these investigations using the Workbench and Dashboard features. First, identify the Payment Intent of the payment you want to investigate. When you access the details page of that Payment Intent from the Dashboard, you can check the resource details from the Inspector tab of the Workbench. The Inspector tab offers several functions, including the Overview tab. Here you can view the raw JSON data of the resource.

For example, if you want to check which payment methods were presented to the customer in that payment, check the payment_method_types attribute from the JSON data in the Overview tab. The payment methods presented to the customer by Stripe are stored here in an array. For instance, you might see that one payment methods were presented: a credit card payment and Link.

You can also use the Workbench to see what API request created this Payment Intent. Click the Logs tab to view the API request history related to the resource you are currently viewing. By looking at the request POST body, you can determine whether the list of payment methods presented to customers was created by your application code or by settings in the Stripe Dashboard. In some cases, the request might not include payment_method_types. This would indicate that the Payment Intent was applied according to the payment method display rules set in the Stripe Dashboard.

If you want to investigate what information Stripe sent to the integrated application, use the Events tab. Here you can check the history of Webhook events related to the resource you're investigating in a list. The Events tab is particularly useful if you want to customize customer emails based on payment methods or send data to an external analysis platform. You can use it to check what data is sent and when.

By utilizing Stripe's dashboard and tools in this way, developers can respond to requests from business teams more efficiently. This approach not only reduces the time required to develop payment-related functions but also lightens the operational and maintenance burden by minimizing application code. Furthermore, you can quickly investigate customer inquiries and internal questions using the workbench.

Conclusion

In online payments, offering customers their preferred payment methods leads to improved conversion rates and sales. For example, some Stripe user implemented BNPL payments through Klarna to attract younger customers. This strategy increased their average order value (AOV) by 16%. Using Stripe, they deployed new payment methods within four days, creating a system to quickly test and implement sales-boosting strategies.

To optimize payment methods, you need a variety that is tailored to customer location and order amount. However, these customizations can complicate code and make testing and troubleshooting more challenging.

Stripe Elements allows for optimization through localization and rearrangement of payment methods without additional coding. You can enable desired payment methods in the dashboard, specifying amount limits and geographic availability as needed. To minimize risk, you can conduct A/B testing experiments directly from the dashboard. These features help recapture potentially lost sales.

During the experimental period, you can use the Inspector function in the Workbench to quickly investigate payments and gather information for testing and debugging your application.

Start optimizing your payment form with Stripe today to prevent lost sales at the final stage of your order flow.