Skip to main content

Example: Testing a PayPal Button

Beta

Experimentation is in beta.

Our app currently accepts credit card payments only. We suspect offering PayPal will increase completed checkouts, and we want to prove it on a small slice of traffic before rolling it out. This page walks through the whole experiment.

1. Create the flag​

Create a multivariate flag called paypal_button. The flag's current behaviour (no PayPal) is the control; add one variation named show-paypal.

Your code will branch on the variation name, not the flag value, so keep names short and stable.

2. Create the metric​

With a warehouse connected, create a metric:

  • Name: Checkout Completion
  • Measure: Occurrence
  • Direction: Higher is better
  • Event name: checkout_completed

3. Create the experiment​

Create an experiment on paypal_button:

  • Hypothesis: "Offering PayPal at checkout will increase checkout completion by at least 10% within 30 days."
  • Rollout: 10%, split evenly. 5% of identities see PayPal, 5% act as control, and the remaining 90% stay out of the experiment entirely.
  • Primary metric: Checkout Completion, expected to increase.

Launch it from the Review step.

4. Instrument the checkout​

On the checkout page, resolve the flag (recording the exposure) and branch on the variation name. When a checkout completes, send the conversion event. The SDK must be initialised with enableEvents: true and the user identified; see Run an Experiment.

import flagsmith from '@flagsmith/flagsmith';
import { useExperiment } from '@flagsmith/flagsmith/react';

const CheckoutPage: React.FC = () => {
 // Evaluates the flag and records the exposure
 const flag = useExperiment('paypal_button');
 const showPaypal = flag?.enabled && flag?.variant === 'show-paypal';

 const onOrderConfirmed = () => {
  // Record the conversion
  flagsmith.trackEvent('checkout_completed');
 };

 return (
  <>
   <CardPaymentForm onConfirmed={onOrderConfirmed} />
   {showPaypal ? <PayPalButton onConfirmed={onOrderConfirmed} /> : null}
  </>
 );
};

Outside React, flagsmith.getExperimentFlag('paypal_button') does the same job as the hook. The same flow works in Python; see Run an Experiment for both SDKs.

If checkout is available to visitors who aren't logged in, identify them with a persistent anonymous GUID so bucketing and events stay consistent; see Identities.

5. Read the results and roll out​

On the experiment page, check the Exposures panel: enrolment should track about 10% of your identities, split 50/50. Then watch the results as conversions arrive; see Analyse an Experiment.

When the result is conclusive, end the experiment, roll the PayPal button out to everyone, and clean up the flag.