A/B Testing using Statsig with Next.js application

Written by: Bishakh Neogi

Problem Statement

Imagine you’re a developer working on a Next.js web application. You want to improve user experience by testing different features, but you’re unsure which version will perform better. It would be best to have a robust A/B testing solution that works seamlessly with your tech stack and offers reliable data-driven insights. Enter Statsig — a tool that simplifies A/B testing and helps you make better decisions based on real user data.

Ideation

Our beloved Upadhyay ji faced a daunting challenge. Upadhyay ji was working on a web application and wanted to ensure users had the best experience possible. However, Upadhyay ji was torn between two design choices for a crucial feature. Should the button be green or red? Should the headline be bold or italic? These questions plagued Upadhyay ji, and the fate of the application hung in the balance.

One day, while scouring the internet for solutions, Upadhyay ji stumbled upon the magical world of A/B testing. By comparing two versions of an app or webpage, A/B testing could identify the better performer based on real user data. Upadhyay ji realized this was the key to unlocking the application’s potential. But where to start?

Finding the perfect match?

Upadhyay ji first tried Firebase A/B testing, but alas, it was not suited for web applications.

Reference: https://groups.google.com/g/firebase-talk/c/PizmSOf1Mv4?pli=1

Frustrated, Upadhyay ji almost gave up hope. But then, a beacon of light appeared on the horizon — a tool named Statsig, renowned for its community support and ease of use. With renewed determination, Upadhyay ji decided to give Statsig a try.

Setting Up Statsig

The steps were straightforward and very easy:

1. Signup/Login to Statsig: Upadhyay ji created an account and logged in.
2. Create a Project: Upadhyay ji set up a new project in Statsig.
3. Feature-Flag Creation: Upadhyay ji moved to the feature-flag section and created a flag for the versions to be tested.
4. Code Integration: With the feature flag ready, it was time to integrate Statsig into the codebase.
   

The Integration Process

Upadhyay ji added the necessary packages:

yarn add statsig-react
yarn add statsig-node

Next, Upadhyay ji updated the environment variables and configured the Statsig provider in the _app.tsx file:

import { StatsigProvider, StatsigSynchronousProvider } from "statsig-react";
export default function App({ Component, pageProps }: AppProps) {
  const user = { userID: "1267" };
  return (
    {pageProps.initializeValues ? (
      <StatsigSynchronousProvider
        sdkKey={process.env.NEXT_PUBLIC_STATSIG_CLIENT_KEY!}
        initializeValues={pageProps.initializeValues}
        user={user}
      >
        <Component {...pageProps} />
      </StatsigSynchronousProvider>
    ) : (
      <StatsigProvider
        sdkKey={process.env.NEXT_PUBLIC_STATSIG_CLIENT_KEY!}
        waitForInitialization={true}
        user={user}
      >
        <Component {...pageProps} />
      </StatsigProvider>
    )}
  )
}

For server-side rendering, Upadhyay ji added the following code in componentName.tsx:

export async function getServerSideProps() {
  const user = { userID: "980" };
  await Statsig.initialize(process.env.NEXT_PUBLIC_STATSIG_SERVER_SECRET!);
  return {
    props: {
      initializeValues: Statsig.getClientInitializeResponse(user),
    },
  };
}

export default function ComponentName() {
  const { value } = useGate("feature");

  const discoverTextClass = cn({
    "text-green-400": value,
    "text-red-400": !value,
  });

  return (
    <p className={discoverTextClass}>Hello World</p>
  );
}

The Results

With Statsig integrated, Upadhyay ji could now run A/B tests effortlessly. For example, to test the text color of a paragraph:

• Option A: <p>Hello World</p> in green (flag set to true)
• Option B: <p>Hello World</p> in red (flag set to false)

Statsig dynamically served different versions to users and collected data on which version performed better. Based on this data, Upadhyay ji could confidently choose the winning version and improve the user experience.

Overcoming Obstacles

During the testing phase, Upadhyay ji encountered an issue with Statsig being blocked by Jio Telecom service. However, switching to Airtel resolved the problem. Upadhyay ji reported the issue to the community, who promised to look into it. Despite this hiccup, the overall experience with Statsig was smooth and rewarding.