Create an experiment

This page will help you to configure, start and stop your AB test experiment.

Prerequisites before creating an experiment

Before creating an experiment, you need to have a notice declared in the backoffice.

  • This can be a notice running since a while on your website. In that case you don't have to create a new notice and you can read the paragraph "Create an experiment" below. The experiment will run instead of your current notice.

  • Or it can be a new notice that you will implement when you will start your experiment (for example if you are a new client and want to start by an experiment). In that case, you need to create a notice first. Read this page to learn how to create a notice.

Create an experiment

To create an experiment, click on "Experiment" in the left menu.

Then click on "Add experiment" on the left top.

In the list of sources that appears, select the one on which you want to apply the experiment. It can be wether a web source or a mobile app source.

Then select the Notice Version ID you want to use for the experiment. We call this notice, the "parent version". Then click on "Create experiment". A copy of this notice will be created in the experiment.

Your experiment has now an "Open" status. Meaning that it is created, but not yet activated on your website or app.

  • ID: Experiment ID

  • Source: Source where the experiment will be implemented

  • Parent version ID: ID of the notice from the one the experiment has been created (the parent notice).

  • Note: you can add note by clicking one the 3 dots at the right. This note will not appear in the experiment on the website, it's just for you own usage (for example to distinguish 2 variations you could add this notes: Notice with banner / notice without banner).

  • Status:

    • Open: the experiment is created, but not yet activated on your website or app

    • Running: the experiment is live on your website or app

    • Done: the experiment has been stopped

  • Update date: date of the last update

  • Test start date: date when you start the experiment

  • Test end date: date when you stop the experiment

  • 3 dots at the right: if you click on this dots, you can edit the note field or delete the experiment.

To see the details of the experiment, click on the line of the experiment.

The details of the experiment are displayed:

  • Ressource url: the url is the same as the parent notice.

  • HTML script tag: the script is the same as the parent notice.

  • Start experiment: click on this button when your experiment is ready.

  • Add variation: click on this button to add a variation.

  • Line with a test number 1: this is the first variation of this experiment, automaticaly created from the parent notice.

Add notice variations for the experiment

To be run, an experiment needs at least 2 variations. We call "Variation" a notice with some settings. Each variation can have differents settings. For example: - Variation 1: notice with banner, UI in a single step, CTA accept/Configure in the banner, CTAs enabled in notice. - Variation 2: notice without banner, UI in several steps, no CTA in the notice.

There are 2 ways to create a new variation:

  • Click on "Add variation"

  • Or create a duplicate from the first variation.

1) Add a variation through the "Add variation" button

Click on "Add variation". The list of existing notice from the same parent notice appears. You can choose to create a new variation from:

  • The same parent notice

  • The 1st variation created in this experiment

  • Or a variation of another experiment, created previously from the same parent notice (if you have already created a previous experiment)

Select one or several notice(s) and click on "Apply".

A new variation is created.

2) Add a variation through the "Duplicate" button

Click on the 3 dots at the end of the line of the variation you want to copy, and click on "Duplicate".

A new variation is created.

You can create as many variations as you want, but we recommend not to create too many. The larger the number of variations, the lower their weight, and the less frequently appear on your site.

Edit and configure a variation

Each variation can be edited and configured with its own settings.

Click on the 3 dots at the end of the line of the first variation and click on "Edit".

The notice form appears, with the same configuration fields than a notice. You can modify all settings.

The field "Weight" is specific to the variation form. You need to fill in this field with a number corresponding to a percentage. This percentage will define the display frequency of the variation, compared to other variations of the experiment.

For exemple you can have a weight of 50/50 or 30/70 or 30/20/50, or 25/25/25/25, etc.

Preview or delete a variation

2 other actions are enabled for each variation: - You can preview your settings by clicking on "Preview". - You can delete a variation by clicking on "Delete".

Organize your variations

If you want to easily identify the variations in the variations table, you can modify the "Note" field in the editing form. This text will not appear in the notice on your website or app.

Start an experiment

When all your variations are ready to test, you can start the experiment.

Make sure all your settings are well defined before clicking on "Start experiment", as you could no more modify them when the experiment will be running.

Click on "Start experiment".

If you haven't fill the field "Weight" for each variation, or if the total weight is not 100%, the experiment cannot start.

There can't be 2 experiments running at the same time on the same source. If another experiment is already running on the source, you need to stop it before starting a new one.

Few seconds after clicking on "Start experiment", the experiment will run, and its status is "Running", with a "Test start date":

At this stage, the experiment can no more be modified. You can only view details (throught the variation form), but settings cannot be updated. You can still see a preview of each variation, by clicking on "Preview".

While an experiment is running, the parent notice cannot be modified or deleted.

While the experiment is running, you can check the statistics for each variation. To do so, read the dedicated section.

Stop the experiment

You can run an experiment as long as you want and stop it whenever you want. When you consider your test is done, click on "Stop experiment".

The "Stop experiment" button changes to "Confirm stop". You need to pick one of the variations before confirming. The variation you choose is the one that will be permanently implemented on your website or Mobile application.

Once you have picked one of the variations, click on "Confirm stop". The experiment stops immediately (its status is "Done" and a "Test end date" is filled in) and the notice you have picked is now running on your website or application.