Run experiments on your website

Editors set up experiments in CMS Orbit: a variant of one block, such as the hero, or of a whole page, a goal such as Booking paid, and the share of visitors taking part. See Run an experiment. This page is about the website's side.

With @bifrost-orbit-cms/sdk 0.11 or later and pages rendered with CmsPage or CmsLivePage, experiments run with no code. Add one line to your proxy so visitors can take part without cookies, and subscribe your webhook to experiment changes.

What the SDK does

A page with an experiment arrives with page.experiments, which carries every variant. The answer is the same for every visitor, so it is cached once everywhere, and the visitor's browser picks the variant:

1. CmsPage renders each tested block once per variant, each in a wrapper with display: contents, so the wrappers don't change your layout. Only the original is visible in the HTML; the variants are hidden and carry data-nosnippet, so search engines don't quote them. A whole-page test renders the whole block list once per variant.
2. A small inline script, rendered by CmsPage before the blocks, decides each visitor's variant while the HTML is parsed, before the first paint, so there is no flicker. React then hydrates the same markup.
3. The script reports an exposure once the tested part is on screen, and conversions when the visitor reaches a goal. Both go to CMS Orbit's collector in small batches.
4. After client-side navigation a MutationObserver finds the next page's experiments, so it works with the Next.js router.

Visitors without JavaScript, bots, and visitors with Do Not Track or Global Privacy Control see the original and aren't counted.

Give visitors a variant without cookies

The script needs a seed per visitor to choose a variant. With the space's default setting, visitors who accept cookies get the cms_ab cookie; before that, and with the setting Cookieless only, the seed comes from your server. In Next.js, wrap the proxy's response with withExperiments:

1import { NextResponse, type NextRequest } from 'next/server'2import { withExperiments } from '@bifrost-orbit-cms/sdk/next/experiments'34export function proxy(request: NextRequest) {5  return withExperiments(request, NextResponse.next())6}

If your proxy already returns a response, pass that instead, as northwilds.com does with its language rewrites: return withExperiments(request, route(request)).

Set CMS_EXPERIMENTS_SECRET on the server to a long random value, the same on every server, and keep it secret:

CMS_EXPERIMENTS_SECRET=8f3c…  # openssl rand -hex 32

Without it the SDK makes a random secret each time the server starts, so visitors get a new variant after every restart, and a different one on each server.

withExperiments works out a seed for the day: HMAC-SHA-256 of the date, your host, the visitor's address with its last part removed (the last byte of an IPv4 address, everything after the first three groups of an IPv6 address) and the browser's user agent, keyed with the secret. It reads the address from CF-Connecting-IP, X-Real-IP or the first X-Forwarded-For entry. The seed goes to the browser in the page's Server-Timing header, with the visitor's country from CF-IPCountry or X-Vercel-IP-Country for country targeting:

Server-Timing: cmsab;desc="3f9a1c0b7e2d4a6c8b10", cmsgeo;desc="NO"

Only GET page loads get it; prefetches, bots, and requests with Do Not Track or Global Privacy Control don't. The seed is never stored, and the page's HTML stays the same for everyone.

Not on Next.js? experimentSeed(request) from the same entry point returns the seed for any Fetch API Request, or null, and you set the header yourself.

Templates that don't use CmsPage

Pages your website renders with its own template, such as a trip or product page, can't show block variants, but goals on them still need to count, such as a booking made on a trip page for an experiment on the home page. Add CmsExperiments to them:

1import { CmsExperiments } from '@bifrost-orbit-cms/sdk/react'23<>4  <CmsExperiments page={page} />5  <TripTemplate page={page} />6</>

It renders nothing when the page has no experiments and no goals.

Page fields

A whole-page variant can change the page's own fields, such as its title, which your template reads. useExperimentPage lays the visitor's variant over them in a client component:

1'use client'2import type { CmsPage } from '@bifrost-orbit-cms/sdk'3import { useExperimentPage } from '@bifrost-orbit-cms/sdk/react'45export function PageTitle({ page }: { page: CmsPage }) {6  const shown = useExperimentPage(page)7  return <h1>{shown.fields.title as string}</h1>8}

The first render shows the original, as the server did, and the variant follows straight after hydration. Blocks don't need this.

Count your own events

Goals of the kind An event match an event name, ignoring case. The script counts:

• convertExperiments('Booking paid') from @bifrost-orbit-cms/sdk/react
• window.dispatchEvent(new CustomEvent('cms-orbit:convert', { detail: { name: 'Booking paid' } })), for code without the SDK
• every insights.track(name) call, when Insights Orbit's script is on the page, such as Booking paid when the website's booking form reports a paid booking to Insights Orbit

When Insights Orbit's script is on the page, each exposure is also sent to it as an Experiment viewed event with the experiment's key (experiment), the variant's name (variant) and its key, a to d (arm). If the Insights script loads later than the experiment is seen, the event waits for it. Insights Orbit remembers the variant for the rest of the visit, without a cookie, so its reports, goals and funnels can be split by variant. This needs version 0.11.0 of the SDK or later.

Clicks, forms, page visits and Visitors who stay need no code. A conversion counts only for a visitor who has seen a variant of that experiment, and each goal once per visitor.

Consent

The space's setting decides when the cms_ab cookie may be used: Cookie after consent (the default), Cookieless only or Cookie without asking. With Cookie after consent, the script treats the visitor as consenting when:

• your consent banner calls experimentsConsent(true) from @bifrost-orbit-cms/sdk/react
• window.cmsOrbitConsent is true when the page loads, or Google Consent Mode has analytics_storage granted in dataLayer by then
• Insights Orbit's insights.consent() is called

Call experimentsConsent(true) when the visitor accepts during the visit: the other two are read once, when the script first chooses a variant. experimentsConsent(false) or insights.consent(false) deletes the cookie and stops storing variants. window.cmsOrbitConsent = false overrides Google Consent Mode.

The cookie is first-party, lasts 13 months and holds a random ID, when it was set and the visitor's variant in each experiment: cms_ab=<id>.<set at, seconds in base 36>.<experiment>-<variant>_<experiment>-<variant>. Only the cookie tells new visitors from returning ones (returning means it was set more than 30 minutes ago), so experiments aimed at new or returning visitors show cookieless visitors the original.

Caching and refresh

The answer for a page is the same for every visitor, so nothing about your caching changes. Starting, pausing, stopping or choosing a winner changes the answer of the page, and running experiments add their goals to every page's answer, so CMS Orbit sends the experiments.changed webhook and the SDK's createRevalidateHandler refreshes every page (cms).

New webhooks are subscribed to Experiment changed already. Webhooks that listen for Page published get experiments.changed too, so a website set up earlier refreshes without changes. A webhook with neither shows a change once the page's cache runs out. See Refresh the website when content changes and Events and webhooks.

In the editor's preview, CmsLivePage drops experiments and shows what is being edited, the page or a variant.

Preview a variant

Add #cms-arm=<experiment key>:<variant key> to a page's address to force a variant, such as #cms-arm=k7m2qx9wpa:b. CMS Orbit's Link and Preview link buttons do this for you. Forced views are never counted. Before the experiment starts, its variants are only in draft answers, so the link is a preview link; while it runs, the link is the live page.

What the page's answer carries

1{2  "experiments": {3    "collect": "https://…/cms/api/cdn/v1/experiments/collect",4    "assignment": "consent",5    "tests": [6      {7        "key": "k7m2qx9wpa",8        "status": "running",9        "control": "a",10        "traffic": 100,11        "audience": { "devices": [], "visitors": "all", "utm": { "source": null, "medium": null, "campaign": null }, "countries": [] },12        "arms": [13          { "key": "a", "name": "Original", "weight": 50 },14          { "key": "b", "name": "Variant B", "weight": 50, "swap": { "hero-1": { "id": "hero-1", "type": "hero", "fields": { "heading": "Ski the Lofoten ridges" } } } }15        ],16        "goals": [17          { "key": "goal", "goal": { "kind": "event", "name": "Booking paid" } },18          { "key": "engaged", "goal": { "kind": "engaged" } }19        ]20      }21    ],22    "goals": [{ "key": "k7m2qx9wpa", "goals": [{ "key": "goal", "goal": { "kind": "event", "name": "Booking paid" } }] }]23  }24}

Goal keys are goal for the experiment's goal, g1 to g3 for guardrails and engaged for Visitors who stay. A goal's kind is click, form, page (path and match, exact or prefix), event (name), insights (an Insights Orbit goal, whose rule is a page or an event rule) or engaged.

The collector

POST the collect address with a JSON body of up to 20 events and 16 KB. It needs no key and allows any origin.

1{2  "e": [3    { "x": "k7m2qx9wpa", "v": "1gpjbpa07yg5e0", "t": "exposure", "a": "b" },4    { "x": "k7m2qx9wpa", "v": "1gpjbpa07yg5e0", "t": "conversion", "g": ["goal", "engaged"] }5  ]6}

It answers { "data": { "accepted": 2 } }. Only running experiments count; a visitor counts once, on their first exposure, and each goal once for a visitor who has an exposure. Requests without a user agent, from bots or with Do Not Track or Global Privacy Control get 204 and count nothing. Too many requests from one address get 429 with Retry-After, a body over 16 KB 413, and a body that can't be read 400.

Bucketing

Every SDK must put the same visitor in the same variant, so the choice is a fixed calculation from the seed. The seed is the cms_ab cookie's ID, or the cookieless seed from the server.

• Hash: 32-bit FNV-1a: start at 0x811c9dc5; for each UTF-16 code unit, XOR it in, then multiply by 0x01000193 modulo 2³². Keys and seeds are plain ASCII, so bytes work the same.
• Taking part: fnv1a("${seed}:t:${key}") % 10000. If that is at least traffic × 100, the visitor sees the control and isn't counted.
• Variant: bucket = fnv1a("${seed}:${key}") % 10000, point = bucket × total / 10000, where total is the sum of the weights (negative weights count as 0). Walk the arms in order, adding each weight; the first arm where point is below the running sum is the variant.
• Visitor key: pad7(fnv1a("${seed}:v:${key}")) + pad7(fnv1a("${key}:v:${seed}")), where pad7 writes the number in base 36 and keeps the last 7 characters, padded with 0.

A variant stored in the cms_ab cookie wins over the calculation. A winner test shows the arm with weight 100; a preview test shows the control unless forced.

Test vectors, from the JavaScript SDK (fnv1a, experimentBucket, experimentArm, experimentVisitorKey):

The arms are a, b (50/50) and a, b, c (34/33/33), in that order.

Other languages

The PHP, Swift and Kotlin SDKs read experiments from the page's answer like any other field. Until they render experiments themselves, a website in another language can:

1. Show the original by default. Doing nothing is always safe: visitors see the page as it is published and nothing is counted.
2. To run experiments, pick each visitor's variant with the bucketing above, from the cms_ab cookie or a daily seed made on your server as described for withExperiments. Respect the space's assignment, Do Not Track and Global Privacy Control.
3. Render the variant's blocks from swap or components, and its fields from fields. A server-rendered page that varies per visitor must not be cached for everyone.
4. Post an exposure when the visitor sees the tested part, and conversions for the goals, to the collector. Post from the browser where you can: the collector ignores requests without a user agent or with one that looks like a tool, such as curl or python, and limits requests per address.

The simplest route on a server-rendered website is to include the SDK's runtime: EXPERIMENTS_RUNTIME and the settings as <script type="application/json" data-cms-experiments>, with each arm in an element with data-cms-x (the experiment's key) and data-cms-arm (the variant's key), as CmsPage renders them.

Privacy and the GDPR

• Cookieless by default before consent. Nothing is stored on or read from the visitor's device; the seed is made on your server, changes daily and can't be turned back into an address without your secret. This is designed to need no consent under ePrivacy rules, but whether it does in your country is for your data protection officer to judge.
• The cookie only with consent, unless the space chooses Cookie without asking, which is for places where the law allows it, such as outside the EU or a setup the French CNIL exempts for A/B testing.
• Minimal data in CMS Orbit. The collector receives an experiment key, a pseudonymous visitor key that differs per experiment, the variant and goal keys. It stores no address, user agent or cookie.
• Kept only as long as needed. Each visitor's record is deleted at the latest 90 days after the experiment ends; the daily totals stay with the experiment. Removing the experiment deletes both.
• Opt-outs are respected. Do Not Track and Global Privacy Control mean the original and no counting, in the browser and in the collector.
• Hosted in the EU, with the rest of CMS Orbit.