Metadata, sitemaps and robots with the SDK

Editors manage SEO in the CMS: titles, descriptions, share images, no-index switches, redirects and the sitemap. Your website's job is to output it. The SDK does almost all of it. This page shows each part, building on Connect a website.

Page metadata

1import type { Metadata } from 'next'2import { toMetadata } from '@bifrost-orbit-cms/sdk/seo'34export async function generateMetadata(props: Props): Promise<Metadata> {5  const result = await resolveFor(props)6  return result.kind === 'page' ? toMetadata(result.page) : {}7}

toMetadata(page) reads page.seo, which the CMS has already completed with the space's defaults, and returns:

CMS Orbit checks the result on the live website about once a day: it loads each published page in Production on a verified domain and compares its <link rel="alternate" hreflang> and <link rel="canonical"> tags with what it sent. Pages that leave them out, or change them, show up under Languages in SEO › Health. The check identifies itself as CMS Orbit SEO check in its user agent, and reads only the page's <head>.

Combine it with your own metadata

Next.js merges a page's metadata into the layout's, one top-level key at a time. A few things to know:

• Title templates. The CMS title already has the template from SEO › Settings applied, such as "Hiking boots | Acme Outdoors". Do not also set title.template in your layout, or the site name appears twice. If you must, return title: { absolute: page.seo.title }.
• metadataBase. Set it in the root layout to your website's address, so relative image addresses in your own metadata become full ones. The CMS's addresses are already full.
• Open Graph is replaced, not merged. If the layout sets openGraph and the page returns one too, the page's wins entirely. Add your own fields to the page's: { ...meta, openGraph: { ...meta.openGraph, siteName: 'Acme Outdoors' } }.
• Not found and redirects. Return {} or a short title for anything that is not a page, as above.

1const meta = toMetadata(result.page)2return {3  ...meta,4  icons: { icon: '/icon.svg' },5  other: { 'theme-color': '#ffffff' },6}

Staging and preview websites

robots.txt on a non-indexable environment, such as Staging, refuses everything (see below). To be sure nothing on a staging website is indexed even when a page is reached directly, override robots there:

const indexable = process.env.SITE_INDEXABLE === 'true'return { ...meta, robots: indexable ? meta.robots : { index: false, follow: false } }

Set SITE_INDEXABLE=true only on the live website. Canonical links on staging still point at the live addresses, which is what you want: search engines that do find a staging page are told the live one is the real one.

Without the SDK, Pages Router websites and other frameworks can write the same tags from page.seo; Delivery API lists every field.

Structured data

import { JsonLd } from '@bifrost-orbit-cms/sdk/seo'<JsonLd page={result.page} />

JsonLd writes page.seo.jsonLd as one <script type="application/ld+json">: a single object when there is one, a list when there are several. It writes nothing when there are none. Characters that could end the script early are escaped, so editors' text cannot break out of it.

The CMS builds the structured data from the page and the space's settings, such as breadcrumbs, the organisation and the page type. Place JsonLd anywhere in the page; search engines read it wherever it is.

To add your own, write another script next to it:

<script type="application/ld+json" dangerouslySetInnerHTML={{ __html: JSON.stringify(mine).replace(/</g, '\\u003c') }} />

page.seo.breadcrumbs has the same trail as a list of { name, url }, for drawing visible breadcrumbs.

robots.txt

1import { createRobotsRoute } from '@bifrost-orbit-cms/sdk/next'2import { cms } from '@/lib/cms'34export const GET = createRobotsRoute(cms)

It serves the environment's robots.txt as plain text, cached for five minutes:

• Production gets the robots.txt editors set in SEO › Settings, with Sitemap: lines for each sitemap.
• Any environment that is not indexable, such as Staging, always gets User-agent: * and Disallow: /, whatever the settings say.

So the same code is safe on every environment, as long as each website uses a key from its own environment. cms.getRobots() answers { indexable, content } if you would rather build it yourself.

Remove app/robots.ts if you have one; Next.js does not allow both.

Sitemaps

The CMS lists every published page with an address, skipping pages that are no-index or left out of the sitemap by editors. Each entry has the page's full address, its last change, change frequency and priority if set, its images and its other languages. Addresses come from Configuration › Domains, so set those up first.

A sitemap file holds up to 50,000 addresses. Bigger regions are split into several files.

One region

1import { createSitemapRoute } from '@bifrost-orbit-cms/sdk/next'2import { cms } from '@/lib/cms'34export const GET = createSitemapRoute(cms)

It reads the client's defaultRegion in its default language and serves the XML with image and hreflang entries, cached for five minutes. createSitemapRoute(cms, { locale: 'en-DE' }) serves another language.

For a region with more than 50,000 pages, serve the extra files too. createSitemapRoute reads the file number from a name like sitemap-2.xml, in the route's file parameter or the address. Send those names to a route with a rewrite, because a dynamic segment at the top level would clash with the page route:

1const nextConfig = {2  async rewrites() {3    return [{ source: '/sitemap-:n(\\d+).xml', destination: '/api/sitemap/sitemap-:n.xml' }]4  },5}6export default nextConfig

1import { createSitemapRoute } from '@bifrost-orbit-cms/sdk/next'2import { cms } from '@/lib/cms'34export const GET = createSitemapRoute(cms)

cms.getSitemap({ page: 1 }) answers totalPages, so you know how many files there are.

Several regions or languages

Each region and language has its own sitemap at its own address, such as /de/sitemap.xml, and one sitemap index lists them all.

1import { createSitemapIndexRoute } from '@bifrost-orbit-cms/sdk/next'2import { cms } from '@/lib/cms'34export const GET = createSitemapIndexRoute(cms)

The index lists sitemap.xml, sitemap-2.xml and so on under each region's address, such as https://www.example.com/de/sitemap.xml. A region and language with no domain is left out.

Then serve each region's sitemap. With regions mapped from the address prefix as in Regions and languages:

1import { sitemapXml } from '@bifrost-orbit-cms/sdk/seo'2import { cms } from '@/lib/cms'3import { regionFor } from '@/lib/regions'45export async function GET(_request: Request, { params }: { params: Promise<{ region: string }> }) {6  const region = regionFor((await params).region)7  if (!region) return new Response('Not found', { status: 404 })8  const sitemap = await cms.getSitemap({ region: region.key })9  return new Response(sitemapXml(sitemap), {10    headers: { 'Content-Type': 'application/xml; charset=utf-8', 'Cache-Control': 'public, max-age=300' },11  })12}

sitemap.xml is a fixed folder name, so it takes priority over [[...slug]] under [region]. If your address prefixes are the regions' keys or short codes, export const GET = createSitemapRoute(cms) does the same in one line: it passes the region parameter straight to the API.

For each language of a region, pass locale to getSitemap, at an address of your choice, and make sure Configuration › Domains builds the same address for that language.

With Next.js's own sitemap file

app/sitemap.ts is the other way, and lets you change addresses, for example to your staging host:

1import type { MetadataRoute } from 'next'2import { cms } from '@/lib/cms'34export default async function sitemap(): Promise<MetadataRoute.Sitemap> {5  const { entries } = await cms.getSitemap()6  return entries.map((entry) => ({7    url: entry.loc,8    ...(entry.lastmod ? { lastModified: entry.lastmod } : {}),9    ...(entry.priority !== null ? { priority: entry.priority } : {}),10    alternates: { languages: Object.fromEntries(entry.alternates.map((alt) => [alt.hreflang, alt.href])) },11  }))12}

Use either this or app/sitemap.xml/route.ts, not both.

Keeping sitemaps fresh

Sitemap answers are tagged cms:sitemap, which the webhook refreshes whenever a page is published, unpublished or deleted and when a release runs. The five-minute Cache-Control lets search engines and CDNs keep a copy briefly.

Tell search engines where the sitemap is once, in Google Search Console and Bing Webmaster Tools. The Sitemap: lines in robots.txt tell everyone else.

What editors control

Most SEO needs no code. Editors set, in the CMS:

• each page's SEO title, description, share image, canonical address and no-index and no-follow switches;
• whether a page is left out of the sitemap;
• redirects, which resolve follows for you;
• the title template, default description and share image, social profiles, robots.txt and verification codes in SEO › Settings.

SEO › Health shows pages with problems, such as missing descriptions. See SEO health and SEO metadata.