With the popularity of React Framework / Single Page Application(SPA) there’s a problem arises, SEO. Search Engine Optimization (SEO) is an essential aspect of web development that can significantly impact the visibility and success of a website. While traditional single-page applications (SPAs) built with React often face challenges in SEO.

Why did this problem arise, I hear you ask well:

Initially, React applications were built around the concept of Client-Side Rendering (CSR):

• Rendering Process: The browser downloads the minimal HTML from the server, and then JavaScript takes over to render the rest of the UI dynamically.
• Performance: While CSR provided rich interactive experiences, it had performance drawbacks, especially for initial page loads and SEO, as the content was not immediately available to search engines or users without JavaScript.

NextJS to the rescue

Next.js emerged as a solution to improve the performance and SEO of React applications by enabling Server-Side Rendering:

• SSR Benefits: With SSR, the server sends a fully rendered page to the client, which means the content is immediately visible to users and search engines.
• Next.js: Next.js made SSR easier by providing a framework around React to handle server rendering, routing, and optimization out of the box.

We now understand that Next.js is the solution, but how do we practically make our app SEO-friendly? For this we need to first understand how can crawlers detect our site?

What are crawlers and what do they need?

Web crawlers, also known as bots or spiders, are automated programs used by search engines to browse the internet and index web pages. They are responsible for discovering new and updated web pages and collecting information about them to be added to the search engine's database.

Web crawlers work by starting at a list of known URLs and then following links on those pages to discover more URLs. They read the content of each page, record information about it, and store this information in a database. This process continues until all linked pages have been visited.

However, not all pages are equally important or relevant to the user. Some pages may be duplicates of others, some may be under development, and some may be full of irrelevant content. This is where sitemaps become useful.

Sitemaps

A sitemap is an XML file that lists all the pages on a website, allowing search engines to crawl and index them more efficiently. It serves as a roadmap to the site's content, helping search engines discover and understand the structure of the website. Without a sitemap, search engines might miss pages, especially those deep within the site hierarchy, resulting in lower visibility and ranking in search results.

Let’s see how we can leverage NextJS to create sitemaps

Creating a Dynamic Sitemap:

There are two ways to create a sitemap depending on the router, which is page router and app router.

**Next.js (Page Router):**

Before the introduction of the file-based metadata API in Next.js 13.3, one common approach was to use getServerSideProps to dynamically generate the sitemap on each request. Here's a step-by-step guide to creating a sitemap using getServerSideProps in a Next.js application before version 13.3:

**Step 1: Create a New Page for the Sitemap**

Developers would create a new page specifically for the sitemap, typically named sitemap.xml.js, inside the pages directory. This page's purpose was to generate XML content and serve it as the response to requests to /sitemap.xml.

**Step 2: Fetch Data for Dynamic URLs**

The getServerSideProps function would be used to fetch data from an API or other data sources. This data would contain the information needed to construct the URLs for the sitemap, including dynamic page URLs.

**Step 3: Generate XML Sitemap Content**

A function called generateSiteMap would be written to take the fetched data and convert it into a sitemap XML string. This function would iterate over the data to create URL entries for each page, including static and dynamic pages.

**Step 4: Set Response Headers and Send the Sitemap**

Inside the getServerSideProps function, after generating the sitemap, the content type header would be set to 'text/xml', and the XML sitemap string would be sent as the response using res.write(sitemap) followed by res.end().

Here's an example of what that code might look like:

// pages/sitemap.xml.js
const EXTERNAL_DATA_URL = "https://jsonplaceholder.typicode.com/posts";

function generateSiteMap(posts) {
  return `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      <url>
        <loc>https://yourdomain.com</loc>
      </url>
      <url>
        <loc>https://yourdomain.com/guide</loc>
      </url>
      ${posts
        .map(({ id }) => {
          return `
          <url>
            <loc>${EXTERNAL_DATA_URL}/${id}</loc>
          </url>
        `;
        })
        .join("")}
    </urlset>
  `;
}

function SiteMap() {
  // getServerSideProps will do the heavy lifting
}

export async function getServerSideProps({ res }) {
  const request = await fetch(EXTERNAL_DATA_URL);
  const posts = await request.json();
  const sitemap = generateSiteMap(posts);

  res.setHeader("Content-Type", "text/xml");
  res.write(sitemap);
  res.end();

  return { props: {} };
}

export default SiteMap;

---

**Next.js 13.3 (App Router)**

With the release of Next.js 13.3, creating a sitemap has become more straightforward. You can now simply create a sitemap.js file in the app directory and export a default async function that returns an array of URL objects. This function should gather all the necessary URLs from your static and dynamic content and return them in the required format:

// app/sitemap.js

import { getSortedPostsData } from "../lib/posts";

const URL = "https://yourdomain.com";

export default async function sitemap() {
  const posts = getSortedPostsData().map(({ id, date }) => ({
    url: `${URL}/blog/${id}`,
    lastModified: date,
  }));

  const staticRoutes = ["", "/about", "/contact"].map((route) => ({
    url: `${URL}${route}`,
    lastModified: new Date().toISOString(),
  }));

  return [...staticRoutes, ...posts];
}

That's all you need to do. After deploying your application, you can access the sitemap at this URL: https://yourdomain/sitemap.xml. It will display a list of URLs useful for web crawlers.

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>https://ripeseed.io/</loc>
<lastmod>2021-10-15T06:27:48+00:00</lastmod>
<priority>1.00</priority>
</url>
<url>
<loc>https://ripeseed.io/our-work</loc>
<lastmod>2024-01-16T12:38:57.117Z</lastmod>
<priority>0.97</priority>
</url>
<url>
<loc>https://ripeseed.io/contact-us</loc>
<lastmod>2024-01-16T12:38:57.117Z</lastmod>
<priority>0.95</priority>
</url>
..........
..........
..........
<loc>https://ripeseed.io/blog/improving-development-flow-in-a-multi-project-repo</loc>
<priority>0.24</priority>
</url>
</urlset>

Next.js has transformed the process of building SEO-optimized web applications with React. Traditional Single Page Applications (SPAs) struggled with SEO due to client-side rendering, which delayed content presentation to users and search engines. Next.js solves this issue with server-side rendering (SSR) and static site generation (SSG), ensuring immediate content visibility.

With Next.js, dynamic sitemap generation is simplified, promoting more efficient search engine indexing. In essence, Next.js enables the creation of high-performing, interactive web applications that are also SEO-friendly. This aligns with modern web development practices that focus on both visibility and functionality.