Base44 sitemap.xml and robots.txt: what they contain and how to fix them

Base44 generates both files for you. robots.txt needs no attention on most apps. The sitemap needs reading, because it lists every page file in your app, including password-reset and admin screens, and none of the records your app shows. Fix it with per-page indexing toggles, a replacement file, or a sitemap generated from your data by a backend function.

The files are one part of the Base44 SEO checklist. Here’s what’s in each and how to change it.

What Base44 generates#

According to Base44’s SEO documentation:

• sitemap.xml is served at /sitemap.xml. “Public, indexable pages are included, while private or internal pages are excluded.”
• robots.txt is created “for both free and paid apps. It allows search engines and AI crawlers to access your public content while protecting non-public or technical routes from indexing.”
• llms.txt, if you switch it on from the SEO & GEO page, gets the same page list.

Look at your own:

curl -s https://yourdomain.com/robots.txt
curl -s https://yourdomain.com/sitemap.xml | grep -o '<loc>[^<]*</loc>' | sed 's/<[^>]*>//g'

What we found in 47 Base44 sitemaps#

In our test of 47 live Base44 sites, every one served a valid sitemap and robots.txt. The contents had the same two problems across the board.

Records never appear. The 47 sitemaps held 672 URLs between them, from 4 to 40 per site. Not one pointed at an individual record. A Base44 app shows records through a template page, like /BlogPost?id=abc or /series/<id>, and the sitemap lists the template once (or not at all) and none of its records.

Utility pages do. 143 of the 672 URLs (21%) were pages like /forgot-password, /reset-password, /login, /admin, /dashboard or /checkout, and 35 of the 47 sites had at least one. Base44 excludes pages that require login. A password-reset form doesn’t require login, so it goes in.

Neither problem stops Google from indexing your site, but both weaken the signal. A sitemap is where you tell Google which pages matter. One where a fifth of the entries are forms, and the pages you actually want found are missing, says the opposite.

Fix 1: turn off indexing for utility pages#

The quickest fix, and the one to do first.

1. Open your app’s Dashboard and go to the SEO & GEO page.
2. Open Advanced Settings.
3. Turn off the indexing toggle for login, sign-up, forgot-password, reset-password, admin, dashboard, account, checkout, success and cancel pages.

Base44 removes each page from sitemap.xml and llms.txt “immediately” and adds a noindex tag. Check the result:

curl -s https://yourdomain.com/sitemap.xml | grep -iE 'password|login|admin|dashboard|checkout'
# no output = clean

Fix 2: replace the sitemap with your own file#

Use this when you want full control of a mostly static list, such as a marketing site with a fixed set of pages.

1. On the SEO & GEO page’s Setup checklist tab, turn off Generate sitemap.xml.
2. Add your file at public/sitemap.xml in the Code tab (or ask the AI chat to create it).
3. Publish.

Base44 serves public/sitemap.xml at /sitemap.xml. With the toggle off and no file in place, /sitemap.xml returns a 404, so do both steps together. A static file has one weakness: it goes stale the moment you add a page and forget to update it.

A minimal file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url><loc>https://yourdomain.com/</loc><lastmod>2026-09-20</lastmod></url>
  <url><loc>https://yourdomain.com/pricing</loc><lastmod>2026-09-12</lastmod></url>
  <url><loc>https://yourdomain.com/about</loc></url>
</urlset>

Google ignores priority and changefreq, so leave them out. Only include lastmod if it’s accurate: a date that changes on every publish teaches Google to ignore it.

Fix 3: generate the sitemap from your records#

This is the fix that matters if your records are the pages people search for: products, listings, articles, profiles. Base44’s docs support it directly: “you can serve the sitemap from a backend function and submit its URL in Google Search Console. Google accepts sitemaps from any path on your verified property when you submit them through Search Console.”

Backend functions run on Deno and need the Builder plan or higher. A function that lists static pages plus every published post might look like this. The entity name, fields and URL pattern are placeholders for your own:

// base44/functions/sitemap/entry.ts
import { createClientFromRequest } from "npm:@base44/sdk";

const SITE = "https://yourdomain.com";
const STATIC_PAGES = ["/", "/pricing", "/about", "/blog"];

export default async function (req: Request): Promise<Response> {
  const base44 = createClientFromRequest(req);
  // Read with service-role access so the function works without a signed-in user.
  const posts = await base44.asServiceRole.entities.BlogPost.filter({ status: "published" });

  const urls = [
    ...STATIC_PAGES.map((p) => `<url><loc>${SITE}${p}</loc></url>`),
    ...posts.map(
      (post) =>
        `<url><loc>${SITE}/BlogPost?id=${post.id}</loc>` +
        `<lastmod>${new Date(post.updated_date).toISOString().slice(0, 10)}</lastmod></url>`
    ),
  ];

  const xml =
    `<?xml version="1.0" encoding="UTF-8"?>` +
    `<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls.join("")}</urlset>`;

  return new Response(xml, { headers: { "Content-Type": "application/xml" } });
}

The shape follows Base44’s backend functions reference and the SDK’s filter() method. Your entity and field names will differ, so the easiest route is to ask Base44’s AI chat to write the function for your entities and then review it. Two rules from Base44’s docs: the function must not require authentication, since crawlers can’t sign in, and each fetch of the sitemap counts as a function invocation.

Submit the function’s full URL in Search Console, for example https://yourdomain.com/functions/sitemap. You can keep the generated /sitemap.xml for your pages alongside it.

robots.txt: leave it, mostly#

Base44’s robots.txt allows public content and AI crawlers, and there’s no documented way to replace it from the dashboard. For most apps that’s what you want.

Two points worth knowing:

• Disallow doesn’t remove a page from Google. A disallowed URL can still be indexed from links, just without its content. To keep a page out of results, use the per-page indexing toggle, which adds noindex.
• Blocking AI crawlers is a business decision. If you don’t want your content used by AI assistants, you’d need rules for GPTBot, ClaudeBot and the rest. If you do want to appear in AI answers, the default is right. The AI search guide covers which crawlers do what.

What to do next#

1. List your sitemap URLs with the curl command above.
2. Turn off indexing for every utility page on the list.
3. If records matter, add a backend-function sitemap and submit it in Google Search Console.
4. Recheck the sitemap after each round of new pages.