What Is an XML Sitemap? A Guide to Submitting Yours to Search Engines

What Is an XML Sitemap?

An XML sitemap is a file written in XML format that reports the addresses of the pages on a website to search engines. Its purpose is simple: to tell search engine bots directly which pages exist on your site and, where possible, when they were last updated. This makes it easier for bots to discover pages, especially on sites with a weak internal link structure, on newly launched sites, and on sites with a very large number of pages.

A sitemap does not raise a page's ranking; it only helps that page be discovered and crawled. How the page performs in search results is a matter of content quality, links and technical SEO as a whole.

Which Sites Benefit Most From a Sitemap?

E-commerce sites with a weak internal link structure and a great many product or category pages, single-page applications (SPAs) that render their content with JavaScript, and new sites that have yet to attract many external links are the site types that benefit most from a sitemap. On sites like these, bots can struggle to discover pages through internal links alone; a sitemap makes that discovery direct and reliable. On a small corporate site of a few pages with a strong internal link structure, the effect of a sitemap is more limited.

The Difference Between a Sitemap and robots.txt

robots.txt and sitemap.xml are usually mentioned together, but their functions are opposites. robots.txt tells bots where not to go; sitemap.xml suggests to bots where they should look. The two must not contradict each other: if a URL you have added to the sitemap is Disallowed in robots.txt, the bot will not crawl that address even though it sees it in the sitemap.

The Sitemap Protocol and File Structure

Sitemap files are written according to Sitemap Protocol 0.9 — a standard format using the http://www.sitemaps.org/schemas/sitemap/0.9 XML schema, commonly accepted by Google, Bing and the other major search engines. The file consists of a <url> block for each page inside a <urlset> root element:

• <loc>: The absolute (full) URL of the page. The only required field.
• <lastmod>: The date the page was last updated (YYYY-MM-DD). Entered accurately and kept current, it helps crawlers prioritise re-crawling pages that have changed.
• <changefreq>: The estimated change frequency of the page (always, hourly, daily, weekly, monthly, yearly, never).
• <priority>: A suggested value between 0.0 and 1.0 expressing the relative priority of the page among the site's own pages.

Do priority and changefreq Actually Do Anything?

Google has stated publicly that the priority and changefreq values are largely disregarded in the modern indexing process; these fields are a suggestion, not a commitment. An accurately entered lastmod value, by contrast, genuinely helps. When preparing a sitemap, therefore, it is smarter to spend most of your energy keeping lastmod accurate and to leave the other two fields at sensible defaults.

Even so, setting changefreq close to the page's real update frequency rather than picking it at random keeps the file meaningful and makes it easier, when you later review the sitemap, to remember which group of pages changes how often:

Sitemap Size Limits and the Sitemap Index

A single sitemap file can contain at most 50,000 URLs and 50 MB (uncompressed). On large sites that exceed this limit, a sitemap index file (<sitemapindex>) pointing at several sitemap files is used. For small and medium-sized sites a single sitemap.xml is usually enough.

The structure of a sitemap index file is very close to that of an individual sitemap file; it simply holds <sitemap> entries instead of <url> ones:

• <sitemapindex>: The root element wrapping all the sitemap references.
• <sitemap>: A separate block for each child sitemap file.
• <loc>: The full address of the child sitemap file in question.

Publishing the Sitemap File and Submitting It to Search Console

Once you have created the file, upload it to the root directory of your site, for example https://example.com/sitemap.xml. Then complete two steps:

• Add the line Sitemap: https://example.com/sitemap.xml to your robots.txt file; this makes it easier for bots to discover the sitemap automatically.
• Submit the sitemap directly through Google Search Console; this can speed up initial discovery and indexing.

Which Pages Should Go in the Sitemap?

Only canonical pages that are indexable and reachable with a 200 status code should be added to the sitemap. Redirecting URLs, links that return a 404, pages carrying a noindex tag, and duplicate URLs arising from parameter differences should not appear in the sitemap; they make the bot spend its time on unnecessary pages and reduce the reliability of the sitemap.

Common Sitemap Mistakes

• Including pages in the sitemap that redirect, return an error (404) or carry a noindex tag.
• Writing a relative path in the <loc> field; this field must always be an absolute URL (starting with http:// or https://).
• Not keeping the sitemap current — failing to regenerate the file when new pages are added and old ones removed.
• Trying to collect a site that exceeds the 50,000 URL limit in a single file.

Build One Quickly With an XML Sitemap Generator

Writing a sitemap out as XML by hand takes time, and a small syntax error can invalidate the entire file. KEYDAL's XML sitemap generator takes a plain list of URLs (one per line) and produces a valid sitemap.xml with changefreq, priority and lastmod options. The tool validates every line entered, flags invalid lines that do not start with http:// or https:// separately, and runs entirely in your browser.

Once the file is published, you can track over time how much of your site is being discovered by comparing the number of submitted URLs in the Sitemaps report in Search Console with the number of crawled and indexed pages.