What Is a Sitemap. Why Does It Matter?
A sitemap helps search engines locate and understand URLs on a website. A sitemap is especially useful for sites that add content often or have a layout.
Sitemaps can appear in formats such as XML sitemaps RSS feeds, HTML sitemaps and other structured formats.
<H2>What Is an XML Sitemap?</H2>
An XML sitemap is a file that lists URLs from a website in a format that search engines can easily read.
The main purpose of a sitemap is to help search engines find pages that might be hard to reach by crawling. This is especially useful for sites with navigation.
An XML sitemap can include:
- website URLs
- The date a page was changed
- Details about updated content
- details for types of content
A sitemap does not replace regular crawling. Search engines can still find pages by following links and other clues. Instead a sitemap tells search engines about URLs that a website considers important.
<H2>Does Every Website Need a Sitemap?</H2>
Not always.
A small website with a structure may be easy for search engines to crawl without a sitemap. However keeping a sitemap can still be helpful especially if the site is expected to grow.
Many modern content management systems and hosting platforms can create sitemaps automatically. If a sitemap is already being created correctly there is generally no reason to remove it.
As a site becomes bigger. Publishes more content having a sitemap can become more helpful.
Sitemaps for Updated Websites
Sitemaps can be especially helpful for sites that add or change content regularly.
News sites, stores, publishing platforms and large content sites may have thousands or even millions of URLs that change over time.
For example if a product page has information, an accurate lastmod value can tell search engines that the page has been updated.
News sites can also use special news sitemap features to help search engines find published content.
What About Priority and Change Frequency?
Older sitemap documentation often included fields such as:
- priority
- changefreq
These fields are not as useful as many site owners think.
The priority value was hard to use because site owners often marked every page as important. If everything is marked as important the signal becomes less useful.
Similarly changefreq is often too general to describe how a site changes. A site may update a lot without every page needing to be crawled at that rate.
For sites the URL itself and an accurate lastmod value are pieces of information.
Understanding the lastmod Value
The lastmod value tells search engines when the content of a URL was last changed significantly.
It should show a change to the page not the current date, whenever the sitemap is updated.
For example changing the content of an article could justify updating its lastmod date. Updating the sitemap without changing the article would not.
Accurate modification dates can help search engines know which pages have recently changed.
Canonical URLs in a Sitemap
A sitemap should generally include the URLs that a site considers to be the versions.
Sites sometimes have versions of a page because of tracking parameters, URLs, filters, dates or other URL variations.
For example a site might have:
- https://example.com/product
- https://example.com/product?source=newsletter
If the second URL is for tracking the main version would normally be the one included in the sitemap.
Keeping URLs in the sitemap can help show the site’s URL structure. It helps search engines understand which versions are meant to be indexed.
XML Sitemaps vs. RSS Feeds
XML sitemaps and RSS feeds can both help search engines find updated content. They serve purposes.
An RSS feed usually lists. Updated URLs. This makes it useful for showing content.
An XML sitemap can list URLs. It gives a view of a site.
For a site with changing content using both can make sense. The RSS feed can highlight changes while the XML sitemap can list URLs.
Sitemap Size Limits
Sitemap files have limits.
A single sitemap can have:
- up, to 50,000 URLs
- up to 50 MB of data
Sites often split URLs into multiple sitemap files. Sitemap files can also be compressed with gzip to make them smaller.
When many sitemap files are needed a sitemap index file can organize them.
<H2>What Is a Sitemap Index?</H2>
A sitemap index is simply a sitemap that lists sitemaps.
Than putting thousands or millions of URLs in one file a large site can create several sitemap files and put those files inside a sitemap index.
For example a website might have sitemaps for:
- Products
- Blog posts
- Categories
- News articles
The sitemap index can then direct search engines to each of these files.
The sitemap index itself can be submitted through search engine webmaster tools. The sitemap index can also be mentioned in robots.txt.
<H2>Where Should a Sitemap Be Located?</H2>
A sitemap does not have to use the filename sitemap.xml.
The website can use any filename if needed. However using a name like sitemap.xml can make the file easier for people and automated systems to find.
The sitemap can also be listed in the website’s robots.txt file:
Sitemap: https://example.com/sitemap.xml
If the sitemap uses a filename or location the sitemap can still be submitted directly via search engine webmaster tools.
Should You Hide Your Sitemap?
Some website owners think about using a hard-to-guess sitemap filename because they do not want the sitemap to be easily seen.
There is usually a benefit for search engine ranking to doing this.
The sitemap is meant to help search engines find URLs so making the sitemap fairly easy to locate can be useful. Listing the sitemap in robots.txt is one way to show the sitemap’s location to crawlers.
If the sitemap is not easy to find the sitemap can still be submitted directly via search engine tools. Other crawlers may not automatically find the sitemap.
Sitemaps and AI Crawlers
AI-related crawlers can also look at XML sitemaps. Rss feeds while exploring websites.
Making discovery files easy to find can therefore help different automated systems understand the structure and available content of the website.
However an XML sitemap is mainly a discovery tool for search engines and crawlers. The sitemap should not be treated as a way to control AI training or decide how AI systems use the website content.
What Is an HTML Sitemap?
<H2>An HTML sitemap is different from an XML sitemap.</H2>
An HTML sitemap is a web page that has links to areas or pages of the website. The HTML sitemap is mainly designed to help visitors find their way around the site.
Search engines can also follow the links on the HTML sitemap which can help with linking.
However the HTML sitemap does not replace the XML sitemap. The two have purposes.
HTML Sitemap
- Primarily designed for the website visitors
- Contains links
- Helps users navigate the website
- Can improve linking
- Uses HTML
- Is not a replacement for an XML sitemap
XML Sitemap
- designed for search engines
- Uses XML formatting
- Can have large numbers of URLs
- Can include modification dates
- Can support specialized sitemap formats
- Can be submitted through webmaster tools
