What’s a Sitemap? How They Work + Steps to Build One

A sitemap is a list of all the pages on a website to help search engines find and index them. Learn how to properly build one so your website can rank higher on the SERP — fast.

Whether you’re building a website from scratch or revamping an existing one to boost your SEO, you need a sitemap.

By providing a sitemap, you make the search engine crawler bots’ job simpler, which helps them index your website quickly.

Without one? It’s easy for crawlers to miss important pages and slow your SEO efforts down as a result.

Here, HawkSEM lead strategist and SEO expert Yara Askar explains how to build a sitemap, which mistakes to avoid, and offers advanced tips.

XML is a document formatting language that’s easily understood by both humans and machines. (Image: Unsplash)

What is a sitemap?

A sitemap is a structured list of all the pages on your website. Search engine crawlers use this information to find content, understand the layout of your website, and determine the relationship between its pages and files.

Without a sitemap, a crawler would have to use internal links to find other pages on your website.

Unless all of your website’s pages are perfectly interlinked, a search engine is likely to overlook some important pages, which hinders your SEO efforts.

Types of sitemaps

There are five common types of sitemaps:

1. Normal XML sitemaps – XML (eXtensible Markup Language), is the most common type of sitemap that helps search engines index your web pages.
2. XML video sitemap – helps crawlers track the video content on your pages
3. XML image sitemap – allows crawlers to find all images on your website
4. Google News sitemap – helps crawlers find content on websites that can be used for Google News
5. HTML sitemaps – created for users (not crawlers) to simplify navigation and/or replace the search feature

Here’s a basic XML sitemap that includes the location of a single URL. (Image: Google)

How sitemaps work

Think of sitemaps as a roadmap for search engines. They provide search engine bots with an organized list of URLs available on a website with additional information attached, including:

• Update date – shows the date when the webpage was last modified
• Update frequency – the more frequently the sitemap is updated, the more often it needs to be crawled
• The page’s priority – gives crawlers an understanding of which pages are the most important to your website

This information helps web crawlers find and index those pages faster. Here’s how the functionality is broken down:

1. Crawling: Using the URLs listed in a sitemap as their guide, search engine bots crawl a website, enabling them to find pages that might otherwise be overlooked.
2. Indexing: After crawling, search engines use the information in the sitemap to determine when and how to index the page. This information can include when a page was last updated or how frequently it’s likely to change.

Pro tip: Sitemaps can’t contain more than 50,000 URLs or be more than 50MB in size. If your sitemap is bigger than allowed, you probably need to create more than one.

Why your website needs a sitemap

“The biggest benefit is assisting search engines in easily finding every important page of a website and ensuring it gets crawled,” says Askar.

Search engines only crawl a certain number of pages when they visit your website. So if you don’t create a sitemap with the right page priority tags, the crawler may overlook important pages during its visit.

The higher the page’s update frequency and priority, the more frequently the page is crawled.

Other key benefits include:

• Properly structured XML sitemaps help search engines crawl your pages more efficiently than they would without a sitemap.
• Sitemaps allow you to set priorities for the URLs. This helps you to direct crawlers to high-priority pages.
• You can change update dates and update frequency details to bring crawlers back to your website when necessary.
• If you’re creating a new website, a sitemap can help crawlers discover it more quickly.
• XML sitemaps help you avoid duplication issues. If another website copies your content, you can use the sitemap’s “last modified” information to show who the original content creator is.
• Sitemaps automatically notify search engines whenever you update your pages, so they come and crawl them faster.
• A sitemap report can help you discover errors in your website structure.
• An XML sitemap improves overall SEO efforts, while an HTML sitemap can improve the user experience.

Although all websites should have a sitemap, they are especially important for websites that:

• Are brand new
• Have hundreds or thousands of pages (such as an ecommerce site)
• Have a deep website architecture
• Add new pages frequently
• Update existing content frequently
• Have weak internal linking
• Have a weak external link profile

Case study: How sitemap optimization improved performance

DILO, a full-service emission-free gas-handling product and service provider, experienced a decrease in organic traffic after a site redesign. To address this, they partnered with Askar to enhance their brand’s industry leadership and increase organic exposure.

In addition to optimizing their technical SEO, content, and conversion rate optimization (CRO), Askar created an international sitemap and a robots.txt file to host the sitemap.

The results?

• 36% increase in organic clicks
• 24% increase in Google clicks for high-value keywords
• 46% increase in Google impressions

Read the full case study here.

Once you create a sitemap, you have to submit it to the search engine. (Image: Unsplash)

How to create a sitemap

Creating a sitemap doesn’t have to be complicated. Here are 8 quick steps to create and submit yours:

1. Choose the sitemap type

XML (for SEO) or HTML (for users). At this stage, it’s also important to make sure you noindex or block irrelevant pages via robots.txt or noindex tags.

2. Gather URLs

Collect all important URLs from your site. The most efficient way to do this is with a website crawling tool like Screaming Frog SEO Spider or XML-Sitemaps.com.

You can also accomplish this with Google Search Console (if your site is already indexed), a CMS plugin like Yoast SEO, or manually if your site is small enough.

3. Use a generator or manually write the XML file

Again, take advantage of one of the available tools provided by your site’s content management system. Alternatively, you can leverage an XML sitemap generator like Screaming Frog or XML-Sitemaps.

4. Use online validators to check for errors

Google Search Console, XML Sitemaps Generator, or SEOptimer can check your sitemaps for errors.

5. Upload the sitemap to your site’s root directory

You can do this with a site transfer protocol (FTP) or cPanel; however, if you use a CMS platform like WordPress, SEO plugins like Yoast SEO automatically generate and host the sitemap for you.

For the Yoast SEO plugin, the sitemap is usually located at https://yourdomain.com/sitemap_index.xml

6. Submit to search engines

Use Google Search Console and Bing Webmaster Tools to submit your sitemap.

To submit the sitemap to Google:

• Go to Google Search Console
• Choose “sitemaps”
• Paste the sitemap’s URL under “Add a new sitemap”
• Click “submit”

To submit the sitemap to Bing:

• Go to Bing Webmaster Tools
• Go to Sitemaps in the left-hand menu
• Paste the sitemap URL
• Monitor your sitemap status and make sure it’s being crawled correctly

7. Add the sitemap to your robot.txt file

The file is located in the root directory of your web server. To add the sitemap, you need to open the file and add a line that looks like this: “Sitemap: <sitemap URL>.”

8. Make regular updates

Keep the sitemap updated as you add or modify content.

Best practices for beginners

There are three big best practices, according to Askar:

1. Only include indexed pages
2. Add <lastmod> tag
3. Make sure to submit the sitemap file to Google Search Console so Google knows about it

“I’ve seen a lot of clients that have a sitemap file but never submitted the sitemap URL to GSC,” she explains.

Pro tips for sitemap optimization

Askar offers some additional advanced insights for a properly optimized sitemap:

1. Keep your URL structure consistent: If the domain includes www make sure that URLs in the sitemap include www.

2. Only include indexable pages: Avoid adding pages that have a noindex tag.

3. Do not include URLs with parameters: These can lead to duplicate content issues, making it harder for search engines to determine which version of the page to index.

4. Don’t forget to include the lastmod tag: This helps search engines like Google, Bing, or Yahoo understand how frequently to crawl and re-index pages.

5. For large websites with thousands of pages, use sitemap index files to organize all the URLs: This would be a list of URLs to individual sitemap files, and each list includes a list of pages on the website. For example, website.com/post-sitemap.xml (blog sitemap) vs. website.com/page-sitemap.com (pillar pages sitemap).

6. Update your sitemaps whenever you make changes to pages, whether that’s adding new content or removing outdated content.

7. If you use a “noindex” tag, make sure that page doesn’t end up on your sitemap: Sitemaps make it clear which pages you want crawled. Robots.txt files, on the other hand, clarify which pages you don’t want to appear on your site map.
8. Only include important pages in your sitemap to avoid maxing out your crawl budget.
9. Include key metadata, such as:

• <lastmod> (last modification date)
• <changefreq> (frequency of updates)
• <priority> (relative importance)

This allows sitemaps to provide extra context to search engines, which ensures the most important website content is indexed in a timely manner.

Here’s an example of what this would look like in an XML file:

<url>

<loc>https://www.example.com/page1</loc>

<lastmod>2025-03-25</lastmod>

<changefreq>monthly</changefreq>

<priority>0.8</priority>

</url>

Top tools to create a sitemap

“Yoast is a great WordPress SEO plugin that generates sitemaps,” says Askar. Other common tools include:

• Google Search Console Tools
• Bing Webmaster Tools
• Subscription-based tools like Screaming Frog

Do sitemaps impact SEO?

Yes, sitemaps impact SEO. That said, “they’re more critical for larger sites that have a complex website structure, ensuring that all important pages are found,” says Askar.

“Sitemaps are valuable but not the most crucial SEO strategy, in my opinion.”

She adds that search engines can find and index pages on their own without a sitemap, but it could take longer for pages to be discovered. Plus, if there’s no internal linking in place, some pages might not get discovered at all.

In other words? A sitemap is important, but a holistic SEO strategy is the priority.

Can sitemaps help with indexation speed for specific pages?

Sitemaps help search engines find all site pages more efficiently, rather than waiting for bots to find the pages on their own. The faster you can get a page indexed, the faster it’ll start to rank.

Are there any cases where you would have XML and HTML sitemaps?

“HTML sitemaps are helpful for extremely large ecommerce sites,” says Askar. “Oftentimes, you’ll find them published in the footer of a website, and their only use is to help users navigate the website more easily.”

In other words? XML sitemaps are important for search engines, but a visitor doesn’t need the XML sitemap to navigate a website.

The takeaway

A website sitemap is a useful tool that helps search engines and users navigate your website. While a submitted sitemap doesn’t automatically improve your SEO efforts, it can increase your website’s visibility for crawlers.

Using sitemaps as part of a larger digital marketing strategy will help you rank higher in the search engine results pages (SERPs) and see a bigger return on your investment.

Want to learn more about sitemaps and other aspects of technical SEO? We’re here to help.

This article has been updated and was originally published in July 2021.