A Beginner's Guide to Sitemap Testing: Part 2

Sitemaps are crucial for helping search engines understand websites. When sitemaps are invalid, they make it difficult for search engines to crawl new pages efficiently. Regularly checking the sitemap is important to ensure that search engines can read it.

In our previous blog post, 'A Beginner's Guide to Sitemap Testing: Part 1', we gave a quick overview of different sitemap types like XML, mobile, video, and image sitemaps.

In part 2, lets dive and explore popular tools that can ensure that a sitemap is correctly formatted, error-free, and optimized for search engines. By the end of this post, you will be well equipped to test your sitemap like pro!

Test cases for sitemap testing

To ensure that the website's sitemap is functioning properly, consider the following test cases:

Validate the structure: Verify that the sitemap is formatted properly and follows the correct XML schema. Test that the sitemap starts with the <?xml version="1.0" encoding="UTF-8"?> tag and ends with the closing </urlset> tag.

Verify all URLs: Verify that all URLs in the sitemap are accessible and return a 200 HTTP status code. Test whether all URLs are valid and reachable.

Check for duplicate URLs: Test whether the sitemap does not contain any duplicate URLs.

Verify any broken links: Test whether all URLs in the sitemap are valid and do not return a 404 or other error status code.

Check for redirects: Test to ensure none of the URLs redirect to another URL.

Check for correct hierarchy: Verify that the URLs in the sitemap are organized in a logical structure with the main pages at the top and the sub-pages nested underneath.

Check the size: Test that the sitemap is not too big, as large sitemaps are burdensome for search engines to process.

Check the frequency: Test whether the frequency of page updates is set correctly.

Check the priority: Test whether the page priority is set correctly.

Verify on different devices and browsers: Test that the sitemap is fully responsive and compatible with multiple devices and browsers.

Test sitemap submission: Test submitting the sitemap to search engines like Google, Bing, and others using the webmaster tools to check if it is indexed and if there are any errors.

By creating and executing these test cases, it can ensure that the sitemap is accurate, up-to-date, and error-free, which will help search engines better understand and crawl the website.

Testing sitemaps using various tools

Google Search Console

Check the submitted sitemap in Google Search Console:

Navigate to the "Sitemaps" section under the "Index" tab in the sidebar.

View the status and details of the submitted sitemaps by navigating to the sitemaps report in the website's search console.

Check the "Submitted Sitemaps" section on this page for information about the sitemap.

Upon successful process of the sitemap index, it signifies that Google has successfully crawled and analyzed the sitemap, ensuring seamless navigation and understanding of the website content.

To check the Coverage Report for the sitemap, click on “SEE INDEX COVERAGE”

Using the coverage report check how many URLs Google found in the sitemap and how many of those pages ended up in the Google index.

The sitemap report displays the number of web pages and the status of each URL. It can include the number of valid URLs, any excluded URLs, and other relevant information.

Check the error of any “Excluded” pages.

6 URLs in the sitemap are getting a “Duplicate, submitted URL not selected as a canonical” message. These pages should not be indexed, so it should be removed from the sitemap.

XML Sitemap Validator

XML sitemap validator is a tool that checks if a sitemap follows the standard protocol and has all the required elements and attributes. It can also find errors, like invalid URLs or incorrect formatting, and suggest solutions.

The validator makes sure that search engines can crawl and index the website correctly and identify any issues that could affect the site's search engine rankings. There are several free XML sitemap validators available online, such as the W3C Markup Validation Service and XML Sitemap Validator.

Note: There are paid tools available that can handle all of a website's SEO functionality.

SEOptimer

SEOptimer is a website that offers various SEO tools, including a sitemap checker. The Sitemap Checker tool can be used to validate and test the sitemap. It checks for errors, such as broken links, and provides a report on the status of each URL in the sitemap.

Some features of SEOptimer are:

Validate the sitemap structure.

Check if all URLs are accessible.

Check for any duplicate URLs.

Check for any broken links.

Check for the correct format.

It's important to note that having a sitemap does not guarantee that all the pages on the site will be indexed, but it does help to ensure that the search engine knows all the pages on the site.

SEOptimer Sitemap Checker is a useful tool for identifying any issues and is a valuable resource for improving the website's SEO.

Real-time examples

Let's consider how to perform real-time sitemap testing using Screamingfrog, a popular SEO tool.

We will test a sitemap using Screaming Frog - European Bartender School.

Open Screamingfrog and click on the "Sitemaps" tab.

Click "Add" and enter the URL of the website.

Add the sitemap and click "Start" to begin the crawl.

Wait for the crawl to complete, and click on the "Sitemaps" tab again.

We see a list of all the pages in the sitemap with their HTTP status codes and other information.

Here are two screenshots of Screamingfrog's sitemap testing feature.

We will test another sitemap using Screamingfrog - Legal Aid DC. Below are some aspects that should be checked while testing using Screamingfrog.

Check for broken links: Look for URLs that return a 404 HTTP status code. These are broken links and should be removed or fixed.

Check for redirects: Look for any URLs that redirect to another page. Check whether the redirect is correct and the final URL is included in the sitemap.

Check for duplicate content: Look for URLs with the same content as another page on the website. It can hurt the search engine rankings, so fix any duplicates.

Check for missing pages: Look for any pages on the website not included in the sitemap and add them.

Here are some screenshots for the Legal Aid site using Screamingfrog.

Conclusion

XML Sitemaps play a crucial role in ensuring that search engines can easily discover and understand the content on a website. When sitemaps are invalid, any new primary pages are crawled less efficiently, which can negatively impact a website's search engine rankings. It's therefore important to regularly check the sitemap to ensure that search engines can read it.

Sitemaps also serve as a canonical signal for Google's crawling and indexing system. This is particularly useful for larger websites with duplicate content, where stronger canonical signals like 301 redirects cannot be implemented. By including the canonical version of each page in the sitemap, search engines can better understand which version of the page is the preferred one.

In addition to their technical benefits, sitemaps improve the overall user experience (UX) of a website. By including all the pages on a website in the sitemap, visitors can more easily navigate the site and find the information they need. This can lead to increased engagement, longer session times, and ultimately, better conversions. Therefore, maintaining a well-structured and regularly updated sitemap not only benefits search engines but also contributes to a positive user journey on the website.

‍