Home | sitemaps

Archives For sitemaps

Today I was listening to the webinar given a few months ago by Rand Fishkin on “Getting Value from XML Sitemaps, HTML Sitemaps + Feeds”. It gives some great information (if you’re a Pro member, you can find it here), though I heard some information that made me ask a couple questions and investigate.

I actually asked myself, “Wait, so do I have to submit different sitemaps for different search engines?”

Background

A few months ago, they had Duane Forrester from Bing on WhiteBoard Friday who said that they do not want anything in our sitemaps except for the clean, end-of-the-trail URLs. No redirects, no 404s. From this information, he said, they have the ability to learn who is and who is not submitting trustworthy and clean sitemaps, so only submit clean URLs.

In the webinar, Rand and his co-host (I think it was Jen Lopez) said that they had previously recommended that you upload the full sitemap, complete with 301s, so that the engine knows exactly what you are doing. They were wondering on the webinar if Bing was perhaps different.

Bing and Google May Have The Same Recommendations

Today I was rebuilding the sitemap for the site I work on as an in-house SEO. (Full disclosure: I hate building sitemaps, but it must be done). We recently switched the site over to HTTPS, but I figured “Hey, no rush on a new sitemap, because Google is fine with 301 redirects.”

Wrong.

I saw this today when I went into Google Webmaster Tools:

"We recommend that your Sitemap contain URLs that point to the final destination (the redirect target) instead of redirecting to another URL." (CLICK TO ENLARGE)

And this is the error in full (URL removed for privacy reasons):

301 GWT Error

"HTTP Error: 301"

What To Do?

Anytime you do a site upgrade or migration, you need to submit a new, up-to-date sitemap. If you have taken the time to plan out your migration (and you should), build this into your schedule. In my case, I have to go back through and rebuild the sitemap with HTTPS, which fortunately is not too hard (find and replace HTTP with HTTPS).

Why Should We Care About Clean Sitemaps?

It seems that Google is also wanting clean URLs in sitemaps, because it helps with indexation. I do not know if they have the same capabilities as Bing, or if sites could be penalized for bad sitemaps, but this is information we should take into account. Google is giving us a hint, I think.

I do also think that we should strive to submit clean information to the search engines, as it does make their job easier. And what’s the worst that could happen? Sitemaps are used in indexation, so we’re just going to have a bunch of well-optimized, 200-status returning URLs in sitemaps all over the Internet that help the search engines make sense of our websites.

That’s a good thing.

Some More Sitemap Resources

Dr. Pete on SEOmoz about Xenu and Screaming Frog
Information from SEOmoz on Google Sitemap Creator
Google Sitemap Generators (VERY ADVANCED)

Sitemap (n): A file placed on your web server, in XML format, to alert search engines to a website’s existing pages. Submitted manually to search engines using Webmaster Tools.

I just made up that definition, but it is also deceptive. I’ve been in web development for about 5 years, and I still hate sitemaps. At least, I did. I do sometimes.

Sitemaps can be incredibly helpful for websites, especially new websites, when trying to be discovered and ranked by search engines. They can be easy or terribly difficult to create, depending on the size, scope, and format of your website. I’ll cover this in a later post.

Basic Sitemaps for Beginners

Sitemaps for indexing, and therefore SEO, purposes are created in XML (eXtensible Markup Language) format. To learn more about the format for XML sitemaps, read this article on Sitemaps.org, the first result when you search “xml sitemap format”, and a surprisingly good resource for beginning.

A sitemap can, and possibly should, be generated every time you update your website by adding a new content page or blog post. Sitemaps should be placed in the root folder of your domain (so if your website is http://littlebluewidgets.com, your sitemap path should read http://littlebluewidgets.com/sitemap.xml).

Now, this is not an error-proof way of telling the search engines that you have a sitemap or that they should look for it. After all, if they have not yet found your site, a sitemap simply being placed in your root folder is not going to do you any good. You should use the search engine webmaster tools.

Webmaster Tools

Both Google and Bing have been nice enough to provide webmasters (and SEOs) with Webmaster Tools. Google’s is found here and Bing’s is found here. You must register your website with each tool and then place the given code into your site’s header (or use Yoast SEO for WordPress to do it automatically on your WordPress blog). Wait a few minutes (I usually wait a couple hours) and then verify your site.

Once inside, find the Sitemaps area where you can submit the path of your sitemap on your website.

Google

To find the sitemaps area in Google, simply log into your Webmaster account and click your website. Once the Dashboard appears, click “Submit a Sitemap”, which is found in the lower right-hand corner of the page.

Click "Submit A Sitemap" to move to the Sitemaps page

When you move to the Sitemaps window, click the “Submit A Sitemap” button and the below window will appear:

Google Sitemaps Window

Click "Submit A Sitemap" and enter your sitemap path

If you have entered the correct path, you should see a green check mark under the Status column.

Now on to Bing!

Bing

Submitting a sitemap to Bing is a bit trickier. Once you sign into Bing Webmaster Tools and select your website, you will land on the Dashboard page. Click the “Crawl” section, which is just to the right of “Dashboard” on the top navigation.

Bing Dashboard Screenshot

Click "Crawl" to move to the next step.

Once in the Crawl section, click “Sitemaps” on the left side of the screen:

Bing Crawl Webmaster Tools

Click "Sitemaps" to progress to the next step

Then click “Add Sitemap” and the following screen will appear:

Sitemaps for Beginners

Enter your sitemap path into this box

As long as your sitemap is in its proper place and you designate the correct path, you should see “Success” under the “Status” column.

Conclusion

I have taken you through the steps to submit your sitemap to the main search engines, Google and Bing. I highly recommend that you use a website CMS or blogging platform that allows you to automatically generate and submit your sitemap to the search engines.

I personally use Yoast SEO for WordPress, which I use to update the sitemap (I’ll be doing so directly after this post goes live) and ping Google and Bing to alert them that I have updated my website. This increases the likelihood of your new content being indexed.

Questions? Comments? Leave them in the Comments section!