Why Aren't My 301 Redirects Canonicalizing?


Staff member
  • #1
This is a weird little SEO related issue that I've had for a few months now. For years and years, I thought that 301 redirects were directives, meaning, not only do search engines have to follow them, but they also have to merge, or canonicalize, the redirected page with the one being redirected to. I thought this was a directive, not a suggestion. I never thought for a minute that Google and the other search engines would indeed follow the link, but them think about whether or not they actually want to merge it with the target URL. Strange.

I'll give you a few examples of what I'm referring to here.

I have a classifieds website that ran on the same software for about 14 years (Geodesic Solutions). One day, the developer decided to stop updating the software, so I figured I better move on to another vendor. I found the software you're currently using (Xenforo) and set up a new site. Then, I used 301 redirects to redirect the homepage and many of the category pages from old software to the new. In the beginning, I thought everything was going fine. When I clicked on one of the old links, I'd get redirected to the new. And after I checked the server log files, I found that Google and the others were crawling these redirected links as well. In my mind, I thought that once the search engines crawled the links, they'd flip them right over to the new ones and that would be the last I heard of it. Well, apparently it's not that easy.

It's been almost a year and many URLs that should have forwarded haven't yet. Granted, there were tons of them, but still, I thought the process would have occurred somewhat faster. I will tell you that the homepage URL switched over almost immediately as did most of the category URLs. Some category URLs haven't yet though, which made me curious.

I did some search around the internet and discovered, I believe, from John Mueller that redirected URLs actually aren't directives. They're treated more like canonical link elements in that they're suggestions. While anyone, whether it be a human being or a crawler, is forced to physically redirect from one URL to another when landing on a 301 redirect, Google's index treats things somewhat differently. From what I learned, Google takes how the original URLs are linked to into account. Basically, if the original URL still has links pointing to it while the new URL has none, Google will still show the old URL. It may canonicalize it with the new, but the old will remain visibly. I believe that as the links to the old URL fade out into the depths of history and as the new URL gains links, the new will be the one that's shown in the search results. This is all somewhat confusing, but I think everything eventually gets sorted out in time. Well, when you're doing a site migrations, anyway.

This next instance is somewhat the same as what I just shared, but different. In Xenforo software, there are many permanent 301 redirects. This is just the way forums are built. They've got tons of features and there's no way around setting the URL structure up the way it is. Basically, there are three different types of these redirects. The first redirects quoted replies to a hashtag portion of the same thread. The next redirects to the latest post in a thread, again, with a hashtag. And finally, similarly to the last, many URLs are redirected to various locations in a thread. All of these links use 301 redirects to forward one type of URL to a thread URL with a hashtag at the end of it.

When I first installed this software and noticed this, I didn't think anything of it. I figured that these forwarded URLs would silently canonicalize with the new and that would be that. Well, again, apparently Google doesn't think the same way. I'll just tell you now that out of 9,000 threads on one of my sites, I've got about 700 of these "redirected" URLs that haven't taken yet. They actually take precedence over the canonical thread URL itself. The thread URL is nowhere to be seen. I find this odd. To test things to see if it was only my site having this issue, I checked a few of the largest XenForo forums on the internet, including the XenForo community itself. Lo and behold, each site I checked has got the same issue. This confirmed what Google stated - that links that come from strong pages will take precedence over other links that come from pages that aren't as strong.

I believe I've found a solution for this second type of occurrence. If you'd like to read about it, please do here. Also, if you've got anything you'd like to add about this topic, please do so below. I love to learn about this sort of thing.