Web Development

Finally!

Yesterday I FINALLY figured out why Google Search Console (GSE) was showing so many pages with 302 redirects in our link budget. For over a year, I thought this issue was relate to either Apache2 or DBSEO (our PHP module which rewrites our links to be SEO friendly). However, when I breached this subject with the developer of DBSEO, he chastised me repeatedly and told me that it is not possible that DBSEO was causing these redirects, so loathed by Google. If fact, the DBSEO coder basically implied I (their paid and licensed customer) was an idiot for even thinking it might be DBSEO causing the problem. So, even with my "amazing" debugging skills, this problem remained unsolved for a long time.

So, yesterday, caught up on a few forum coding tasks, I decided to dive into this problem again. Frustrated after searching the entire Apache2 configuration and looking at the DBSEO PHP code base again for the culprit of this 302 redirect bug filling up our link budget with Google, I asked Akshay Hedge to help me troubleshoot, messaging over Whats App. Akshay , bringing an outsider view into this, followed the same steps that I would have (and had been doing). We searched the Apache2 configuration deeply looking for redirect directives. We found none. We search the core PHP files for the same redirect-related directives and keywords. We found nothing. Both Akshay and I, looking at the sample links in GSE, got the same 302 redirects which Google complains about.

Then, having exhausted all possibilities in the realm of this quantum reality we are in, I did what I should have done a year ago and found the core of the problem. I took the link in GSE causing the 302 and compared it to the link after the 302. This seems obvious now and was the most basic thing I should have done first, not last. However, having Akshay on Whats App confirming that there was no Apache2 or PHP error outside of DBSEO, helped "free my mind" from the fact that the DBSEO developers had insulted me repeatedly and when I pointed the finger at their DBSEO code, I found that DBSEO was the problem by simply comparing the links.

The forum software here (a legacy vB PHP system which has been seriously modified over the years and does not really resemble vB anymore) has a "convenience" URL parameter called "goto". This URL parameter is used to direct users to new posts and looks like this, for example:

Code:

https://www.unix.com/your-favorite-discussion.html?goto=newpost.

DBSEO rewrites this URL with URL parameter as:

Code:

https://www.unix.com/your-favorite-discussion-new-post.html

and when this link is indexed by Google, it (as it should) classifies this link with "new-post" embedded in the URL as a 302 redirect.

If I just remove this feature, users will complain that the URLs do not take them to new posts (and some will complain strongly), so I basically rewrote DBSEO to stop rewriting all possible "goto" URL parameters for unregistered (guest, bots, etc.) users.

Problem solved.

I don't think I would have seen this obvious DBSEO URL rewriting issue if it was not for Akshay working with me on Whats App. Akshay helped me confirm that DBSEO was the root cause, despite the strong objections and insulting innuendo when I breeched this issue with DBSEO in 2018 and again in 2019.

Thank you Akshay for spending a few hours with me on Whats App and helping me confirm the obvious now, that DBSEO was the root cause of this 302 issue. Of course, finding the root cause of a problem is the hard part, and I wrote some new code shortly after to fix this problem, forever.

This problem is solved, finally. I am very happy to have this nagging problem "figured out". I am not accustomed to having technical or coding issues been so difficult to debug, but DBSEO was adamant that their software was not the core problem, when if fact, it turns out that 100%, it was the core problem.

DBSEO, rewriting the "goto" URL parameter was the root cause, and I have disabled that (for unregistered users, bots, etc. ... not registered member logged in) with some custom PHP code. I cannot force Google to drop those "excluded 302 links" from their index (there are over 500,000 of them) which is hurting our link budget with Google (and I do not want to force a 301 or 404), but at least the problem will stop getting worse and that is the first step down this road to recover from the damage to our Google search link budget caused by DBSEO.

Reference: GSE "Excluded"

I am happy to be part of forums and your leadership team, finally it's solved, once bots reindex sitemap everything should be fine.

Yes, this problem was more of an "annoyance" that a site-crippling problem.

However, I also would like to note that, you have shown great technical leadership skills at unix.com, as follows:

Thank you again Akshay.

PS: Just awarded you your "the next level up" "Blue Dev Ops" Badge. Well deserved.

This is huge, congrats, great effort guys...

Looks like the site has the following current search position for the "unix" keywords:

Update:

Looks like the site continues to move up in SERP results with Google, staying around the same with Bing and currently down a bit with DuckDuck has the following current search position for the "unix" keywords:

Traffic from search referrals continues to rise (not taking into consideration the US Thanksgiving holiday, of course).