For over a decade, unix.com has been in the top tier for search referrals. The keyword "unix" used to rank #4, and when it was down, it was #9. At times, we were close to #2 on Google for the "unix" keyword. Now, in some geos (in the US for example yesterday), in Google search the "unix" keyword search puts unix.com on page 4.

In other words, unix.com is getting crushed in Google search results based on recent changes to their search algorithm.

This problem started around the end of March, early April.

Our "on page SEO" is great. I have had it reviewed by a number of "SEO people" and they all say it's fine and offer no suggestions for "on page" SEO improvement.

Originally, I thought there was a bug in our Apache2 server causing "soft 404 errors", but after digging deeper into this, I confirmed:

Google Webmaster Tools Shows Problems with Soft 404 Errors

These "soft 404 errors" in Google Search Console are not really "errors". These Google search "soft 404 errors' are generated by Google's internal "AI" (or their algorithms) and they are basically penalizing the forum format where people tend to ask short questions and replies (which are often but not always, short replies).

Yesterday I confirmed this "penalty" by examining dozens of links where Google Search Console reported "soft 404 errors" and in each case I could correct this "error" by adding more text to the posts or replies.

So, basically we are being penalized now for being a "forum" and not a "articles" site.

This seems to correspond to advice from SEO'ers recently (who were not aware of the soft 404 error issue on this site) that we transform the site to a "news article site"... something I have been very hesitant to do (and am not going to do); as I have always wanted unix.com to be a resource to help unix and linux users on a day to day basis when they are in need of help.

Unfortunately, our traffic from Google Search referrals is tanking because of changes in Google's algorithm, and this is bring less and less users to the site with their questions.

This is not the case in Bing, where we rank very high and the "unix" keyword bounces from the #1 to the #3 position daily for this site. But Bing does not not generate much traffic when compared to Google, so unix.com is getting "killed" by Google now due to our decade long forum format.

When I was testing the Google "soft 404 errors" yesterday, I found that if a user had posted a message with a title like:

"Please help me with my file not found error"

Google ranked this as a "soft 404 error" because of the "file not found" in the title, even if there were pages of replies.

In other words, Google's current indexing algorithm is penalizing unix.com and the overall forum format and using some kind of "AI" to even look at the content. If the content had words like "file not found", they classify this as a "soft 404 error". This really took me by surprise. I expected the "thin content" to be problematic, but not "file not found" in a forum user's unix filesystem question.

So, in a nutshell, this is the reason that are visitors have dropped around 50% since the beginning of March. We are getting crushed by Google's algo changes this year.

I do not have any good answer to this. I cannot change how Google does things. Our on-page SEO is fine.

My deep analysis yesterday confirmed that Google is penalizing us ("soft 404 errors", a Google term not a real HTTP error) because we are using a "forum format" and many of the questions and answers are short replies and not long articles with a lot of content. This is especially true of short questions with no answers or questions with deleted answers, etc.