How to Diagnose a Google Penalty
How to Diagnose a Google Ranking Ban, Penalty, or Filter
The following is an excerpt (with some recent modifications and editorial comments) from our book WordPress Search Engine Optimization (now in second edition!). You can buy the book at Amazon.
If you undertake black or gray hat techniques, you run a fair chance of having your site penalized in the search results. But even if you are not engaged in these techniques yourself, your site may be punished for associating with black hat purveyors. Hosting on a shared server or sharing domain registration information with bad neighborhoods can lead to to ranking problems, if not punishment. Certainly linking to a bad neighborhood can lead to discipline. If you purchase a domain, you’ll inherit any penalties or bans imposed on the prior version of the website.
There are a wide range of penalties and ranking filters that search engines impose and a still-wider range of effects that those penalties produce. In diagnosing and correcting ranking problems, more than half the battle is figuring which penalty, if any, is imposed and for what violations. Ranking problems are easy to fix but arduous to diagnose with precision. Sudden drops in rankings might lead you to suspect that you’ve received a penalty, but it might not be a penalty at all.
In the following section we’ll look at some specific penalties, filters, conditions, and false conditions, and how to diagnose ranking problems.
Google Ban
The worst punishment that Google serves upon webmasters in a total ban. This means the removal of all pages on a given domain from Google’s index. A ban is not always a punishment: Google “may temporarily or permanently remove sites from its index and search results if it believes it is obligated to do so by law.” Google warns that punishment bans can be meted out for “certain actions such as cloaking, writing text in such a way that it can be seen by search engines but not by users, or setting up pages/links with the sole purpose of fooling search engines may result in removal from our index.”
One of the most newsworthy instances of a total ban was when Google, in 2006, issued a total ban to the German website of carmaker BMW (http://www.bmw.de). The offense? Cloaked doorway pages stuffed with keywords that were shown only to search engines, and not to human visitors. The incident became international news, ignited at least partially by the SEO blogging community. BMW immediately removed the offending pages and within a few weeks, Google rescinded the ban.
How to Diagnose a Total or Partial Ban
To diagnose a full or partial ban penalty, run the following tests and exercises:
- Check Google’s index. In the Google search field, enter the following specialized search query: “site:yourdomain.com.” Google then returns a list of all of your site’s pages that appear in Google’s index. If your site was formerly indexed and now the pages are removed, there is at least a possibility that your site has been banned from Google.
- Check if Google has blacklisted your site as unsafe for browsing (type http://www.google.com/safebrowsing/diagnostic?site=mysite.com with your domain at the end).
- Check for Nofollow/Noindex settings. It might seem obvious, but check to make sure you haven’t accidentally set your WordPress site to Noindex. To check, go to your WordPress Dashboard and click the “Privacy” option under “Settings.” If the second setting, “I would like to block search engines, but allow normal visitors” is set, then your site will promptly fall out of the index. A stray entry in a robots.txt file or in your WordPress template file can instruct search engines not to index your entire site.
- Check Google Webmaster Tools. Sometimes, but not always, Google will notify you through your Webmaster Tools account that your site has been penalized. But you won’t always receive this message, so you can still be penalized even if you don’t receive it. See the image below for an example message.
PageRank Adjustment/PageRank Penalty
An alternative penalty short of an outright ban is a PageRank adjustment. The adjustment can be partial (a drop from a PR4 to a PR2) or can be full (a drop to PR0). With a PageRank adjustment, Google simply adjusts or removes the PageRank value for a site. Google often imposes this punishment upon low-value general directories that sell links. Part of the difficulty with diagnosing and repairing a PageRank penalty is that the PageRank that Google shows to users is historical, sometimes six months pass between PageRank updates.
How to Diagnose a PageRank Penalty
To diagnose a Google PageRank penalty, run the following tests and exercises:
- Check your inbound links. Whenever your PageRank drops, the most likely reason is that you’ve lost valuable links. Check your link profile in Yahoo Site Explorer. Have you lost any premium, high-PR links you had formerly? Use the reliability of the PageRank algorithm to help diagnose: if you have a PR4 link pointing into one of your pages, and that PR4 link has only one outbound link, that one link alone will be strong enough to make the destination page a PR1 or a PR2. If despite such a link your page remains a PR0, that raises the likelihood of a PageRank penalty.
- Check all pages. Be sure to check every page on your site, you might just have your PageRank shifting around within your site. It is true, however, that generally your home page will have the highest PageRank value of any page of your site. So, if you’ve got a PR0 on all pages including the homepage, a PageRank penalty is suspect.
- Check canonicalization. Recall the “www” and “non-www” distinction and that search engines see these as separate domains in some cases. WordPress handles this automatically, but some online tools don’t check this for you so you have to be sure your are checking both the www and non-www versions of your domain.
- Compare PageRank. Compare Google’s reported PageRank score for your pages with SEOmoz’ mozRank. Typically, these two scores will correlate loosely (within about 10%). If the Google score is much lower than the SEOmoz mozRank score, it’s likely that Google is trimming some PageRank. You can see the SEOmoz Page Rank score with the free SEO Site Tools plugin or by visiting http://www.opensiteexplorer.org/.
- Check internal links. In Google Webmaster Tools, Google reveals its profile of internal links on your site. See the figures below for examples of an unhealthy internal link profile, and a healthy link profile. If your site has 100 indexed pages, but Webmaster Tools references only a handful of links, it means that Google is not properly processing your internal links. We need to be careful here because a range of conditions can cause this. It can potentially arise from a PageRank penalty but also from poor internal navigation structure.
The -950 Ranking Penalty
Google sometimes employs a -950 ranking penalty to individual pages (but not to entire sites) for particular search queries. The -950 penalty means that for a particular search, your page would have 950 positions added to it. So, a term for which you ranked on page one of Google’s search results in position three, you’d now rank on page ninety-five of the search results at position 953. Sound harsh? It is, and Google has made faint references to it as a penalty for over-optimization. Some SEO professionals contend that they have seen the penalty imposed for shady link building practices.
How to Diagnose a -950 Ranking Penalty
Diagnosing a -950 ranking penalty is easy: try search terms for which you formerly ranked (hopefully you noted their exact former position) and follow the search results out to page 95 or 96. Remember that you can always set Google to display 100 results instead of ten by using the advanced search option at Google.com, which is convenient for checking ranking position in the 100s and above.
The -30/-40 Ranking Penalty
Google often serves up another variety of penalty: it’s the -30 or -40 position penalty. This is an often-imposed penalty, and is applied by Google to entire sites, not just particular pages and not just for particular search queries. This penalty is common enough to trip up legitimate webmasters for very minor oversights or offenses. Most signs point to the -30 penalty being applied algorithmically and is “forgivable,” so changing the condition that led to the penalty automatically reverses the penalty. This penalty has historically been imposed upon sites for serving up poor quality content. For example, the penalty has been imposed upon sites that display thin content. Thin content is content that is partially generic, as with an affiliate site repeating common descriptions of products it sells. Low-value directories have also been served this penalty.
How to Diagnose a -30/-40 Penalty
If you suspect that your site has been been hit with a -30/-40 penalty, there is one sure-fire test to determine if you tripped the penalty. Perform a Google search for your domain name, with out the “www” and without the “.com” or “.net” part of the domain. This search, in normal circumstances, should return your site at or near the first position (depending a bit on the competition of that term). If this test yields your site showing up in a position dropped to the 40s or 50s, it is almost certainly is a -30/-40 penalty.
False Positives That Aren’t Penalties
Don’t assume you’ve been penalized by Google just because your rankings drop or because your rankings remain poor for a new site. Ranking positions can jump around naturally, especially just before algorithm updates, when Google updates its search engine rules. You may also have lost one or more valuable inbound links, that can lead to a drop in rankings. You may also be alternating between Google’s personalized search modes. Personalized search is a Google feature that returns results based on your personal browsing habits. So, if you’ve visited your own website in the past few days, Google will return your website near the top of the results, figuring that it’s one of your personal favorites. Personal search is a convenience tool, but it doesn’t return true rankings. To see actual ranking results you need to make sure personalized search is off. To do this, look on any Google search results page in the upper left hand corner for “Personalize Search On.” Click on the link just under it that reads, “Turn it off.”
Google penalties are almost never imposed for no reason at all. Yes, Google imposes penalties on light offenders while more egregious violations go unpunished. While that might not seem fair, it doesn’t change the fact that if you have perfectly complied with Google’s Webmaster Guidelines, you are extremely unlikely to be penalized. If you’ve been penalized, there’s a reason.
Brilliant analysis and nicely written, too. Many thanks! I learned a lot about the Google penalty process.
Hi Has google confirmed any of the -950 or -40 statements you made? I’ve not seen those before or is based on data built up by an seo firm?
It’s all part of the collective intelligence on the topic–in all fairness to readers I have not received confirmation from Google on the specific penalties I have outlined here. I have personally seen a -40 in action.
Thanks really helpful. It gave clear problem and verification (solution ) layout and this has been the most helpful site so far.
Wow, I can’t thank you enough for this insight. My site appears to have received the 30/40 penalty and had no clue what the issue could be until reading your post. At least now I know what to focus on to restore. On a side note, the fools over at the google webmasters forum are about useless and don’t seem to have 1/2 the clue you do. Thanks again!
Thank u for your useful information.
Thank you, this is really very helpful.
a good article, thanks. should we design for viewer or for google?
well i dont think i got the 30/40 hit.. i got the full hit.. my rank was 6 years old 50k into it and bang!
back to 0 in google 3 weeks ago! after spending 50k i dont trust google any more! i have 5 people out of work this just sucks! thanks google i did nothing wrong .. i use google webmaster tools I try everything i can no luck.. it might take 3 months to get back up but i dont have the time i cannot pay my guys for 3 months of no work! we were on the top one for remote computer repair and so on! help!
That’s what i was looking. Great !
I wants to know that the -30/ -40 penalty checking method you describe above as same for the blogs (abc.blogspot or xyz.wordpress) ?
Very nice article Michael, you write about all the possible penalties but you forgot the 400 penalty, i have 3 websites with the 400 one and i would like to know if anyone knows how i can get out, thanks.
I’d like to take a look at your circumstances…can you email me at michael[@]tastyplacement.com?
Dear Michael, thanks for the article! I still have some questions: I think that one of our sites has undergone a penalty but I am not sure. The site is neither completely banned, nor has lost pagerank. The site lost some positions but not as you described in the article. It is neither a -950 nor a -30/40 penalty. The site was listed for about 300 keywords mostly on page one. But within one day almost all rankings lost between 2 to 10 positions. Some lost 20 to 50. Some stayed and some even improved. Do you have an explanation? I am kind of desperate.
doesn’t sound like a penalty at all, just an algorithm shift…from the date you posted this I will be this happened about january 10th. A lot of shifting in ranking happened then. We benefited, but of course not everyone does during these updates.
Thanks for the article. I feel a bit better. My site’s new but the impressions have been steadily going up until the past couple of days where they dropped twice. I’m hoping it’s just a change in algorithm or due to links from big sites going to competitors. I’ll wait a few days to see what happens.
Hi Michael,
thanks for your awesome article. My website “echeion dot it” suddenly dropped from Page Rank 2 to 1 but it doesn’t look like there were any penalties. Maybe the algorithm has changed, hopefully this is the reason.
In google webmaster I noticed also in the last couple of days a stead increase of query impressions, from 3500 a day to 8000 a day. So this should be good but the page rank is lower instead….do you know if this is good or bad?
Thanks
Hi Michael, thanks for writing this article, very clear.
One of our site a few days ago disappeared from google search results even though is still indexed.
Pagerank went from 1 to none, neither a 950 nor a 30/40 penalty.
For a few keywords we were ranking on page one and now all seems to be disappeared along with our homepage and other pages.
We’re kind of desperate and with no clue whether we were penalized or else.
Does this sound as penalty to you?
Dear Michael, thanks for the article. A couple of days ago, i realised that the homepage of one of my sites have lost its index on google search. Using site:myhomepage have shown that my homepage has completely disappeared even after having it up on index for over 2 years. I have been wondering for days what happened and had been resubmitting sitemap on webmasters with a return of half of my pages unindexed when previously it was 100% indexed. Today i checked on the privacy settings on wordpress and realised that it was switched to prevent search engines from indexing. I have since turned it on and resubmitted sitemap on webmasters and still did not get my homepage indexed. any idea if i will be penalized for this? I just realised that the setting on privacy has been turned off for almost 2 weeks and am worried that i will never get my homepage indexed again. There are no messages on webmasters and have check that the site is not blacklisted on google. Any ideas? Thanks in advance.
I don’t think you’ll be penalized for that. Typically, the sites come back in about 2 weeks.
Google’s JohnMu says that you should simply submit a reconsideration request if you are unsure if you have a manual penalty. You will get a response immediately if you don’t and you will wait up to 3 weeks for your request to be looked at so they can tell you it failed and that you have a penalty.
Hello!
I’ve read the article with interest as my PR recently dropped from 4 to 0. Looks like I might’ve been penalized and I even think I might know the reason.
What I lack here – not a word about how can we check / make sure / ask Google about it? Ask for reconsideration if I identify the offence and remove it?
Or is it just so obviousl fruitless anyway that you just don’t mention it?
Greetings!
I’ve been writing on my site for 2.5 years now. At one point, I had a PR of 2. But for over a year now, It’s been a 0. I’ve since switched from blogger to wordpress, and I’m running adsense, and I know everything is fine and legit there. But for the life of me, I can NOT figure out why my site has a 0 PR. I looked on http://www.opensiteexplorer.org/ and I have a rank on there of 4.something. So I’m guessing I’ve been penalized, but I honestly have NO idea how or why I would have been. I don’t have anything on there that would be worse than a rated R movie, I don’t link farm, and I honestly have NO idea what the issue could be. Can anyone help?
Hi Michael,
Great post and I love this article. After check my site I found that some site I am hosted with were responsible for malware distribution. How hard do you think this will effect my ranking?
Hi Michael,
My web directory is hit by the 30-40 penalty i guess as it is in the 40’s on google without the www and .com . The strange thing is the PR has also been cut by half. As i see from your article the 30-40 penalty can be reversed by improving the content of the website but how to get the PR back. Is it two penalties or what?
My website is a web directory.
Regards
I miss those days where ranking on G was easy as pushing a
button.
Today you have to be extremely careful with your SEO
or you could be easily out of the game if a new update comes.
. And think that your suggestions are great to survive all this.
Thank you for sharing your info. I really appreciate your efforts and I am
waiting for your next write ups thank you once again.
Our site had a PR score of 3 on google. We build a brand new site on wordpress, much of the content and pages stayed the same. The PR score never transferred over to the newly designed site, it remained a 0. Why is that? All other indicators look fine. We’re indexed well and rank well on google.
Thankss Bud ! this help me alot with my site !