Short story

The problem was given. At the end of the linked posts there is an advised “reverse cloaking” solution link, but all I achieved after implementing that was the Unreachable network error at Google Webmaster Tools.

Days were gone, but finally I found this thread at WebmasterWorld.com. IncrediBILL suggested a solution which based on the reverse-forward DNS robot validation with the following php script:

// Get the user agent.
$ua = $_SERVER['HTTP_USER_AGENT'];
// Check the user agent to see if it's identifying itself as a search engine bot.
if(strstr($ua, 'msnbot') || stristr($ua, 'Googlebot') || stristr($ua, 'Yahoo! Slurp')){
// The user agent is purporting to be MSN's bot or Google's bot or Yahoo! Slurp.
// If the user agent string is spoofed, we won't find googlebot.com in the host name.
// Get the IP address requesting the page.
$ip = $_SERVER['REMOTE_ADDR'];
// Reverse DNS lookup the IP address to get a hostname.
$hostname = gethostbyaddr($ip);
// Check for '.googlebot.com' and '/search.live.com' in hostname.
if(!preg_match("/\.googlebot\.com$/", $hostname) &&!preg_match("/search\.live\.com$/", $hostname) &&!preg_match("/crawl\.yahoo\.net$/", $hostname)) {
// The host name does not belong to either live.com or googlebot.com.
// Remember the UA already said it is either MSNBot or Googlebot.
$block = TRUE;
header("HTTP/1.0 403 Forbidden");
exit;
} else {
// Now we have a hit that half-passes the check. One last go:
// Forward DNS lookup the hostname to get an IP address.
$real_ip = gethostbyname($hostname);
if($ip!= $real_ip){
$block = TRUE;
header("HTTP/1.0 403 Forbidden");
exit;
} else {
// Real bot.
$block = FALSE;
}
}
}
?>

The original script didn’t validate the Yahoo Slurp bot, but I additonally completed the script with it.

So all you need to do is to download the installation package and implement them according to attached guides. If you are a WordPress I have a really good new for you. Due to mosquito a WordPress plugin is also available with guide as well.

Yes, and don’t forget about the testing. Open your Firefox browser and insert “about:config” without quotes into the adress bar and press enter. Press mouse right click, select new string and add “general.useragent.override” as name and “Googlebot/2.1 (+http://www.googlebot.com/bot.html)” as value. Refresh your site after implementing the defending script/plugin and you will see exactly what the robot will see when come through a proxy site.

Downloads

Installation pack

WordPress plugin

With this post I would like to draw attention on some brilliant WordPress plugins which could make your ranking easier concerning search engines.

Providing an xml sitemap will increase the visibility of your site in Google’s index. You don’t need to use online or desktop sitemap builder applications anymore, the google sitemap generator plugin will do the job instead of you.

A static sitemap page improves the PageRank flow among the inner pages and helps your visiors to navigate in your content.

The Category Tagging plugin has an excellent feature. It make the most relevant older posts display at the end of the new publications. Besides improving the PageRank flow it makes your visitors stay longer at your site by providing suitable content for their interest.

SEO can be seperated into two basic parts. The firsts is the onpage optimization and the second one is the link popularity increase. With the next two plugins you will be able to manage both much more succesfully:

• The seo title tag plugin let you customize your title tag all around your blog.
• Jim Westergren’s link to me textbox plugin displays a html code of a link points back to your posts in a textarea so your visitor can easily grab it and insert it into their pages if they like your thoughts. For live demo you need to scroll down a little and you will see what is it all about.

If you also have a favourite let me know!

Thank you,
Bagi Zoltán