PHP判断是不是爬虫的方法 这个一般用于防止爬虫 和 seo优化(因为爬虫都是按照第一次打开显示的页面 有些ajax 等需要点击才能显示的就爬不到啦) <pre> <?php // 判断是否搜索引擎机器人访问 function isRobot() { $agent= strtolower(isset($_SERVER['HTTP_USER_AGENT'])? $_SERVER['HTTP_USER_AGENT'] : ''); if(!empty($agent)){ $spiderSite= array( "TencentTraveler", "Baiduspider+", "BaiduGame", "Googlebot", "msnbot", "Sosospider+", "Sogou web spider", "ia_archiver", "Yahoo! Slurp", "YoudaoBot", "Yahoo Slurp", "MSNBot", "Java (Often spam bot)", "BaiDuSpider", "Voila", "Yandex bot", "BSpider", "twiceler", "Sogou Spider", "Speedy Spider", "Google AdSense", "Heritrix", "Python-urllib", "Alexa (IA Archiver)", "Ask", "Exabot", "Custo", "OutfoxBot/YodaoBot", "yacy", "SurveyBot", "legs", "lwp-trivial", "Nutch", "StackRambler", "The web archive (IA Archiver)", "Perl tool", "MJ12bot", "Netcraft", "MSIECrawler", "WGet tools", "larbin", "Fish search", ); foreach($spiderSite as $val){ $str = strtolower($val); if(strpos($agent, $str) !== false){ return true; } } }
return false; } if(isRobot()){ echo'爬虫'; }else{ echo'不是爬虫'; } ?> </pre>
|
请发表评论