Nos dirigimos a :
TOOLS & SETTINGS -> View All General Settings -> Buscamos Search Engine Spiders
Ahi veremos unas cuantas opciones, solo tenemos que ditar una "Spider Bot User-Agent", borramos todo lo que hay ahi y agregamos esta lista tal cual esta
CÓDIGO
:robot/1.0=Neofonie.de
0.1_hseo(at)cs.rutgers.edu=Rutgers.edu
AbachoBOT (Mozilla compatible)=Crawler.de
AbachoBOT=Crawler.de
abcdatos_botlink=Abcdatos.com
About/0.1libwww-perl/5.47=About.com
abot/0.1=Abot.com
abot/0.1 (abot; http://www.abot.com; abot[arroba]abot.com)=Abot.com
accoona=Accoona.com
AcoiRobot=Acoi Picture Robot
Acoon Robot v1.50.001=Acoon.de
Acoon Robot v1.52=Acoon.de
Acoon Robot=Accon.de
ActiveWorlds/3.xx=ActiveWorlds.com
admin[arroba]crawler.de=Crawler.de
admin[arroba]maxbot.com=maxbot.com
ADSARobot=ADSARobot
AESOP_com_SpiderMan=Aesop.com
agadine/1.x.x=Agada.de
Agent-admin/=Daum.net
ah-ha.com crawler (crawler[arroba]ah-ha.com)=Ah-ha.com
ah-ha.com crawler=Ah-ha.com
Aladin/3.324=Aladin.de
Aleksika Spider/1.0=Aleksika.de
AlkalineBOT/1.3=Vestris.com
AlkalineBOT/1.4=Vestris.com
AlkalineBOT=AlkalineBOT
Allesklar/0.1 libwww-perl/5.46=Allesklar.de
AltaVista-Intranet=Altavista.co.uk
AmfibiBOT=Amfibi.com
Amiga-AWeb/3.4.167SE=AWeb.com
amzn_assoc=Amazon.com
AnswerChase PROve x.0=AnswerChase.com
AnswerChase x.0=AnwserChase.com
AnswerChase=AnswerChase.com
Antibot=AntiSeach.net
antibot-V1.1.5/i586-linux-2.2=AntiSeach.net
AnzwersCrawl/2.0=Anzwers
A-Online Search=Jet2Web.com
Aport=Aport.ru
appie 1.1=Walhello.com
appie=Walhello.com
arachnoidea[arroba]euroseek.net=Euroseek.net
Arachnoidea=Euroseek.com
Aranha=Girafa.com
ArchitectSpider=Excite.com
archive.org_bot=Archive.org
archive_org=Archive.org
ask jeeves=Ask Jeeves
ask.24x.info=Ask.24x.info
ASPSeek/1.2.5=ASPSeek.org
ASPseek/1.2.9d=ASPSeek.com
ASPSeek/1.2.x=ASPSeek.org
ASPSeek/1.2.xa=ASPSeek.org
ASPseek/1.2.xx=ASPseek.org
ASPSeek/1.2.xxpre=ASPSeek.org
ASPSeek=ASPSeek.org
asterias/2.0=SingingFish.com
Asterias=SingingFish.com
AtlocalBot=Atlocal.com
AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html)=Atlocal.com
Atomz/1.0=Atomz.com
Atomz=Atomz.com
Augurfind=Augurnet.ch
augurnfind V-1.x=Augurnet.ch
autohttp=Elsop.com
AV Fetch 1.0=Altavista.com
AVSearch=Altavista.com
AVSearch-3.0=Altavista.com
AxmoRobot=Axmo.com
Baiduspider+=Baidu.com
BaiDuSpider=Baidu.com
BanBots/1.2=Banbots.com
BarraHomeCrawler=BarraHome.com
bdcindexer_2.6.2=Business.com
BDFetch=Brandimensions.com
BDNcentral Crawler v2.3=Bdncentral.com
beautybot/1.0=Cosmoty.com
BigCliqueBOT/1.03-dev=BigClique.com
Bilbo/2.3b-UNIX=Biblo Bot
BlackWidow=BlackWidow Bot
Blaiz-Bee/1.0 (+http://www.blaiz.net)=Blaiz.net
Blaiz-Bee/1.0=Blaiz.net
BlitzBOT[arroba]tricus.net (Mozilla compatible)=Blitzsuche.rp-online.de
BlitzBOT[arroba]tricus.net=Blitzsuche.rp-online.de
BlogBot/1.x=Blogdex.net
BlogBot=Blogdex.net
blogWatcher_Spider/0.1=Blogwatcher.pi.titech.ac.jp
Bloglines Title Fetch/1.0 (http://www.bloglines.com)=Bloglines.com
boitho.com-dc/0.xx=Boitho.com
boitho.com-robot/1.x=Boitho.com
BSDSeek/1.0=Inktomi.com
BullsEye=Intelliseek.com
bumblebee[arroba]relevare.com=Relevare.com
Buscaplus Robi/1.0=Buscaplus.com
Buscaplus Robi=Buscaplus.com
CanSeek/=Cansee.ca
ChristCRAWLER 2.0=Christcrawler.com
ChristCRAWLER=Christcrawler.com
CipinetBot=Cipinet.com
citenikbot/=Cenik.co.uk
ClariaBot/1.0=Searchscout.com
Claymont.com=Claymont.com
CLIPS-index=Cips-index.imag.fr
Clushbot/3.3-BinaryFury (+http://www.clush.com/bot.html)=Clush.com
Clushbot/3.3-BinaryFury=Clush.com
Combine/x.0=Combine Bot
Computer_and_Automation_Research_Institute_Crawler=Research Bot
contact/jylee[arroba]kies.co.kr=Kies.co.kr
Convera Internet Spider V6.x=Convera.com
cosmos/0.8_(robot[arroba]xyleme.com)=Xyleme.com
cosmos/0.9_(robot[arroba]xyleme.com)=Xyleme.com
cosmos=Xyleme.com
C-PBWF-ip3000.com-crawler=IP3000.com
CrawlConvera0.1=Converna.net
Crawler (cometsearch[arroba]cometsystems.com)=Findwhat.com
Crawler admin[arroba]crawler.de=Crawler.de
Crawler V 0.2.x admin[arroba]crawler.de=Crawler.de
crawler[arroba]alexa.com=Alexa.com
crawler[arroba]brainbot.com=Brainbot.com
crawler[arroba]fast.no=Alltheweb.com
Crawler=Crawler.de
CrawlerBoy Pinpoint.com=Pinpoint.com
CrawlerBoy=Pinpoint.com
CrocCrawler= CrocCrawler.com
Custom Spider www.bisnisseek.com /1.0=Bisnisseek.com
DaAdLe.com ROBOT/=Daadle.com
DataparkSearch/4.xx=DataparkSearch.com
DaviesBot/1.7=Wholeweb.net
DaviesBot=Wholeweb.net
dbDig=Prairielandconsulting.com
DBrowse 1.4b=DBrowse Bot
DBrowse 1.4d=DBrowse Bot
dCSbot/1.1=Openmarket.com
DeepIndex=DeepIndex.com
DeepIndexer.ca=DeepIndex.com
deepweb=Deepweb.com
Demo Bot DOT 16b= Btopenworld.com
DiaGem/=Skyrocket.gr.jp
DiaGem/1.1 (http://www.skyrocket.gr.jp/diagem.html)=Skyrocket.gr.jp
DiaGem/1.1=Skyrocket.gr.jp
Diamond/1.0=Searchscout.com
Digger/1.0 JDK/1.3.0rc3=Diggit.com
Digimarc WebReader=Digimarc.com
DigOut4U=Arisem.com
DIIbot/1.2=Digital-Integrity.com
DIIbot=Digital-Integrity.com
DittoSpyder=Ditto.com
dloader(NaverRobot)/=Naver.com
dtSearchSpider=Dtsearch.com
Dumbot(version 0.1 beta - dumbfind.com)=Dumbfind.com
Dumbot(version 0.1 beta)=Dumbfind.com
dumrobo(NaverRobot)/=Naver.com
EARTHCOM.info/1.4beta=Earthcom.info
EasyDL/3.02=Keywen.com
EasyDL/3.03=Keywen.com
EasyDL/3.04=Keywen.com
EasyDL/3.xx http://keywen.com/Encyclopedia/Bot=Keywen.com
EasyDL/3.xx=Keywen.com
EchO!/2.0=Voila.com
egothor/3.0a (+http://www.xdefine.org/robot.html)=Xdefine.com
egothor/3.0a=Xdefine.com
EgotoBot/4.8=Egoto.com
Enterprise_Search/1.0.xxx=Innerprise.net
Enterprise_Search/1.0=Innerprise.net
erik[arroba]malfunction.org=Malfunction.org
EroCrawler=Erocrawler.com
ES.NET_Crawler/2.0=Innerprise.net
eseek-larbin_2.6.2=Exactseek.com
ESISmartSpider=Travel-Finder.com
ExactSeek Crawler/0.1=Exactseek.com
exactseek-crawler-2.63=Exactseek.com
Exalead NG/MimeLive Client=Exabot.com
Excalibur Internet Spider V6.5.4=Excalib.com
ExperimentalHenrytheMiragoRobot=Mirago.co.uk
EyeCatcher (Download-tipp.de)/1.0=Download-tipp.de
EyeCatcher=Download-tipp.de
EZResult=Ezresults.com
Fast Crawler Gold Edition=Alltheweb.com
FAST Data Search Crawler=Alltheweb.com
FAST Enterprise Crawler 6 (Experimental)=Alltheweb.com
FAST Enterprise Crawler 6 used by FAST (FAST)=Alltheweb.com
FAST Enterprise Crawler 6 used by FAST=Alltheweb.com
FAST Enterprise Crawler 6=Alltheweb.com
FAST Enterprise Crawler/6 (www.fastsearch.com)=Alltheweb.com
FAST Enterprise Crawler/6=Alltheweb.com
FAST FirstPage retriever (compatible; MSIE 5.5; Mozilla/4.0)=Alltheweb.com
FAST FirstPage retriever=Alltheweb.com
Fast PartnerSite Crawler=Alltheweb.com
FastCrawler 3.0.1 (crawler[arroba]1klik.dk)=Alltheweb.com
FastCrawler 3.0.1=Alltheweb.com
FAST-WebCrawler/2.2.10=Altheweb.com
FAST-WebCrawler/2.2.6=Altheweb.com
FAST-WebCrawler/2.2.7=Altheweb.com
FAST-WebCrawler/2.2.8=Altheweb.com
FAST-WebCrawler/3.2 test=Altheweb.com
FAST-WebCrawler/3.2=Altheweb.com
FAST-WebCrawler/3.3=Altheweb.com
FAST-WebCrawler/3.4/Nirvana=Altheweb.com
FAST-WebCrawler/3.4/PartnerSite=Altheweb.com
FAST-WebCrawler/3.5=Altheweb.com
FAST-WebCrawler/3.6/FirstPage=Altheweb.com
FAST-WebCrawler/3.6=Altheweb.com
FAST-WebCrawler/3.7/FirstPage=Altheweb.com
FAST-WebCrawler/3.7=Altheweb.com
FAST-WebCrawler/3.8/Fresh=Altheweb.com
FAST-WebCrawler/3.8=Altheweb.com
FAST-WebCrawler/3.x Multimedia (mm dash crawler at fast dot no)=Altheweb.com
FAST-WebCrawler/3.x Multimedia=Altheweb.com
FAST-WebCrawler=Alltheweb.com
Faxobot/1.0=FaXo.com
Felix - Mixcat Crawler=Mixcat.com
0.1_hseo(at)cs.rutgers.edu=Rutgers.edu
AbachoBOT (Mozilla compatible)=Crawler.de
AbachoBOT=Crawler.de
abcdatos_botlink=Abcdatos.com
About/0.1libwww-perl/5.47=About.com
abot/0.1=Abot.com
abot/0.1 (abot; http://www.abot.com; abot[arroba]abot.com)=Abot.com
accoona=Accoona.com
AcoiRobot=Acoi Picture Robot
Acoon Robot v1.50.001=Acoon.de
Acoon Robot v1.52=Acoon.de
Acoon Robot=Accon.de
ActiveWorlds/3.xx=ActiveWorlds.com
admin[arroba]crawler.de=Crawler.de
admin[arroba]maxbot.com=maxbot.com
ADSARobot=ADSARobot
AESOP_com_SpiderMan=Aesop.com
agadine/1.x.x=Agada.de
Agent-admin/=Daum.net
ah-ha.com crawler (crawler[arroba]ah-ha.com)=Ah-ha.com
ah-ha.com crawler=Ah-ha.com
Aladin/3.324=Aladin.de
Aleksika Spider/1.0=Aleksika.de
AlkalineBOT/1.3=Vestris.com
AlkalineBOT/1.4=Vestris.com
AlkalineBOT=AlkalineBOT
Allesklar/0.1 libwww-perl/5.46=Allesklar.de
AltaVista-Intranet=Altavista.co.uk
AmfibiBOT=Amfibi.com
Amiga-AWeb/3.4.167SE=AWeb.com
amzn_assoc=Amazon.com
AnswerChase PROve x.0=AnswerChase.com
AnswerChase x.0=AnwserChase.com
AnswerChase=AnswerChase.com
Antibot=AntiSeach.net
antibot-V1.1.5/i586-linux-2.2=AntiSeach.net
AnzwersCrawl/2.0=Anzwers
A-Online Search=Jet2Web.com
Aport=Aport.ru
appie 1.1=Walhello.com
appie=Walhello.com
arachnoidea[arroba]euroseek.net=Euroseek.net
Arachnoidea=Euroseek.com
Aranha=Girafa.com
ArchitectSpider=Excite.com
archive.org_bot=Archive.org
archive_org=Archive.org
ask jeeves=Ask Jeeves
ask.24x.info=Ask.24x.info
ASPSeek/1.2.5=ASPSeek.org
ASPseek/1.2.9d=ASPSeek.com
ASPSeek/1.2.x=ASPSeek.org
ASPSeek/1.2.xa=ASPSeek.org
ASPseek/1.2.xx=ASPseek.org
ASPSeek/1.2.xxpre=ASPSeek.org
ASPSeek=ASPSeek.org
asterias/2.0=SingingFish.com
Asterias=SingingFish.com
AtlocalBot=Atlocal.com
AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html)=Atlocal.com
Atomz/1.0=Atomz.com
Atomz=Atomz.com
Augurfind=Augurnet.ch
augurnfind V-1.x=Augurnet.ch
autohttp=Elsop.com
AV Fetch 1.0=Altavista.com
AVSearch=Altavista.com
AVSearch-3.0=Altavista.com
AxmoRobot=Axmo.com
Baiduspider+=Baidu.com
BaiDuSpider=Baidu.com
BanBots/1.2=Banbots.com
BarraHomeCrawler=BarraHome.com
bdcindexer_2.6.2=Business.com
BDFetch=Brandimensions.com
BDNcentral Crawler v2.3=Bdncentral.com
beautybot/1.0=Cosmoty.com
BigCliqueBOT/1.03-dev=BigClique.com
Bilbo/2.3b-UNIX=Biblo Bot
BlackWidow=BlackWidow Bot
Blaiz-Bee/1.0 (+http://www.blaiz.net)=Blaiz.net
Blaiz-Bee/1.0=Blaiz.net
BlitzBOT[arroba]tricus.net (Mozilla compatible)=Blitzsuche.rp-online.de
BlitzBOT[arroba]tricus.net=Blitzsuche.rp-online.de
BlogBot/1.x=Blogdex.net
BlogBot=Blogdex.net
blogWatcher_Spider/0.1=Blogwatcher.pi.titech.ac.jp
Bloglines Title Fetch/1.0 (http://www.bloglines.com)=Bloglines.com
boitho.com-dc/0.xx=Boitho.com
boitho.com-robot/1.x=Boitho.com
BSDSeek/1.0=Inktomi.com
BullsEye=Intelliseek.com
bumblebee[arroba]relevare.com=Relevare.com
Buscaplus Robi/1.0=Buscaplus.com
Buscaplus Robi=Buscaplus.com
CanSeek/=Cansee.ca
ChristCRAWLER 2.0=Christcrawler.com
ChristCRAWLER=Christcrawler.com
CipinetBot=Cipinet.com
citenikbot/=Cenik.co.uk
ClariaBot/1.0=Searchscout.com
Claymont.com=Claymont.com
CLIPS-index=Cips-index.imag.fr
Clushbot/3.3-BinaryFury (+http://www.clush.com/bot.html)=Clush.com
Clushbot/3.3-BinaryFury=Clush.com
Combine/x.0=Combine Bot
Computer_and_Automation_Research_Institute_Crawler=Research Bot
contact/jylee[arroba]kies.co.kr=Kies.co.kr
Convera Internet Spider V6.x=Convera.com
cosmos/0.8_(robot[arroba]xyleme.com)=Xyleme.com
cosmos/0.9_(robot[arroba]xyleme.com)=Xyleme.com
cosmos=Xyleme.com
C-PBWF-ip3000.com-crawler=IP3000.com
CrawlConvera0.1=Converna.net
Crawler (cometsearch[arroba]cometsystems.com)=Findwhat.com
Crawler admin[arroba]crawler.de=Crawler.de
Crawler V 0.2.x admin[arroba]crawler.de=Crawler.de
crawler[arroba]alexa.com=Alexa.com
crawler[arroba]brainbot.com=Brainbot.com
crawler[arroba]fast.no=Alltheweb.com
Crawler=Crawler.de
CrawlerBoy Pinpoint.com=Pinpoint.com
CrawlerBoy=Pinpoint.com
CrocCrawler= CrocCrawler.com
Custom Spider www.bisnisseek.com /1.0=Bisnisseek.com
DaAdLe.com ROBOT/=Daadle.com
DataparkSearch/4.xx=DataparkSearch.com
DaviesBot/1.7=Wholeweb.net
DaviesBot=Wholeweb.net
dbDig=Prairielandconsulting.com
DBrowse 1.4b=DBrowse Bot
DBrowse 1.4d=DBrowse Bot
dCSbot/1.1=Openmarket.com
DeepIndex=DeepIndex.com
DeepIndexer.ca=DeepIndex.com
deepweb=Deepweb.com
Demo Bot DOT 16b= Btopenworld.com
DiaGem/=Skyrocket.gr.jp
DiaGem/1.1 (http://www.skyrocket.gr.jp/diagem.html)=Skyrocket.gr.jp
DiaGem/1.1=Skyrocket.gr.jp
Diamond/1.0=Searchscout.com
Digger/1.0 JDK/1.3.0rc3=Diggit.com
Digimarc WebReader=Digimarc.com
DigOut4U=Arisem.com
DIIbot/1.2=Digital-Integrity.com
DIIbot=Digital-Integrity.com
DittoSpyder=Ditto.com
dloader(NaverRobot)/=Naver.com
dtSearchSpider=Dtsearch.com
Dumbot(version 0.1 beta - dumbfind.com)=Dumbfind.com
Dumbot(version 0.1 beta)=Dumbfind.com
dumrobo(NaverRobot)/=Naver.com
EARTHCOM.info/1.4beta=Earthcom.info
EasyDL/3.02=Keywen.com
EasyDL/3.03=Keywen.com
EasyDL/3.04=Keywen.com
EasyDL/3.xx http://keywen.com/Encyclopedia/Bot=Keywen.com
EasyDL/3.xx=Keywen.com
EchO!/2.0=Voila.com
egothor/3.0a (+http://www.xdefine.org/robot.html)=Xdefine.com
egothor/3.0a=Xdefine.com
EgotoBot/4.8=Egoto.com
Enterprise_Search/1.0.xxx=Innerprise.net
Enterprise_Search/1.0=Innerprise.net
erik[arroba]malfunction.org=Malfunction.org
EroCrawler=Erocrawler.com
ES.NET_Crawler/2.0=Innerprise.net
eseek-larbin_2.6.2=Exactseek.com
ESISmartSpider=Travel-Finder.com
ExactSeek Crawler/0.1=Exactseek.com
exactseek-crawler-2.63=Exactseek.com
Exalead NG/MimeLive Client=Exabot.com
Excalibur Internet Spider V6.5.4=Excalib.com
ExperimentalHenrytheMiragoRobot=Mirago.co.uk
EyeCatcher (Download-tipp.de)/1.0=Download-tipp.de
EyeCatcher=Download-tipp.de
EZResult=Ezresults.com
Fast Crawler Gold Edition=Alltheweb.com
FAST Data Search Crawler=Alltheweb.com
FAST Enterprise Crawler 6 (Experimental)=Alltheweb.com
FAST Enterprise Crawler 6 used by FAST (FAST)=Alltheweb.com
FAST Enterprise Crawler 6 used by FAST=Alltheweb.com
FAST Enterprise Crawler 6=Alltheweb.com
FAST Enterprise Crawler/6 (www.fastsearch.com)=Alltheweb.com
FAST Enterprise Crawler/6=Alltheweb.com
FAST FirstPage retriever (compatible; MSIE 5.5; Mozilla/4.0)=Alltheweb.com
FAST FirstPage retriever=Alltheweb.com
Fast PartnerSite Crawler=Alltheweb.com
FastCrawler 3.0.1 (crawler[arroba]1klik.dk)=Alltheweb.com
FastCrawler 3.0.1=Alltheweb.com
FAST-WebCrawler/2.2.10=Altheweb.com
FAST-WebCrawler/2.2.6=Altheweb.com
FAST-WebCrawler/2.2.7=Altheweb.com
FAST-WebCrawler/2.2.8=Altheweb.com
FAST-WebCrawler/3.2 test=Altheweb.com
FAST-WebCrawler/3.2=Altheweb.com
FAST-WebCrawler/3.3=Altheweb.com
FAST-WebCrawler/3.4/Nirvana=Altheweb.com
FAST-WebCrawler/3.4/PartnerSite=Altheweb.com
FAST-WebCrawler/3.5=Altheweb.com
FAST-WebCrawler/3.6/FirstPage=Altheweb.com
FAST-WebCrawler/3.6=Altheweb.com
FAST-WebCrawler/3.7/FirstPage=Altheweb.com
FAST-WebCrawler/3.7=Altheweb.com
FAST-WebCrawler/3.8/Fresh=Altheweb.com
FAST-WebCrawler/3.8=Altheweb.com
FAST-WebCrawler/3.x Multimedia (mm dash crawler at fast dot no)=Altheweb.com
FAST-WebCrawler/3.x Multimedia=Altheweb.com
FAST-WebCrawler=Alltheweb.com
Faxobot/1.0=FaXo.com
Felix - Mixcat Crawler=Mixcat.com