geek_stuff/web & network 2007. 6. 20. 21:45

로봇 리스트..

한동안 블로그의 방문자가 300명을 벗어나지 못하더니 어느날 부터인가 1000명이 찾아오기 시작..

그러고선 하루에 방문자가 2000명이 넘기기까지 했습니다.

이게 어찌된 영문인가 곰곰히 생각해보니

검색봇이 제 블로그를 하루에도 몇번씩 긁어가는가봅니다.
(검색봇: 야후나 네이버 같은 검색엔진이 자신들의 검색결과를 나타내기 위해서 임의의 사이트를 방문해서 정보를 긁어오는 프로그램을 검색봇이라고 합니다)

전세계에 있는 검색봇을 일일히 다 찾아내기란 거의 불가능하고... 뭐 그래서 안되겠다, 대책을 세워야지 하면서 고민을 하다가 어느날 우연히 로봇리스트를 어디선가 찾아서 바탕화면에 고이 저장해 두었습니다.

귀차니즘으로 인한 2개월 숙성과정을 거친 후 crizin님께서 만드신 로봇의 방문횟수 제외 플러그인에 있는 로봇리스트가 부족하길래, 추가해봤습니다.


며칠 후...


하루에 2000명 방문자는 역시나 구라였던 겁니다. 네네, 하루에 100명이 찾아주시는군요 네.;;

하하 2000명 가라 유저(?)들 보다는 100명의 진짜유저가 더 소중히 여겨지는 게 당연한겁니다.

방문해주셔서 감사하구요, 아래는 선물로 그 로봇리스트를 붙여 넣었으니까..  진실을 원하시는 분은

티스토리 플러그인에서 로봇의 방문횟수 제외 플러그인의 설정안에 아래 리스트를 Copy & Paste하셔서 붙여넣으시길 바래요 :~D

사용자 삽입 이미지



1Noonbot,Accoona-AI-Agent,Allblog.net,Baiduspider,Blogbeat,Crawler,DigExt,DrecomBot,Exabot,FeedChecker,FeedFetcher,Gigabot,Googlebot,
Googlebot-Image,HMSE_Robot,IP*Works!,IRLbot,Jigsaw,LWP::Simple,Labrador,MJ12bot,MirrorChecking,MissiguaLocator,MSNBot,NG/2.0,NaverBot,
NutchCVS,PEARHTTP_Request,PostFavorites,SBIder,W3C_Validator,WISEbot,Y!J-BSC,Yahoo!Slurp,Yahoo-MMCrawler,ZyBorg,voyager,archiver,carleson,cfetch,sogou,compatible;Eolin,cowbot,empas,favicon,feedfinder,findlinks,
genieBot,ichiro,ia_archiver,kinjabot,larbin,lwp-trivial,msnbot,psbot,urllib/1.15,wisebot,ADSAComponent,Almaden,Almaden c01,Almaden wf224,Almaden wf55,Amfibibot 0.06,Amsterdam 2002,AnswerBus,Antibot 1.1.11,Antibot 1.1.13,Antibot 1.2.0,AOL 3.0,Appie 1.1,Argus 1.1,Art-Online 0.9,Ask Jeeves,Ask Jeeves Teoma,ASPseek 1.2.10,ASPseek 1.2.11,ASPseek 1.2.12,AvantGo 3.2,BaiDuSpider,BaiDuSpider,Battlebot,BDFetch,BDNcentral Crawler 2.3,BDNcentral Crawler 2.3,Big Brother,BlogBot 1.2,Boitho-dc 0.4,Boitho-dc 0.5,Boitho-dc 66,Boitho-robot 1.0,Boitho-robot 1.1,BorderManager 3.0,BrailleBot 1.0,BruinBot,Bumblebee 1.0,CARI Crawler,CARI Crawler,Cd34 0.1,Cerberian Drtrs,Cerberian Drtrs 3.0,Cerberian Drtrs 3.0,CipinetBot,Clushbot 2.1,Clushbot 3.21,Clushbot 3.23,Clushbot 3.24,Clushbot 3.6,Clushbot 3.9,ComMOOnity LambdaMOO 1.8.1,ConveraCrawler 0.1,ConveraCrawler 0.1,ConveraCrawler 0.2,ConveraCrawler 0.5,Cosmos 0.9,Cowbot 0.1,Cowbot 0.1.1,Crawl_Application,CrocCrawler 3.3,CrocCrawler 4.3,Custo 2.0,CydralSpider 1.9,DeepIndex,DeMozulator 1.0,DoCoMo 1.0,Dual Proxy,Dumbot,EARTHCOM.info 1.2,EmailSiphon,Enterprise_Search 00.136,E-SocietyRobot,Exactseek-crawler 2.63,Exactseek-crawler 2.63,Explorer 6,FAST Enterprise Crawler 6,FAST FirstPage retriever,FastBug,FAST-WebCrawler .2,FAST-WebCrawler 3.4,FAST-WebCrawler 3.6,FAST-WebCrawler 3.6,FAST-WebCrawler 3.7,FAST-WebCrawler 3.8,FAST-WebCrawler 3.x,Favicon,FDSE robot,Filangy 0.01,Filangy 1.01,FindLinks 0.71,Findlinks 0.82,Findlinks 0.87,Findlinks 0.89,Firefly 1.0,FlickBot 1.1,FlickBot 2.0,Fluffy the spider,FusionBot,FyberSpider,GAIS Robot 1.1A2,Gaisbot 3.0,GalaxyBot 1.0,Gatherer 0.9,Gazz 5.0,Generic,GeonaBot 1.0,GeonaBot 1.1,GetRight 4.5,Gigabot 1.0,Girafabot,Goldfire Server,Googlebot 2.1,Googlebot Test,Googlebot-Image 1.0,Green Research, Inc.,GregBot,Grub-client,Grub-client 0.3.0,Grub-client 1.0.3,Grub-client 1.0.4,Grub-client 1.0.5,Grub-client 1.0.6,Grub-client 1.0.7,Grub-client 1.07,Grub-client 1.1.1,Grub-client 1.2.1,Grub-client 1.3.1,Grub-ent 1.3.7,Grub-client 1.4.3,Grub-client 1.5.3,Grub-client 2.3,GulperBot 0.2.4,Harvest-NG 1.0.2,Hatena Antenna 0.4,Hget 0.3,Hitwise Spider 1.0,Htdig,Htdig 3.1.5,Htdig 3.1.6,Html Link Validator,Httpcheck 1.0,HTTPConnect,Httpget 5.2.2,HTTrack 3.0x,Ia_archiver,IcabotAccept,Ichiro 1.0,IconSurf 2.0,ICS 1.2.105,Iltrovatore-Setaccio,IlTrovatore-Setaccio 0.03,Iltrovatore-Setaccio 0.3,IlTrovatore-Setaccio 1.2,IlTrovatore-Setaccio 1.2,Imagefetch 0.1,Indy ibrary,InelaBot 0.2,InfoSeek Sidewinder 1.0A,Infoseek SideWinder 1.45,Infoseek SideWinder 2.0B,Ingrid 3.0,Inktomi slurp,InternetLinkAgent 3.1,IPiumBot,IRLbot 1.0,Istarthere,Java 1.4.0,JoBo 1.3,K2spider,KMcrawler,Knowledge 0.2,Knowledge 0.3,Knowledge Engine,Kuloko-bot 0.2,Larbin,Larbin 2.6.2,Larbin 2.6.3,Larbin 2.63,Libwww-MGET 1.0,Libwww-perl,Libwww-perl 5.3.2,Libwww-perl 5.4.0,Libwww-perl 5.48,Libwww-perl 5.50,Libwww-perl 5.51,Libwww-perl 5.52,Libwww-perl 5.53,Libwww-perl 5.63,Libwww-perl 5.64,Libwww-perl 5.65,Libwww-perl 5.66,Libwww-perl 5.68,Libwww-perl 5.69,Libwww-perl 5.74,Libwww-perl 5.75,Libwww-perl 5.76,Libwww-perl 5.800,Libwww-perl 5.801,Libwww-perl 5.802,Libwww-perl 5.803,LimeBot 1.0,Linkbot 3.0,LinkLint-checkonly 2.3.5,Linknzbot,Linknzbot 2004,Links SQL,Lite Bot 0616B,LNSpiderguy,Look,Lwp-request 1.29,Lwp-request 1.35,Lwp-request 1.36,Lwp-request 2.01,Lwp-request 5.48,Lwp-request 5.65,Lycos_Spider,Mediapartners-Google 2.1,Mercator 2.0,Metacarta,MetaGer-LinkChecker,Microsoft URL Control,Microsoft-ATL-Native 7.00,MicrosoftPrototypeCrawler,Moget 1.0,Moget 2.1,MozDex 0.04,MozDex 0.05,Mozilla,MS Data Access,MS FrontPage 4.0,MS FrontPage 5.0,MSIECrawler,Msnbot 0.1,Msnbot 0.11,Msnbot 0.3,Msnbot 1.0,MSProxy 2.0,MSRBot 0.1,MysteryBot,NaverBot 1.0,NaverBot_dloader 1.5,NaverRobot 1.0,NaverRobot 1.5,NetAnts 1.25,NetNoseCrawler 1.0,NetNoseCrawler 2.0,NetResearchServer,NetResearchServer 2.4,NetResearchServer 2.5,NetResearchServer 2.7,NetResearchServer 2.8,NetResearchServer 2.9,NetResearchServer 3.4,NextGenSearchBot 1,NG 1.0,None,NPBot,NuSearch Spider,NutchBot 0.05,NutchBot 0.06,NutchCrawler,NutchOrg 0.03,Obot,Ocelli 1.3,OmniExplorer_Bot 1.07,Openbot 3.0,OrangeBot,Organica 2.24,Overture-WebCrawler 3.8,OWR_Crawler 0.1,Parabot,Patwebbot,Pavuk 0.9,PEERbot,PipeLiner 0.3,Planethosting,Polybot 1.0,Pompos 1.1,Pompos 1.2,Pompos 1.3,Portal Manager 0.7,Potbot 1.0,Program Shareware 1.0.3,ProWebGuide Link Checker,Psbot 0.1,Pverify 1.2,PWS.Kiosk,QPCreep Test Rig,QuepasaCreep,QuepasaCreep 0.9.14,QuepasaCreep v0.9.13,Reifier,Rico 0.1,RixBot,RoboPal,RobotMidareru 0.7,Robots.txt ,alidator,Robozilla 1.0,RPT-HTTPClient 0.3,SafariBookmarkChecker 1.25,SafariBookmarkChecker 1.26,Scooter 1.0,Scooter 1.1,Scooter 3.0,Scooter 3.2,Scooter 3.3,Scrubby 2.1,Scrubby 2.2,Search Agent 1.0,SearchSpider 1.1,Seekbot 1.0,Semanticdiscovery 0.1,Sensis.au Web Crawler,Sherlock 1.3,Sherlock_spider,Sitecheck,Sitescooper 3.1.2,SiteXpert,SlySearch 1.3,Sohu-search,Speedy Spider,SpeedySpider,SpiderKU 0.9,SpiderMonkey 7.04,SpiderMonkey ,.06,SpurlBot 0.2,Sqworm 2.9.85,Star Downloader,Steeler 1.3,SuperCleaner 2.56,SYCLIKControl LinkChecker,Szukacz 1.5,Tarantula,TclSOAP 1.0,TclSOAP 2.3,Teradex ,apper,Teradex_Crawler,TheSuBot 0.1,TheSuBot Beta,Thumbshots-de-Bot,TimboBot 0.9,Tkensaku 0.9,TranSGeniKBot,TranSGeniKBot www.tsgk.net,TulipChain 5.7,TulipChain 5.94,TulipChain 6.01,TulipChain 6.02,TulipChain 6.03,TurnitinBot 1.4,TurnitinBot 1.5,TurnitinBot 2.0,TutorGigBot 1.5,Tutorial Crawler 1.4,UdmSearch 3.1.20,UIowaCrawler 1.0,UIowaCrawler 2.0,Unchaos_crawler 2.0.2,unknown,Updated 0.1,USyd-NLP-Spider,Vagabondo 2.0,Vagabondo 2.1,Vivante Link ,hecker,Void-bot 0.1,VoilaBot 1.6,VSE 1.0,Vspider,W3C_Validator 1.183,W3C_Validator 1.305.2.109,W3C_Validator 1.305.2.12,W3C_Validator 1.305.2.137,W3C_Validator 1.305.2.148,W3C-checklink 2.90,W3C-checklink 3.6.2.3,W3C-checklink 3.9.2,W3C-checklink 4.0,W3C-checklink 4.1,Webbot,Webclipping,Webcollage 1.102,Webcollage 1.104,Webcollage 1.87,Webcollage 1.93,Webcollage 1.94,WebcraftBoot,WebcraftBoot 1.4.1,WebcraftBoot 1.4.2,WebFilter Robot 1.0,Webinator-indexer 2.56,WebRACE 1.1,WebSauger 1.20,WebSearch,WebSearch 2.0.1,WebSearch 3.0.1,WebSearch 3.1.3,WebSearch 3.1.6,Webwombat,Webyield robot,Wget 1.5.2,Wget 1.5.3,Wget 1.5.3.1,Wget 1.6,Wget 1.7,Wget 1.8,Wget 1.8.1,Wget 1.8.2,Wget 1.9,Wget 1.9.1,Willow 2.1,Wotbox ,lpha0.5.1,Www.ciml.co.uk,WWWeasel 1.00,Xenu Link Sleuth 1.1a,Xenu Link Sleuth 1.2b,Xenu Link Sleuth 1.2d,Xenu Link Sleuth 1.2e,Xenu Link Sleuth 1.2f,Yahoo! ,lurp,Yahoo-MMCrawler,YottaCars_Bot 4.12,Zao 0.1,Zao 0.2,Zao-Crawler,Zeus 3140,Zeus 57657,ZipppBot 0.11,ZoomSpider,ZyBorg 1.0