Icy Phoenix

     
 


Post new topic  Reply to topic 
Page 1 of 1
 
 
Reply with quote Download Post 
Post SEO - Robots list 
 
This topic is for help ban of spam and un-wanted robots. use this with this other topic: (Mail Spammers list and spam users)

As some people don´t want their sites indexed by any search robot/engine, I started this topic for reporting search robots.
There are some robots that ignore the disallow or go to that directories. Also there are ons that recopilate the "mailto:" HTML´s lines and send spam to that adresses.

Please, use this format for the robot reporting:

Good/Bad robot:

Ip/s that the robot uses:
The user-agent the robot gives:
Why it is good or bad:
Other data:
 


I don´t mind if cells are not used, but use the horizontal lines if you´re going to report various engines in same post.
 



 
Last edited by novice programmer on Tue 16 Oct, 2007 18:22; edited 2 times in total 
novice programmerSend private message  
Back to topPage bottom
Icy Phoenix is an open source project, you can show your appreciation and support future development by donating to the project.

Support us
 
Reply with quote Download Post 
Post Re: Robots List 
 
Now, the list:

Spoiler: [ Show ]


As this list is so high, I give the lists for download.
Last file in next of my posts in this topic.

Source:
http://www.kloth.net

Last file download:
Download from icyphoenix.com

2004.rar
Description: The existent robots in 2004 
Download
Filename: 2004.rar
Filesize: 8.48 KB
Downloaded: 619 Time(s)
2005.rar
Description: The robots for 2005 
Download
Filename: 2005.rar
Filesize: 8.6 KB
Downloaded: 605 Time(s)
actual.rar
Description: The actual robots 
Download
Filename: actual.rar
Filesize: 7.99 KB
Downloaded: 660 Time(s)

 



 
Edited by novice programmer, Wed 17 Oct, 2007 20:24: Continued adding robots...
Edited by novice programmer, Mon 05 Nov, 2007 21:13: Added download files
novice programmerSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
thanks
 



 
jzSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots list 
 
Nice novice programmer
 




____________
ThE KuKa - www.phpBB-Es.COM - Custom Installations phpBB
 
ThE KuKaSend private messageVisit poster's website  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
Thanks for the info.
 



 
ChYvISend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots list 
 
Wow, thanks for this!  
 




____________
Luca
SEARCH is the quickest way to get support.
Icy Phoenix ColorizeIt - CustomIcy - HON
 
Mighty GorgonSend private messageSend e-mail to userVisit poster's website  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
Thanks for the list.
 




____________
CivUnited
 
tonyf12Send private messageVisit poster's website  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
This is the last part

2003-2002.rar
Description: The robots list of year 2003 and 2002 
Download
Filename: 2003-2002.rar
Filesize: 7.6 KB
Downloaded: 1292 Time(s)

 



 
novice programmerSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
Thank you !!

very usefull info.

 
 



 
amakusaSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
Cheers mate. Hope this will provide with some indepth info on the topic.
 




____________
Mr.Wizzard
***********
Audaces Vincit
 
wizzardSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
Thanks, could do with something like this
 



 
SilentDefenderSend private messageVisit poster's website  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
You can add the robots there to your ctracker blocker by the http agent the requests use.

I am working around SQL queries.....
 



 
novice programmerSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
novice programmer wrote: [View Post]
You can add the robots there to your ctracker blocker by the http agent the requests use.

I am working around SQL queries.....


Thanks man! Nice Work!

 



 
EFNSend private message  
Back to topPage bottom
Reply with quote Download Post 
Post Re: Robots List 
 
This can be very useful, thanks for this  
 



 
stunnah83Send private messageVisit poster's website  
Back to topPage bottom
Reply with quote Download Post 
Post Re: SEO - Robots List 
 
Thank you so much for this list. I have been struggling from keeping spammers away.
 



 
kalkulatorkredytowySend private message  
Back to topPage bottom
Post new topic  Reply to topic  Page 1 of 1
 


Display posts from previous:    

HideWas this topic useful?

Link this topic
URL
BBCode
HTML

HideSimilar Topics





 
Permissions List
You cannot post new topics
You cannot reply to topics
You cannot edit your posts
You cannot delete your posts
You cannot vote in polls
You cannot attach files
You can download files
You cannot post calendar events