Script Pulse

Trade Pulse => Trade Pulse Feature Requests => Topic started by: alex on June 28, 2009, 01:38:05 AM

Title: Fliter SE Bots by useragents - you really missed that
Post by: alex on June 28, 2009, 01:38:05 AM: You forgot about se bots simulating surfers, they all going to noref and sometimes it generates a lot of noice in stats.
Those bots counted as noref and who knows how the hell they counted else when clicking.
There are a little string below from one of tiny sites.

Quote
Mozilla/4.0 (compatible; MSIE 7.0b;
Windows NT 6.0)   1458 (26.66 %)
http://www.cuil.com/twiceler/robot.html)   1022 (18.69 %)
Mozilla/5.0 (compatible; DotBot/1.1;
http://www.dotnetdotcom.org/,
crawler@dotnetdotcom.org)   447 (8.17 %)
Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)   112 (2.05 %)

sometimes it 4-5k with Googlebot/2.1 only.
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: ip0li on June 28, 2009, 07:58:27 AM: why would we filter it? Why is it wrong if they are shown there?
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: alex on June 29, 2009, 02:00:26 AM: - It drops total stats - productivity and everything. I even getting fucking alerts from day by day because of that.
- cant see real ppl coming with noref.
- what else?
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: ip0li on June 29, 2009, 06:18:36 AM: ok I will speak with kildoozer when he is back from vacation. added to the 2do list
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: JuicyAsianRider on September 13, 2009, 02:53:08 PM: yes its really good idea if bots were shown separate but NOT included in the stats total
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: spudman on October 17, 2009, 05:15:26 AM: did anything ever come of this?

i get so many googlebot hits and it does mess up stats, prod etc. what the best thing to do with this?

thanks
Title: Re: Fliter SE Bots by useragents - you really missed that
Post by: ip0li on October 17, 2009, 07:16:06 AM: we will do it in future builds. at the moment we are finalizing our other project and after that is done we are back in 100% to release next updates!