Script Pulse

Trade Pulse => Trade Pulse Support => Topic started by: anthony on April 05, 2011, 07:10:58 PM

Title: Blocking bots
Post by: anthony on April 05, 2011, 07:10:58 PM

hi, after fixing some errors i had on my site using Google Analytics, I started to recieve thousands of hits in my "noref_in". 6576 hits (sofar)from "http://www.google.com/bot.html" and it's messing up my stats.

Question: Should I block these? and if I do, will my pages stop showing up in google search?

thanks,
anthony

Title: Re: Blocking bots
Post by: ip0li on April 06, 2011, 04:23:16 AM

Imho its not good idea to stop it since its google bot. Just ignore it. It should stop within cpl days. U can also see trades productivity (overall) and for each system trade so you are good I believe.

Title: Re: Blocking bots
Post by: anthony on April 06, 2011, 07:32:48 AM

will do and thanks...

Title: Re: Blocking bots
Post by: anthony on April 14, 2011, 10:23:00 AM

Help---

Googlebot is STILL after a week, bagging the crap out of my site. this is the information from the last 24hours,

"noref_in"-----Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)      6093 (94.79 %)

Help, what should I do, it is really messing up my numbers.

thanks,
anthony

Title: Re: Blocking bots
Post by: bbhrucy79 on April 14, 2011, 11:38:16 AM

Do you have anything in your header that may attract big G? Like, meta tags of 'revisit in one day', robot attractors like 'index. follow' etc. or maybe incorrect settings in you .htaccess?

For example if you don't have expiry header in your .htaccess I believe you get more bots.  ::)

Title: Re: Blocking bots
Post by: bbhrucy79 on April 14, 2011, 11:45:48 AM

So if you are leaking out bad traffic like this to your trades, will they consider you 'cheater'?

I do not worry about the numbers and stats (bots mess up my stats big time as well) but I worry about this crap forwarded to my trades - not sure, but do trade scripts look at this traffic as 'cheating/bad traffic' or do they generally recognize that these are bots and do not 'punish' you for sending them?

As for stats, just create group named 'human traffic' and put inside all your trades and SE and bookmark traffic, and in this way you can check what is going on in terms of actual human traffic using 'history'.

Title: Re: Blocking bots
Post by: anthony on April 14, 2011, 06:21:38 PM

yes i have

<META NAME="robot" CONTENT="index,follow"> on my index page and a robots.txt file that has the following lines in it.

User-agent: *
Disallow: /photopost/
Disallow: /tp/
Disallow: /click.php?
Allow: /

If you remove these will google index your website? i don't want to lose the little traffic i get from google..

Title: Re: Blocking bots
Post by: bbhrucy79 on April 14, 2011, 11:58:00 PM

you should remove <META NAME="robot" CONTENT="index,follow">

you wont loose traffic

have good SEO if you want seo traffic, but tags like above, and especially meta tag 'revisit in one day' etc, can hurt your site. Google marks your site as spam if you have the 'revisit in one day' meta tag, I experienced it.

your robots txt looks fine

hth

Title: Re: Blocking bots
Post by: anthony on April 15, 2011, 10:02:10 AM

thanks............