[closed] Bot detection with error emails

#1

While running FarCry 7.0.6, I’ve received several error emails caused by a googlebot visit but having “(not a bot)” in the subject. In fact the only error email received with “(bot)” in the subject was when the user agent was empty. Is there a configuration option I need to set or is this a bona fide bug?

Cheers!

P.S. Sorry for the inactivity recently. Been very busy :tired_face:

#2

I’m not sure why bot detection would be failing but I can point you in the right direction if you want to debug it.

The bot detection will be checked on each request, here:

If “googlebot” is present in the user agent string then it should be detected as a bot, as it’s in the default list of bot user agents (just above on line 44).

It’s possible to turn off bot detection by placing this.botDetection = false in your farcryConstructor.cfm, but I’m assuming that’s not the case in your app.

It seems that the user agent match is case sensitive, although the user agent string is passed to lcase() when reFindAny() is called so that shouldn’t be it:

Please let us know what you find :smile:

1 Like
#3

Oh and specifically related to the data present in an error, this is where the bot detection is checked and turned into the string “bot” or “not a bot” (also worth noting, we shouldn’t be using IIF() and DE() in core):

1 Like
#4

Thanks for the pointers. I’ll do some digging, see if I can find anything and share anything worth sharing.

I couldn’t find this.botDetection in the farcryConstructor.cfm, but I did find <cfparam name="this.botAgents" default="*:+bingbot" /> in Application.cfm. If I understand __plusMinusStateMachine correctly, this should add bingbot to the default list.

Here’s the googlebot user agent in question: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Initial investigation indicates that the bot agents array is empty for some reason …

#5

Yes I also read "*:+bingbot" as adding bingbot to the array. Interesting to note that bingbot is missing from core though, we should definitely add it :smile:

Is that an Application.cfm in your project folder? I’d be surprised if that was still having any effect, FarCry Core has used Application.cfc for quite a few major versions now. If it’s specified in your projects Application.cfc then perhaps that is the problem, the line of code might need to be moved into the farcryConstructor.cfm instead?

#6

It is in Application.cfc which extends="farcry.core.Application". Does this not work?

I’ve found the issue - minor bug in core Application. Just tring to test before I create a pull request.

#7

Bug fixed - I hope! :worried:

1 Like
#8

I think the preferred location to configure it is in your farcryConstructor.cfm:

<cfset this.botAgents = "*:+bingbot">

Your pull request is spot on though, seems like it’s been this way since mid-2010:

Thanks for the fix :slight_smile: It’s merged, and I’ll backport it to the p700 branch as well.

#9

Thanks! Got a couple of questions about this - where’s the best place to put them?

Obviously just me that cares then :smile:

My pleasure. :grinning: I’m just sorry I have been and am still too busy to do much else recently.

As always, I am open to constructive criticism

#10

Anywhere in your projects farcryConstructor.cfm should be fine, e.g. just after the data source and plugin configuration…

<cfset this.botAgents = "*:+bingbot">

Also, I’ve just added bingbot to the default bot list for p700 and p710:

#11

:laughing: My mistake - I meant where is the best place to put the questions, not that line of code.

Nice! That list may need an overhaul but not now. If I get time, I may have a bash.

#12

Ahh, questions about the farcryConstructor.cfm? Perhaps in a new thread here :smile:

A review of current / commonly used bot user agents would be a good idea.

This also needed another patch, now that this.botIPs is an array and not a string:

1 Like