SOLR Pro not indexing until a resave

mpicker · February 3, 2016, 4:41am

Hi,

I’m having some trouble working out a problem with our FarCry 7 driven Intranet and SOLR Pro.

I’ve noticed that when creating new content for the site it won’t appear in the SOLR index unless :

You manually trigger a reindex of the content type from the admin page

OR

Edit the new content and resave

This is happening to all content types.

I’ve added some additional logging lines to /farcrysolrpro/packages/custom/eventHandler.cfc (afterSave function) and can see when resaving content that this function is being called.

No content type cfc’s have custom contentToIndex functions.

The lines I added to the cfc are firstly to tell what content type is being saved, another to show stProperties.objectid and then the if then else statement around line 80ish I have it logging if the item is going to be added to the index, removed from the index or if it is not able to be indexed.

When I create a new record what I’m seeing is:

"Information","ajp-bio-8012-exec-5","02/03/16","15:24:43","DEVELOP_WI_INTRANET","Saving farFU"
"Information","ajp-bio-8012-exec-5","02/03/16","15:24:43","DEVELOP_WI_INTRANET","Now checking record 0CB51AD0-CA2E-11E5-8AB0005056A670E2"
"Information","ajp-bio-8012-exec-5","02/03/16","15:24:43","DEVELOP_WI_INTRANET","Record was not indexable"

And nothing more. I believe that the above is correct as the FURL isn’t something that should get indexed. However what is missing from the above is the processing of the content. If I edit the content again and resave I get:

"Information","ajp-bio-8012-exec-5","02/03/16","15:34:08","DEVELOP_WI_INTRANET","Saving hbAnnouncement"
"Information","ajp-bio-8012-exec-5","02/03/16","15:34:08","DEVELOP_WI_INTRANET","Now checking record 02BD3990-CA2E-11E5-8AB0005056A670E2"
"Information","ajp-bio-8012-exec-5","02/03/16","15:34:11","DEVELOP_WI_INTRANET","Adding 02BD3990-CA2E-11E5-8AB0005056A670E2 record to the index"

Just for a test I set bFriendly=“false” on the content type but all this did was stop the farFU lines from appearing in the log – the hbAnnouncement content still wasn’t added to the index.

Hoping someone can shine some light on the issue?

Cheers
Mark

mpicker · February 3, 2016, 4:51am

I should have also noted that these content types don’t have a status field (so no draft/approved status). My understanding from the docs is that SOLRPro will automatically index content that doesn’t use the publishing model.

jeff · February 3, 2016, 8:08am

Correct

hmm… What is your setup? Version of FarCry 7, version of Solr Pro plugin, etc.

When you edit the content type for Solr Pro, can you confirm that you checked the checkbox for “Index on Save” (see screenshot below)

seancoyne · February 3, 2016, 2:43pm

Do you have a contentToIndex() function or are you doing anything out of the ordinary with the datetimelastupdated field? The indexing engine will run a contentToIndex() function to determine which records should be indexed and it also uses the date updated field to only index records that have changed.

EDIT: Sorry, just realized you said in your OP that you do not use a contentToIndex() function.

mpicker · February 3, 2016, 11:30pm

Hi Jeff,

It was FarCry 7.1.something but as a step in trying to fix this I upgraded to 7.2.4 (which obviously made no difference). SOLR Pro is 1.3.2.

Sorry, should have also mentioned yesterday that I had already made sure the “Index on Save” option was selected. Just double checked and yes it is (on all content types).

I realise this shouldn’t matter but what I might try shortly is to delete the type from SOLR Pro and recreate it to see if that makes any difference.

Cheers
Mark

mpicker · February 3, 2016, 11:34pm

Hi Sean,

No problem With regards to datetimelastupdated - no I don’t believe (I didn’t build this site - it was done by Daemon) anything special is being done with that field. I’ve just had a look in the code and can’t find anything special.

Looks like I need to keep looking.

Cheers
Mark

jeff · February 3, 2016, 11:42pm

What version of ColdFusion/Railo/Lucee?
Do you know if your project (or any other plugins besides fcsp) has a fcTypes file? (/packages/events/fcTypes.cfc)? If yes, do any of them have a “saved” function?
If you’ve made any changes to the type in fcsp, have you made sure that you “reset” the index? It shouldn’t be related to this issue if you have not, but just trying to think of other known problems that people sometimes forget about.

seancoyne · February 4, 2016, 3:58pm

I think Jeff may be onto it. My first guess would be that FC core is not firing the “saved” event for createData() but it is for setData() which would explain why it doesn’t work when the record is created, but it does when you save it again. It is possible that core is being prevented from firing the event due to another conflict (as Jeff pointed out) or perhaps there is a bug. I have a FC 7.2 site I can run a few tests against if I have a moment today.

jeff · February 4, 2016, 8:05pm

Those methods are also written in script which will only run on CF9+ which is the reason I asked what CFML engine and version you are running. Just trying to think of any possible reasons the saved event might not be running.

mpicker · February 5, 2016, 12:12am

Hi Sean and Jeff,

1.What version of ColdFusion/Railo/Lucee?

This is all on ColdFusion (Standard) version 10,0,11,285437

2.Do you know if your project (or any other plugins besides fcsp) has a fcTypes file?

There are only two fcTypes in the whole install, core and fcsp

3.If you’ve made any changes to the type in fcsp, have you made sure that you “reset” the index?

I have been doing that anyway - yesterday even deleted the fcsp type and recreated them but it made no difference.

We use the same SOLR install for all our projects and haven’t noticed any issues with the other projects. However I’m also not aware of any types used in other projects that don’t have the draft/approved publishing workflow. Therefore there could well be a bug in FC that is only happening on types without the workflow.

My next plan is to try and find time (as per usual I’ve got meetings all day) to do a completely fresh install of FC and FCSP onto a spare server and create a new dummy type and see what happens.

Cheers
Mark

mpicker · February 9, 2016, 12:13am

Hi Jeff and Sean,

This is now fixed after a minor change to fcTypes.cfc in the FCSP plugin. I ended up contacting Daemon to have a look at this and Justin found an issue on line 9 of plugins/farcrysolrpro/packages/events/fcTypes.cfc.

From Justin’s emails:

From looking at the SOLR Pro code there is a condition around the solrProEventHandler which could stop the content from being indexed:

github.com

jeffcoughlin/farcrysolrpro/blob/master/packages/events/fcTypes.cfc#L9


component {
	
	variables.solrProEventHandler = createObject("component","farcry.plugins.farcrysolrpro.packages.custom.eventHandler");


	public void function saved(required string typename, required any oType, required struct stProperties, required string user, required string auditNote, required boolean bSessionOnly, required boolean bAfterSave) {
		if (not structKeyExists(arguments.stProperties, "bDefaultObject")) {
			arguments.stProperties.bDefaultObject = false;
		}
		if (arguments.stProperties.bDefaultObject eq false and arguments.bSessionOnly eq false) {
			variables.solrProEventHandler.afterSave(stProperties = arguments.stProperties);
		}
	}


	public void function deleted(required string typename, required any oType, required struct stObject, required string user, required string auditNote) {
		variables.solrProEventHandler.onDelete(typename = arguments.typename, stObject = arguments.stObject);
	}


}

if (arguments.stProperties.bDefaultObject eq false and arguments.bAfterSave eq true) {

bDefaultObject will most likely be true for all new objects, but when this event handler runs for “createData” we know the record will have already been saved to the database.

I think the fix is to have this code check arguments.bSessionOnly not arguments.stProperties.bDefaultObject.

try:

if (arguments.bSessionOnly eq false and arguments.bAfterSave eq true) {

Making this change fixed the issue and new content is now being added to the index.

Thank you everyone for your help with this!

Regards
Mark

jeff · February 9, 2016, 12:15am

Thanks Mark. I’m glad it was tracked down.

mpicker · February 9, 2016, 12:18am

Hi Jeff,

Would you like me to raise an issue in Git for this?

jeff · February 9, 2016, 12:19am

Sure. We still need to test this to make sure it is doing what is expected and that it doesn’t break anything in FC 6. Not sure when I’ll have the time to check that.

seancoyne · February 9, 2016, 6:23pm

v1.4.0 released which fixes this issue. it also adds faceting support that Jeff recently added as well as a couple other bug fixes since the last release.