River in replica set and shard not updating index at all times

[axelfran]

Hi,

In my production environment I have a setup with a mongoDB replica set running on three nodes, and an ElasticSearch cluster running on two (the secondary and primary in the mongoDB replica set).

MongoDB: 2.4.12
Elasticsearch: 1.4.2
River: 2.0.5 (EDIT)

The river is working fine for creating and updating items in the indexed collections, but from time to time it doesn't notice that changes have been made to an item in the database. There doesn't seem to be any system to what trigger it to update in the ElasticSearch index or not. I have even tried to invoke the update directly on the index (using elasticsearch npm module, as I use nodejs), but even this doesn't work all the time.

Have anyone seen a situation like this before, where items in a collection connected to ElasticSearch with a river only are updated sporadically? I have been scratching my head for weeks, and am really grateful for your input, or if you have tips to other solutions for a nice integration with ES and mongoDB.

/Axel

[ankon]

2.0.1 is really quite old, there have been many fixes to similar situations in later versions.

Can you post log ES (and possibly MongoDB) logs from the time around such a missed update?

[axelfran]

Thanks for answering, @ankon! Actually, I'm using river version 2.0.5.

I've attached the latest logs from MongoDB and ES. As you can see I just updated to mongoDB 2.6.6 and have been turning the system on and off quite a few times lately. They didn't tell me much, but maybe you can see something I don't?

Do you thing it may have anything to do with me using sharding in elasticsearch?
uploads.zip

Thank you so much!

[pmjhonwang]

Hi, @ankon
may I have a question aboout which version of es and mongoriver is recommended
I trying to use mongoriver on production envrionment
Great thanks

[EdgeCaseBerg]

I had to troubleshoot something that sounds very similar to this, it turned out that the clocks on my servers running the replica set were all off from each other by a few seconds each. Resyncing the clocks seems to have fixed the problem. Hope that helps anyone else troubleshooting this type of thing