[WIP] Search Engine (issue 535)

[osarrat]

No description provided.

[halfcurry]

Please ignore any local changes I have made. The only one which matters is this solr dependency.

[osarrat]

My two cents about this: instead of modifying the common pom.xml file, you should rather maintain your own Maven settings.xml file as very recently documented here: http://wiki.sigmah.org/doku.php?id=contributorguide:preparebuildenvironment#step_3your_first_build

Enjoy !

[osarrat]

Good frequency of small commits, keep with this habit ! I makes following and understanding easier !

Two general comments:

• please take a look at the Codacy report on your pull request to improve its quality
• your commits have actually broken Contribution rule number one : each commit must be linked to one and only one Mantis issue : http://wiki.sigmah.org/doku.php?id=contributorguide:contributionrules#make_your_work_easy_to_merge

Could you adjust your future work to improve those two issues ?

Good work so far ! Continue like that !

[halfcurry]

Apologies for forgetting to put the issue number each time! The codacy report mostly suggests that I clean up my unused imports. At present I am keeping them just in case I might need to import some stuff again, and I'll clean them up as soon as I'm sure I don't require them at all. And I'll soon modify my pom.xml file as well and remove local settings as you mentioned. Thank you!

[halfcurry]

In my latest commit ( 8dd81b4 ), most projects are opening out on clicking the corresponding search result, but some are facing permission issues. Need to investigate what are the permissions preventing those specific projects from opening out.

[halfcurry]

"Search Filtering" in the commit 049a3ee means the filtering of the query according to need i.e Projects, OrgUnits, Contacts. Note: First-time connection error is not yet resolved. It re-occurs and the temporary fix did not work.
"Filtering" in 9d15b0a means the filtering of the results based on the permissions the currently logged in user has.

[halfcurry]

In the commit dd095d6, I have added an auto index job which does a solr full data import. This job is initialized when the dashboard first loads ( this is a temporary solution just to test whether the job works or not ), and keeps firing every 10 minutes.

[halfcurry]

0604656 : Should be global 'solr' core url.

[halfcurry]

bec9e80: I have added a column in the table organization, hence added a migration sql file as mentioned here: http://wiki.sigmah.org/doku.php?id=contributorguide:contributionrules. I hope there is nothing more to change.

[halfcurry]

It seems there is a jdk version error, i.e I am compiling with JDK 1.8 and the builder is running tests with JDK 1.6 or 1.7 : https://stackoverflow.com/questions/23249331/java-unsupported-major-minor-version-52-0
That is the reason it is not able to pass the tests.

[halfcurry]

By "rudimentary" files results filtering, I mean that only those files which the currently logged in user is an author of are being shown in the search results. On clicking these results, the file successfully downloads. This is the best permission I am able to implement for now ( and is partially useful at the same time ), due to the complexity of the code. For future, a better permission has to be implemented.

[halfcurry]

Hi @osarrat @niksj , I need to write a brief description / gist here as part of the final evaluation. Can I get rights for the same? Currently I am unable to write a description.

[ghost]

Well, for this one i'm 50/50 about integration.

At the first sight, it seems that it's a good basis with not so much impacts, which is nice but on the other hand, while the development has been focused on building a viable POC (which is fine), non-working/limit cases seems not sufficiently considered (e.printStackTrace() instead of exception management). Also, I know that a big issue about search is about keeping the index and the database in sync (involving transaction management to only commit changes in the index if commit in the database has passed) and I know solr is very fast but sometimes corrupts its index, so being able to reset/clear the index and rebuild it (reindexing the whole database) should be available as a feature (and I didn't see it, but I only had a quick review).

Seems a good start with things in the right place through. I think a developer who would have to implement this feature should seriously consider continuing this PR, it may be faster than starting from nothing.

[osarrat]

Thanks @numero-six for this review !

So, if I understand you correctly, your reluctance mainly come from functional issues like a lack of reset button, or improvable error messages. But you consider this piece of work as valuable enough to start to have a search engine in Sigmah.
The good thing is that this search engine feature has been developed so that it can be easily enabled or disabled to some user profiles by a single global privilege. So, in my opinion, if we consider the risk to lose this big amount of work because too different from the master, I would recommend to merge this PR and informing users that it is still an early feature.
Does it seem a good trade-off in your opinion @numero-six ?