Closed
Description
Hey,
My Opengrok version is 1-5-11
When I try to search for definitions I got each appears on the file the variable is defined there, for example,
https://opengrok.checkpoint.com:8443/source/api/v1/search?def=ws_http2_stream_error_action&projects=ivory_main
This is what I get, it should return only line 105, on UI it works and jumps me direct to line 105
{
"time": 11,
"resultCount": 1,
"startDocument": 0,
"endDocument": 0,
"results": {
"/ivory_main/ws/http2/ws_http2_parser.c": [
{
"line": " * fw ctl set int <b>ws_http2_stream_error_action</b> 0/1",
"lineNumber": "103"
},
{
"line": "int <b>ws_http2_stream_error_action</b> = E_WS_HTTP2_STREAM_ERROR_HANDLE;",
"lineNumber": "105"
},
{
"line": "\t\tswitch(<b>ws_http2_stream_error_action</b>)",
"lineNumber": "5009"
},
{
"line": "\t\t\t\tWS_DEBUG(WS_HTTP2, WS_ERROR, (\"invalid stream error action %d\", <b>ws_http2_stream_error_action</b>));",
"lineNumber": "5035"
}
]
}
}
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
vladak commentedon Oct 13, 2021
I assume that the search in the UI is performed by using the 'Definition' field in the search form, correct ?
vladak commentedon Oct 13, 2021
Tried that with search for the
USERNAME_FIELD
definition in OpenGrok source code. Using the search API returns 5 hits inplugins/src/main/java/opengrok/auth/plugin/UserWhiteListPlugin.java
which is where the actual definition is located. The API search results contain just this file even though there are some references to this symbol also inplugins/src/test/java/opengrok/auth/plugin/UserWhiteListPluginTest.java
, otherwise I'd say this is wrongly formed query inside theSearchController
somewhere, however this looks more like a problem with interpreting the search results.honour definition search from API when getting the hits
vladak commentedon Oct 13, 2021
Getting the results of search to the client is actually pretty complex.
There are major differences in how the search is done in the UI and via the API. I will try to summarize it here for future reference since I don't know whole lot about it myself.
When going via the API, it starts in
SearchController#search()
. This usesSearchEngineWrapper
, a private static class located in the same source code file. As suggested by the name, this class is a wrapper to theSearchEngine
class. Thesearch()
method then callsengine.search()
(whereengine
is instance ofSearchEngineWrapper
) which returns the list ofHit
objects which are then wrapped inside theSearchResult
object and this gets converted automatically to the JSON representation returned to the client.Now, for the context (sic!) of this bug,
SearchEngineWrapper#search()
callsSearchEngine#search()
to get the search results and then callsSearchEngine#results()
to convert the search results to the list ofHit
objects. TheSearchEngine
object keeps the list of hits in an array ofScoreDoc
(Lucene) objects. TheSearchEngine#results()
traverses this array and in this particular case uses theContext#getContext()
that fills the list ofHit
objects:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/SearchEngine.java
Lines 467 to 476 in 33eb38a
This method then uses
PlainLineTokenizer
:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/context/Context.java
Line 364 in 33eb38a
Hit
objects to the list for the non-definition tokens indumpRest()
:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/context/Context.java
Lines 404 to 406 in 33eb38a
opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/context/Context.java
Lines 420 to 424 in 33eb38a
opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/context/Context.java
Line 435 in 33eb38a
vladak commentedon Oct 13, 2021
Searching from the UI is a tad different as it uses different search wraper:
opengrok/opengrok-web/src/main/webapp/search.jsp
Lines 81 to 86 in 701f162
It builds
IndexSearcher
inprepareExec()
and callssearch()
inexecuteQuery
- the approach is similar to whatSearchEngine
does for the API based search.The results are then presented via
opengrok/opengrok-web/src/main/webapp/search.jsp
Line 228 in 701f162
Results#printPlain()
. Similarly toSearchEngine
used by the API, theSearchHelper
usesContext
(at least in our case of plaintext) however different method:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/Results.java
Lines 265 to 270 in 701f162
OGKUnifiedHighlighter
that extends Lucene'sUnifiedHighlighter
to display the results. It uses the query to filter the results, I think.So, while
SearchEngine
(used by API) andSearchHelper
(used by UI) both use theContext
, they use radically different pieces of it in the generic case. There is one interesting tidbit whereprintPlain()
falls back to usinggetContext()
aftergetContext2()
indicated no matching context:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/Results.java
Lines 274 to 279 in 701f162
isDefSearch
properly:opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/search/Results.java
Lines 292 to 299 in 701f162
vladak commentedon Oct 13, 2021
The use of
Context
to present the results from the/search
API call likely leads to problems such as #2612, #3090, #3170.vladak commentedon Oct 13, 2021
Looking at #2732, this bug could be fixed if we merged that PR since it replaces the offending
sourceContext.getContext()
call with something else, however it might as well reintroduce the bug - need to check this.honour definition search from API when getting the hits