content API endpoint

[vladak]

Is your feature request related to a problem? Please describe.
#3042 listed a few new API endpoints that would be useful to have. This tracks the addition of the /content endpoint for getting content of a file.

Describe the solution you'd like
Return JSON array of objects:

• line number
• line content (verbatim, i.e. no xref)

The endpoint should support a parameter to specify revision of the file. It may support paging.

[vladak]

The above assumes text files that contain newlines. Perhaps the controller could verify the type of the file (AnalyzerGuru#getAnalyzerFor(file, path) comes to mind and check if returned object is instanceOf TextAnalyzer) and return empty object in case of binary files - these should be possible to get via the /download link (not an API endpoint).

[vladak]

After implementing the above (without pagination) I have to say it is not clear to me how beneficial is to have this endpoint. Anyone (authorized, if authorization config is in place) can download the content of any file, split it into lines and assign them line numbers. It sort of makes sense when combined with the text file detection (as suggested in the above comment) and maybe if pagination (for long files) is supported. @jbaek7023 ?

I was also thinking about exposing AnalyzerGuru#getGenre() to output the genre for given file (need to use it with InputStream otherwise it merely matches path to known prefixes/suffixes). There could be both /file/content and /file/genre endpoints.

[jbaek7023]

I'm trying to implement this Front-End talking to OpenGrok API (Inspired by SourceGraph). Annotation and History can be shown by the toggling "Annotate" or "clicking CL" button.
But as you see, I still need to get the content (line and its content and file structure).

I think your implementation will be super useful in the future for anyone who wants to OpenGrok indexing engine + I don't think we'll need a pagination here.

Thanks again for the quick turnaround...!! Your impact is big.

[idodeclare]

The above assumes text files that contain newlines. Perhaps the controller could verify the type of the file (AnalyzerGuru#getAnalyzerFor(file, path) comes to mind and check if returned object is instanceOf TextAnalyzer) and return empty object in case of binary files - these should be possible to get via the /download link (not an API endpoint).

T field is defined in a Lucene doc for Genre for non-binary input. Only Plain would be applicable to this issue I think. No re-analysis from source text would be needed or desired I think

[jbaek7023]

@idodeclare No, all the DOM tags are all valuable. I just want to override the style.css files. Which makes the analysis from source text will be necessary - I still want to keep the DOM tags () in the JSON data.

(Apologies in advance if this is a dumb thought)

[idodeclare]

@jbaek7023 no I was referring to @vladak mentioning calling AnalyzerGuru#getAnalyzerFor(file, path) to determine what files are plain-text. No need since OpenGrok would have saved into Lucene a T field of PLAIN when applicable.

[vladak]

thanks @idodeclare , that's what I needed.