Skip to content

[BUG] Node stats failing with error illegal_argument_exception #19464

@himkak

Description

@himkak

Describe the bug

On creating the warm index and or indexing docs, if we check the node stats, we get the failure response :

{ "error": { "root_cause": [{ "type": "illegal_argument_exception", "reason": "Values less than -1 bytes are not supported: -2084b" } ], "type": "illegal_argument_exception", "reason": "Values less than -1 bytes are not supported: -2084b", "suppressed": [{ "type": "illegal_state_exception", "reason": "Failed to close the XContentBuilder", "caused_by": { "type": "i_o_exception", "reason": "Unclosed object or array found" } } ] }, "status": 400 }

With stacktrace:

java.lang.IllegalArgumentException: Values less than -1 bytes are not supported: -2576b at org.opensearch.core.common.unit.ByteSizeValue.<init>(ByteSizeValue.java:78) ~[classes/:?] at org.opensearch.core.common.unit.ByteSizeValue.<init>(ByteSizeValue.java:73) ~[classes/:?] at org.opensearch.index.store.remote.filecache.FileCacheStats.toXContent(FileCacheStats.java:148) ~[classes/:?] at org.opensearch.index.store.remote.filecache.AggregateFileCacheStats.toXContent(AggregateFileCacheStats.java:140) ~[classes/:?] at org.opensearch.action.admin.cluster.node.stats.NodeStats.toXContent(NodeStats.java:629) ~[classes/:?] at org.opensearch.action.admin.cluster.node.stats.NodesStatsResponse.toXContent(NodesStatsResponse.java:80) ~[classes/:?] at org.opensearch.rest.action.RestActions.nodesResponse(RestActions.java:223) ~[classes/:?] at org.opensearch.rest.action.RestActions$NodesResponseRestListener.buildResponse(RestActions.java:276) ~[classes/:?] at org.opensearch.rest.action.RestActions$NodesResponseRestListener.buildResponse(RestActions.java:267) ~[classes/:?] at org.opensearch.rest.action.RestBuilderListener.buildResponse(RestBuilderListener.java:53) ~[classes/:?] at org.opensearch.rest.action.RestResponseListener.processResponse(RestResponseListener.java:52) ~[classes/:?] at org.opensearch.rest.action.RestActionListener.onResponse(RestActionListener.java:64) ~[classes/:?] at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:115) ~[classes/:?] at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:109) ~[classes/:?] at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[classes/:?] at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[classes/:?] at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[classes/:?] at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:341) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.finishHim(TransportNodesAction.java:319) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.onOperation(TransportNodesAction.java:304) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:281) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:273) ~[classes/:?] at org.opensearch.transport.TransportService$9.handleResponse(TransportService.java:1815) ~[classes/:?] at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1587) ~[classes/:?] at org.opensearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:1680) ~[classes/:?] at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1660) ~[classes/:?] at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:72) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:332) ~[classes/:?] at org.opensearch.wlm.WorkloadManagementTransportInterceptor$RequestHandler.messageReceived(WorkloadManagementTransportInterceptor.java:63) ~[classes/:?] at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:108) ~[classes/:?] at org.opensearch.transport.TransportService$7.doRun(TransportService.java:1140) ~[classes/:?] at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975) ~[classes/:?] at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[classes/:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] Suppressed: java.lang.IllegalStateException: Failed to close the XContentBuilder at org.opensearch.core.xcontent.XContentBuilder.close(XContentBuilder.java:1047) ~[classes/:?] at org.opensearch.rest.action.RestBuilderListener.buildResponse(RestBuilderListener.java:52) ~[classes/:?] at org.opensearch.rest.action.RestResponseListener.processResponse(RestResponseListener.java:52) ~[classes/:?] at org.opensearch.rest.action.RestActionListener.onResponse(RestActionListener.java:64) ~[classes/:?] at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:115) ~[classes/:?] at org.opensearch.action.support.TransportAction$1.onResponse(TransportAction.java:109) ~[classes/:?] at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[classes/:?] at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[classes/:?] at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[classes/:?] at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:341) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.finishHim(TransportNodesAction.java:319) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.onOperation(TransportNodesAction.java:304) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:281) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:273) ~[classes/:?] at org.opensearch.transport.TransportService$9.handleResponse(TransportService.java:1815) ~[classes/:?] at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1587) ~[classes/:?] at org.opensearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:1680) ~[classes/:?] at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1660) ~[classes/:?] at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:72) ~[classes/:?] at org.opensearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:332) ~[classes/:?] at org.opensearch.wlm.WorkloadManagementTransportInterceptor$RequestHandler.messageReceived(WorkloadManagementTransportInterceptor.java:63) ~[classes/:?] at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:108) ~[classes/:?] at org.opensearch.transport.TransportService$7.doRun(TransportService.java:1140) ~[classes/:?] at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:975) ~[classes/:?] at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[classes/:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] Caused by: java.io.IOException: Unclosed object or array found at org.opensearch.common.xcontent.json.JsonXContentGenerator.close(JsonXContentGenerator.java:494) ~[classes/:?] at org.opensearch.core.xcontent.XContentBuilder.close(XContentBuilder.java:1045) ~[classes/:?] ... 27 more

Related component

No response

To Reproduce

  1. Create Warm Index
  2. Index a doc
  3. Check the node stats using API : _nodes/stats
  4. We get the error response

Expected behavior

Node stats API should work even if search query not executed.

Additional Details

The issue is not related to any plugin

Metadata

Metadata

Assignees

No one assigned

    Labels

    IndexingIndexing, Bulk Indexing and anything related to indexingStorage:RemotebugSomething isn't working

    Type

    No type

    Projects

    Status

    🆕 New

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions