Using aws-es-proxy in conjunction with elasticdump

[puzzledvacuum]

I have multiple projects that use the proxy to gain access to es clusters within aws (moving millions of documents per week) and after bumping the version to either v1.0 or v1.1 the upload stage fails with message:

• Client.Timeout exceeded while awaiting headers

The only thing that helps is rolling back to version 0.9 of the proxy.

[abutaha]

That's an interesting issue. In v1.0+ the proxy is using a customized HTTP client while in v0.9 it was the default client. This change was required to avoid 301 redirections which were introduced in recent versions of Kibana among other additions like HTTP basic authentication.

However, the v1.0+ has a '-timeout' argument which can be used to increase (default timeout is 15s). Would you please give this a try and share your results? Additionally, any extra information you share should help.

Thanks,

[bva]

@abutaha I'm too getting Client.Timeout with 1.1 version of proxy time after time. Restart of program helps, but after 2-3 weeks of working issue repeats. I tried to raise timeout to 4 minutes (should be enough to collect cluster statistics, I use proxy for Prometheus exporter), with no visible difference, issue persists. Attached excerpt from proxy logs.

es_log.txt