exp/api: Add experimental exp module; Add remote API with write client and handler. #1658
Conversation
bwplotka
force-pushed
the
prwclient-em
branch
4 times, most recently
from
17b12a6 to
1985d61
Compare
Signed-off-by: bwplotka <[email protected]>
bwplotka
force-pushed
the
prwclient-em
branch
from
1985d61 to
bffa922
Compare
|
Example use of handler in |
There was a problem hiding this comment.
Both API (client) and handler supports 1.0 and 2.0 Proto messages, but I intentionally only host writev2.Request generated Go code as we plan to deprecate 1.0 and users should use writev2 now. Both API and handler also will work against protobuf message types that use custom generators e.g. with gogoproto or anything else.
Given that the spec is only just finalized recently and (afaik) still no receivers like Mimir support v2, we should include v1 request code imo.
| return &handler{logger: logger, store: store} | ||
| } | ||
|
|
||
| func parseProtoMsg(contentType string) (WriteProtoFullName, error) { |
There was a problem hiding this comment.
Is this something we can expose?
| h.logger.Error("Error decompressing remote write request", "err", err.Error()) | ||
| http.Error(w, err.Error(), http.StatusBadRequest) | ||
| return | ||
| } |
There was a problem hiding this comment.
I wonder if we can expose a hook for the logic of decoding and decompressing part so that downstream project can just use their existing code without many adjustment.
func decodeRequestBody(r *http.Request) (body []byte, err error)
The main logic that can benefit us is handling v1 and v2 proto and response headers. It would be nice to still reuse the other code.
There was a problem hiding this comment.
Yup some requestValidatorDecoder interface that can be overridden downstream would be cool to have.
| h.logger.Error("Error decompressing remote write request", "err", err.Error()) | ||
| http.Error(w, err.Error(), http.StatusBadRequest) | ||
| return | ||
| } |
There was a problem hiding this comment.
Yup some requestValidatorDecoder interface that can be overridden downstream would be cool to have.
| return &handler{logger: logger, store: store} | ||
| } | ||
|
|
||
| func parseProtoMsg(contentType string) (WriteProtoFullName, error) { |
* Address remaining feedback Signed-off-by: Saswata Mukherjee <[email protected]> * Make Write message type more flexible Signed-off-by: Saswata Mukherjee <[email protected]> --------- Signed-off-by: Saswata Mukherjee <[email protected]>
* Move remote write API to client_golang/exp Signed-off-by: Saswata Mukherjee <[email protected]> * Don't use api.Client structs, add options for middleware Signed-off-by: Saswata Mukherjee <[email protected]> * Fix reqBuf usage Signed-off-by: Saswata Mukherjee <[email protected]> * Fix url path Signed-off-by: Saswata Mukherjee <[email protected]> * Add separate mod file (and workspace file) Signed-off-by: Saswata Mukherjee <[email protected]> * Hook exp tests fmt; Test handler error case; Configure backoff Signed-off-by: Saswata Mukherjee <[email protected]> --------- Signed-off-by: Saswata Mukherjee <[email protected]>
bwplotka
changed the title
Signed-off-by: Bartlomiej Plotka <[email protected]>
There was a problem hiding this comment.
Thanks @saswatamcode for helping a lot. Let's get those last things to this branch + quick README and let's get it merged in the experimental module!
* Implement suggestion for Store interface and contentType Signed-off-by: Saswata Mukherjee <[email protected]> * Add README Signed-off-by: Saswata Mukherjee <[email protected]> * Use sync.Pool Signed-off-by: Saswata Mukherjee <[email protected]> * Implement review suggestions Signed-off-by: Saswata Mukherjee <[email protected]> * Release bufs right after compressPayload Signed-off-by: Saswata Mukherjee <[email protected]> --------- Signed-off-by: Saswata Mukherjee <[email protected]>
|
@saswatamcode addressed most if not all comments and we should be good for an experimental module here 💪🏽 @saswatamcode do you mind checking last lint issues? You can push straight to the branch if you want |
Signed-off-by: Saswata Mukherjee <[email protected]>
This proposes Remote Write API directly in client_golang main module. No dependency added (some new small packages are added under the internal vendored code).
Old attempt: #1656
Both API (client) and handler supports 1.0 and 2.0 Proto messages, but I intentionally only host
writev2.Requestgenerated Go code as we plan to deprecate 1.0 and users should usewritev2now. Both API and handler also will work against protobuf message types that use custom generators e.g. with gogoproto or anything else.TODO: