Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide route decorator similar to Grafana Beyla #1596

Open
titaneric opened this issue Sep 2, 2024 · 7 comments
Open

Provide route decorator similar to Grafana Beyla #1596

titaneric opened this issue Sep 2, 2024 · 7 comments
Labels
enhancement New feature or request

Comments

@titaneric
Copy link
Contributor

titaneric commented Sep 2, 2024

Request

Hi, I would like to request a new feature called route decorator similar to Grafana Beyla.
It works like this (in Beyla config)

routes:
  patterns:
    - /user/{id}
    - /user/{id}/basket/{product}

and it would match /user/123 and /user/456 and it would include /user/{id} in the trace property.

Hope that there is a URL parsing function or a dedicated block called matchRoute in log.process component to process the HTTP path like this.

Indeed, we have regex and replace block in log.process component. However, it's not powerful enough.

Use case

Like I mentioned in #1592, I would to pre-process the performance resource metrics sent by Faro SDK. There is a event_data field in the faro.receiver log and it would have the full path of HTTP request. I want to remove any parameters, hostname, and any GET single resource (/user/{id}) before sending to Grafana Loki. By configuring these, I could transform the logql query into Prometheus metrics by Loki rulers.

@titaneric titaneric added the enhancement New feature or request label Sep 2, 2024
@titaneric
Copy link
Contributor Author

I notice that alloy do have loki.template, and it support sprig URL parsing. I think it could meet my need, and I would close this issue.

@titaneric
Copy link
Contributor Author

Reopen here since I want to provide the following config to component or stage.

    - /user/{id}
    - /user/{id}/basket/{product}

@titaneric titaneric reopened this Sep 3, 2024
@ptodev
Copy link
Contributor

ptodev commented Sep 3, 2024

Hi, @titaneric! Do I understand correctly that stage.template can do the replacement you need, but you don't like the ergonomics? Is it because you need to replace more than one value (e.g. id and product), and you need to add multiple stage.template instances?

@titaneric
Copy link
Contributor Author

titaneric commented Sep 3, 2024

Hi @ptodev, thanks for response. Basically, yes, and I think present alloy component could not handle multiple URL paths easily. Multiple stage.replace could possible meet my need, but I have a list of decorated paths, and I think alloy could not dynamically generate the stage.replace blocks so I need a single component and give a list of decorated URLs to them.

I think stage.template or stage.replace could help me to extract the path from the full URL. However, I want more than that. I want to replace any path to given route decorator to minimize the possibility of high cardinality.

Given the URL https://example.com/user/123/wishList/456, and given imagined stage.route_decorator stage in loki.process components in alloy config.

loki.process "extract_path" {
  stage.replace {
    expression = "(?P<extracted_url>http\\S+)"
    replace = `{{  index (urlParse .Value) "path") }}`
  }
  stage.route_decorator {
    source= ""  
    decorator = [
      "/user/{id}",
      "/user/{id}/wishList/{product}"
    ]
  }
}

I hope this config could output "/user/{id}/wishList/{product} .

Hope that stage.route_decorator could replace the log line, and it could also replace the extracted values in loki.process component.
Perhaps a route_decorator function is another possible solution.

@titaneric
Copy link
Contributor Author

There is no further response here, could I volunteer to implement this feature and raise PR?

@ptodev
Copy link
Contributor

ptodev commented Oct 9, 2024

Hi @titaneric, apologies for the delayed response! IIUC, you'd like to replace the log line with a redacted URL? E.g. if the log is "a rq on /user/1234", then the output log will be simply "/user/{id}"? And then you'd like to query the number of "/user/{id}" logs, and generate a metric? I'm just not sure why you wouldn't simply use stage.metrics to generate the metric directly in alloy?

@titaneric
Copy link
Contributor Author

Yes, I have used stage.metrics to generate the the metrics that I want. However, since path like /user/1234 is a relatively high cardinality, and it will cause the Prometheus overhead, and I want to preprocess the path in the Grafana Alloy. If I could provide the redacted URL template similar to Beyla, I could replace the path to the redacted one. I could easily query the Prometheus with path label, and it also don't leave the overhead to Prometheus.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants