Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "When to use Flow" section to the docs #5793

Closed
wants to merge 3 commits into from

Conversation

ptodev
Copy link
Contributor

@ptodev ptodev commented Nov 16, 2023

One of the most common questions which the dev team gets is "how is the Agent different from the OTel Collector?".

This PR adds a new section to the docs which aims to answer this. I added this section to the very beginning of the docs:

Screenshot 2023-11-16 at 17 49 06

The Flow Mode page does a great job of explaining what Flow mode is. I would like this section to be the next one, so that users can immediately see what the Agent is good at, and why they should consider it. Then, they could look into the next section - Set up Flow mode

This PR is still a draft:

  • I need to add YAML and River examples in the page about OTel.
  • I need to add more hyperlinks. E.g. all of the names of Flow components should be hyperlinks to the doc page of that component.
  • I would like to make Venn diagrams which show how Agent relates to the Collector and the Prometheus Agent Mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to add a Venn diagram such as this one in the beginning of this page:

Agent and Collector Venn diagram

Copy link
Contributor

@clayton-cornell clayton-cornell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some first pass thoughts and suggestions.

We need to pick one name for things... it skips about between Grafana Agent... Flow.. Agent... and so on. Same with OpenTelemetry Collector... sometimes it's Collector.... the Collector... and so on.

docs/sources/flow/when-to-use-flow/_index.md Outdated Show resolved Hide resolved
docs/sources/flow/when-to-use-flow/_index.md Outdated Show resolved Hide resolved
docs/sources/flow/when-to-use-flow/_index.md Outdated Show resolved Hide resolved
@ptodev ptodev force-pushed the ptodev/docs-collector-and-agent branch from 569c830 to 51b70ce Compare November 17, 2023 11:11
@clayton-cornell clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Nov 17, 2023
@ptodev ptodev force-pushed the ptodev/docs-collector-and-agent branch from 51b70ce to 7dafe2a Compare November 17, 2023 13:33

The Collector may be a better choice if:
* You are already using the Collector, and do not need any of the features of Flow.
* You prefer YAML syntax over River.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • You have a custom component, which you can incorporate in your own Collector distribution via the Collector Builder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the main point the fact that you can add a custom component? Or is using the Collector Builder the advantage?

Either way, there is nothing stopping someone from forking the Agent and adding their own custom component. We just don't document that process very well, because in the long term we want to solve by developing a plugin-based system which doesn't require a new distribution/fork when a custom component is needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is having a custom component, having a supported (and documented?) way to do that is what determines the tooling.


Collector needs to convert Prometheus signals to the OTLP format to process them.
Flow, on the other hand, can process those signals natively using components such as `prometheus.relabel`, `prometheus.relabel`, and `prometheus.remote_write`.
This could lead to better performance and ease of use.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our tests have shown that this leads to considerable performance gains.

@sh0rez , is there anything public we can link to?

Flow may be a better choice if:
* You prefer River syntax over YAML.
* You would like to collect "profile" telemetry - Flow can do this via Pyroscope components.
* You would like to process Prometheus or Loki telemetry without converting it to OTLP.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the OSS documentation? One extra advantage of Flow is that Grafana Labs offers support for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for OSS, but sometimes the Cloud docs import OSS docs and I'm not sure if this one will end up being included.
I wanted to write this in a more general way without mentioning support commitments, since I'm not sure if we can explain support commitments in simple terms for all users and clients who may be reading the article.

Copy link
Contributor

This PR has not had any activity in the past 30 days, so the needs-attention label has been added to it.
If you do not have enough time to follow up on this PR or you think it's no longer relevant, consider closing it.
The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your PR will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity.
Thank you for your contributions!

@github-actions github-actions bot added needs-attention An issue or PR has been sitting around and needs attention. and removed needs-attention An issue or PR has been sitting around and needs attention. labels Jan 11, 2024
@ptodev
Copy link
Contributor Author

ptodev commented Jan 18, 2024

We are not quite sure where to position this in the docs, so I will close the PR. I hope I can open a similar PR in a few weeks once we have more consensus over how to convey such information.

@ptodev ptodev closed this Jan 18, 2024
@github-actions github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Feb 21, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. type/docs Docs Squad label across all Grafana Labs repos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants