Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support relay of collectd UDP #83

Open
bhakta0007 opened this issue Mar 30, 2021 · 1 comment
Open

Support relay of collectd UDP #83

bhakta0007 opened this issue Mar 30, 2021 · 1 comment

Comments

@bhakta0007
Copy link

bhakta0007 commented Mar 30, 2021

I looked at #54 and #29
this seems to be couple of years old.

Is there no way to use HA-based deployment for collectd based ingest traffic?

What is the problem that one type of UDP payload (line protocol) can be relayed but not the other (Collectd). Is there any technical problem that prevents relay of collectd based ingest data?

@bhakta0007
Copy link
Author

bhakta0007 commented Mar 31, 2021

Just to prove the concept of forwarding the UDP payload works, I made this change and tried it out..

(venv) bhakta@blr:~/dev/influxdb-relay$ git diff .
diff --git a/relay/udp.go b/relay/udp.go
index 081b94d..9a2552e 100644
--- a/relay/udp.go
+++ b/relay/udp.go
@@ -8,8 +8,6 @@ import (
        "sync"
        "sync/atomic"
        "time"
-
-       "github.com/influxdata/influxdb1-client/models"
 )

 const (
@@ -160,38 +158,13 @@ func (u *UDP) Stop() error {
 }

 func (u *UDP) post(p *packet) {
-       points, err := models.ParsePointsWithPrecision(p.data.Bytes(), p.timestamp, u.precision)
-       if err != nil {
-               log.Printf("Error parsing packet in relay %q from %v: %v", u.Name(), p.from, err)
-               putUDPBuf(p.data)
-               return
-       }
-
-       out := getUDPBuf()
-       for _, pt := range points {
-               if _, err = out.WriteString(pt.PrecisionString(u.precision)); err != nil {
-                       break
-               }
-               if err = out.WriteByte('\n'); err != nil {
-                       break
-               }
-       }
-
-       putUDPBuf(p.data)
-
-       if err != nil {
-               putUDPBuf(out)
-               log.Printf("Error writing points in relay %q: %v", u.Name(), err)
-               return
-       }
-
+       // log.Printf("Got packet relay %q from %v: length %v", u.Name(), p.from, len(p.data.Bytes()))
        for _, b := range u.backends {
-               if err := b.post(out.Bytes()); err != nil {
-                       log.Printf("Error writing points in relay %q to backend %q: %v", u.Name(), b.name, err)
+               // log.Printf("Forward data from %v to %v, lengh %v", p.from, b.addr, len(p.data.Bytes()))
+               if err := b.post(p.data.Bytes()); err != nil {
+                       log.Printf("Error forwarding data from %v to %v, lengh %v: %v", p.from, b.addr, len(p.data.Bytes()), err)
                }
        }
-
-       putUDPBuf(out)
 }

What this is doing Is taking the incoming UDP buffer (without trying to parse for the line protocol) and simply writing it out to the list of backends. I am new to golang & influxdb-relay code - I am not completely sure I have written the code in a way there are no leaks, etc).

If you guys think this should work - I can work on a patch that introduces a "collectd" relay (in addition to existing HTTP and udp relays).

Do you guys foresee any issues?

laertis added a commit to 1plusX/influxdb-relay that referenced this issue Jul 25, 2023
…InfulxDB UDP line protocol to support sending over collectd traffic over UDP

influxdata#83
laertis added a commit to 1plusX/influxdb-relay that referenced this issue Jul 25, 2023
…InfulxDB UDP line protocol to support sending over collectd traffic over UDP

> influxdata#83
laertis added a commit to 1plusX/influxdb-relay that referenced this issue Jul 25, 2023
…loring for InfulxDB UDP line protocol to support sending over collectd traffic over UDP

> influxdata#83
laertis added a commit to 1plusX/influxdb-relay that referenced this issue Aug 28, 2023
…loring for InfulxDB UDP line protocol to support sending over collectd traffic over UDP

> influxdata#83
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant