Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to set GOMEMLIMIT on M3 Mac #2305

Open
nicholsn opened this issue Dec 20, 2024 · 2 comments
Open

Failed to set GOMEMLIMIT on M3 Mac #2305

nicholsn opened this issue Dec 20, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@nicholsn
Copy link

What's wrong?

After installing with homebrew and setting up a pipeline to write logs to loki, Alloy fails to send logs and reports an error indicating that the system doesn't support cgroups and cant GOMEMLIMIT.

Steps to reproduce

  • Follow default install instructions
  • Create the config below and restart the service
  • run an app to start generating logs to parse
  • check the alloy logs for the error

System information

M3 Macbook arm64

Software version

Grafana Alloy v1.5.1

Configuration

logging {
  level = "warn"
}

local.file_match "log_files" {
  path_targets = [
    { __path__ = "/tmp/app/logs/server.log" },
  ]
}

loki.source.file "log_scraper" {
  targets        = local.file_match.log_files.targets
  forward_to     = [loki.process.log_processor.receiver]
  tail_from_end  = false
}

loki.process "log_processor" {
  // Parse the log line as JSON and extract fields
  stage.json {
    expressions = {
      ts       = "timestamp",
      level    = "level",
      logger   = "name",
      msg      = "message",
      filename = "metadata.filename",
      funcName = "metadata.funcName",
      lineno   = "metadata.lineno",
    }
  }

  // Convert the extracted timestamp to the appropriate format
  stage.timestamp {
    source = "ts"
    format = "2006-01-02 15:04:05,000"
  }

  // Assign extracted fields as labels
  stage.labels {
    values = {
      level    = "",
      logger   = "",
      filename = "",
      funcName = "",
      lineno   = "",
    }
  }

  // Set the log message to the extracted 'msg' field
  stage.output {
    source = "msg"
  }

  forward_to = [loki.write.loki_output.receiver]
}

loki.write "loki_output" {
  endpoint {
    url = "http://localhost:3100/loki/api/v1/push"
  }
}

Logs

ts=2024-12-20T06:12:53.964364Z level=error source=github.com/KimMachineGun/[email protected]/memlimit/memlimit.go:117 msg="failed to set GOMEMLIMIT" package=github.com/KimMachineGun/automemlimit/memlimit error="failed to set GOMEMLIMIT: cgroups is not supported on this system"
@nicholsn nicholsn added the bug Something isn't working label Dec 20, 2024
@nicholsn
Copy link
Author

I also used the instructions at https://grafana.com/docs/alloy/latest/configure/macos/#configure-the-alloy-service to edit the homebrew recipe and set GOMEMLIMIT=2750MiB, then brew reinstall --formula alloy , and restart the service. This did not fix the issue and resulted in the same error although a comment in the source code indicated that this envrionment variable should override the cgroup derived setting:

// Set the memory limit, this will honor GOMEMLIMIT if set

❯ cat /opt/homebrew/etc/alloy/config.env
GOMEMLIMIT=2750MiB

@dehaansa
Copy link
Contributor

This error should be resolved (no longer attempting to use groups & logging the message) in next release #2264

The environment variable should definitely overrule the cgroup behavior (see https://github.com/KimMachineGun/automemlimit/blob/v0.6.1/memlimit/memlimit.go#L146C2-L150C3 ) so I'm surprised that you weren't able to set GOMEMLIMIT in the appropriate way through homebrew services. I'll find some time to test that behavior and see if I can replicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants