Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When processes fail, provide a stacktrace to the Resque failure queue #62

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Jberlinsky
Copy link

@Jberlinsky Jberlinsky commented Mar 30, 2018

Currently, when goworker workers fail due to an uncaught error, tracking down the root cause of the problem is rather difficult. Only the contents of the error message itself is passed to the Resque failure queue, unlike other languages (e.g. Ruby), which pass the entire stack trace.

Golang's errors do not support stack traces out of the box. The go-errors package makes it possible to attach stack trace information to the errors either at creation-time, or at wrapping-time.

This pull request takes advantage of the idempotency of go-errors's Wrap() function, allowing a worker to either raise a go-errors error object themselves, or automatically wrapping a standard Golang error when the failure is observed by goworker. A worker that throws a standard error will see the same information currently provided to the failure queue, with a stack trace pointing them to goworker/worker.go as the investigation point. A worker that wraps their standard error with go-errors will see that same information, plus a full stack trace to the point that the user instantiated/wrapped the go-errors error object.

@mingan
Copy link

mingan commented Apr 14, 2018

This bothers me as well about goworker 👏.

Have you considered errors library? I assumed it was the de facto standard for this problem and GitHub stars seem to agree.

@benmanns
Copy link
Owner

Thanks for the contribution - I think this is a great idea.

My concerns for this would be:

  • If possible I would prefer to use the more widely used library
  • Stack traces can be pretty large. It looks like go-errors/errors has a default max of 50, but even so it could add 5-10kb per error the the Redis queue which could be problematic with high error rates. Can we provide something like the backtrace config for Sidekiq.
  • How does the schema for the failure object change when it's stored in Redis? Does resque-web show the error and backtrace properly? Ideally we'd use the same format as Resque or Sidekiq implementations. Users may rely on the specific type/format for the Error field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants