Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stemapipe service restart stuck after first attempt #4314

Closed
jeffreymp17 opened this issue Jun 19, 2024 · 6 comments
Closed

Stemapipe service restart stuck after first attempt #4314

jeffreymp17 opened this issue Jun 19, 2024 · 6 comments
Labels
bug Something isn't working stale No recent activity has been detected on this issue/PR and it will be closed

Comments

@jeffreymp17
Copy link

jeffreymp17 commented Jun 19, 2024

Describe the bug
When running Steampipe as a service, when I try to restart the service I'm running in a Linux Docker container, this gets stuck on the first try and I need to kill the process and run it again and when it wakes up it sometimes doesn't recognize certain profiles and says they don't exist
I attached a video below

Steampipe version (steampipe -v)
Example: v0.23.2

To reproduce

Expected behavior
Should restart after command is executed

Additional context
Add any other context about the problem here.

Screen.Recording.2024-06-19.at.2.53.23.PM.mov
@jeffreymp17 jeffreymp17 added the bug Something isn't working label Jun 19, 2024
@idesofoctober
Copy link

I have a different but similar issue, using:

powerpipe v0.4.0
aws plugin v0.139.0
steampipe v0.23.2

I create a config file and then run steampipe service start and then run powerpipe benchmark (aws compliance mod:latest), all in a docker container

Sometimes it works, sometimes I get a slew of errors like:

rpc error: code = Unavailable desc = error reading from server: read unix @->/tmp/plugin2367597389: read: connection reset by peer (SQLSTATE HV000

and

"scanIteratorBase cannot iterate: connection..."

@pskrbasu
Copy link
Contributor

@jeffreymp17 I will try to reproduce this.

@idesofoctober is this something that you are seeing intermittently?

@jeffreymp17
Copy link
Author

This is happening every time

@idesofoctober
Copy link

@pskrbasu I'm looking at some reporting I created around this and across 419 accounts, it definitely DOES NOT happen with all accounts, but it happens consistently on the account it does occur with. Some additional context:

We run this on EKS and each account benchmark run is run in one EKS instance with it's own spc and credentials file (all of them created using the same underlying code). Happy to work with you re: what is the same/different about each account. I saw the issue posted here and thought it might be similar re: the overall database / connection load time/configuration, and my dockerfile does a steampipe service start, aws plugin install, and steampipe service stop. My code is doing a steampipe service start, wait 2 minutes, and then run benchmarks which (I think, my data and troubleshooting hasn't been super diligent) reduced the total number of accounts with this error.

One hypothesis I have (based on nothing tangible, but thought I'd share in case it raises an idea) is that the aws.spc for all of these is using * for region. Could the number or location of regions that are in play causing some difference in the load time (if the load time is in fact the issue?)?

I'm going on vacation soon, for next week, but if there is something I can look at re: differences between the accounts where this occurs and where this doesn't occur, i'll get back to you as soon as I can.

Copy link

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the stale No recent activity has been detected on this issue/PR and it will be closed label Aug 28, 2024
Copy link

This issue was closed because it has been stalled for 90 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale No recent activity has been detected on this issue/PR and it will be closed
Projects
None yet
Development

No branches or pull requests

3 participants