Pass a configuration filepath as an argument to the streampipe command.

[aitoehigie]

Is your feature request related to a problem? Please describe.
I'm always frustrated when I cannot dynamically pass the Steampipe configuration file path (spc file) as an argument to the Steampipe command. This limitation prevents me from seamlessly integrating Steampipe into my web application where each user supplies their API tokens, and the configuration needs to be generated on the fly.

Describe the solution you'd like
I would like the ability to pass the Steampipe configuration file path (spc file) as an argument to the Steampipe command. This enhancement would allow me to dynamically generate the configuration file for each user in my web application context, ensuring that each user's API tokens are used for querying with Steampipe.

Describe alternatives you've considered
One alternative solution could be to set up separate Steampipe instances for each user, each with their own pre-configured spc file. However, this approach would be less scalable and would require significant overhead in managing multiple instances.

Additional context
This feature would greatly enhance the flexibility and usability of Steampipe in web application contexts, enabling seamless integration with user-specific API tokens and configurations. It would simplify the deployment and management of Steampipe in multi-user environments and improve the overall user experience.

[johnsmyth]

We could add the ability to specify the config path but I'm not sure that it would solve your particular problem. The plugin configuration defines connections which are global - they are server-level objects, not client settings. A single Steampipe DB instance has the same connections for all clients. That means that if you want a different connection config for each user you would have to create a DB instance for each user. You can do this today using --install-dir, STEAMPIPE_INSTALL_DIR or the install_dir workspace argument.

An alternative is to create a connection for every user and set the search path on a per-client basis to default to "their" connection. This doesn't preclude them from using any other connection however, just sets the connection that would be used for unqualified queries. You could conceivably use Postgres permissions to lock this down, but you would have to implement that yourself. Steampipe creates a single steampipe user, as the Steampipe CLI is optimized for a single user.

You could also explore Turbot Pipes. Where Steampipe CLI is meant for a single person doing a single thing at a single point in time, Pipes is intended to allow many people doing many things over time.

[aitoehigie]

I will be calling different instances of Steam pipe for each user, so would passing --install-dir <the path to the ephemeral Stripe configuration file> to the Steam pipe command work?

…

On Wed, 20 Mar 2024, 14:53 johnsmyth, ***@***.***> wrote: We could add the ability to specify the config path but I'm not sure that it would solve your particular problem. The plugin configuration defines connections which are global - they are server-level objects, not client settings. A single Steampipe DB instance has the same connections for all clients. That means that if you want a different connection config for each user you would have to create a DB instance for each user. You can do this today using --install-dir, STEAMPIPE_INSTALL_DIR or the install_dir workspace argument. An alternative is to create a connection for every user and set the search path <https://steampipe.io/docs/guides/search-path> on a per-client basis to default to "their" connection. This doesn't preclude them from using any other connection however, just sets the connection that would be used for unqualified queries. You could conceivably use Postgres permissions to lock this down, but you would have to implement that yourself. Steampipe creates a single steampipe user, as the Steampipe CLI is optimized for a single user. You could also explore Turbot Pipes](https://turbot.com/pipes). Where Steampipe CLI is meant for a single person doing a single thing at a single point in time, Pipes is intended to allow many people doing many things over time. — Reply to this email directly, view it on GitHub <#4202 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABPDWG77FCEJZO37FHPWMDYZGID7AVCNFSM6AAAAABE4BUWUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBZGYZDANJQGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[johnsmyth]

I guess I misunderstood your request. It sounded like you wanted to share the same database instance from different clients, with different configurations for each. If you are going to start a separate DB instance for each user then it is reasonable to add a config path arg to the service start command. Is that what you are looking for?

As for the --install-dir, that needs to point to the equivalent of the ~/.steampipe directory. The drawback is that you would have to install the plugins in each of these instances.

[aitoehigie]

Exactly.
I would like to add a config path argument to the Steampipe command.
The steampipe instances aren't long-running instances as I intend calling it from a web server context (with Python's subprocess, then passing a config file containing the Stripe API key, that I created in that request cycle) and displaying the output in a dashboard.
Can I do this currently with Steampipe?

[johnsmyth]

When you say displaying output in a dashboard do you mean a Steampipe/Powerpipe dashboard or your own application's dashboard?

[johnsmyth]

If you are treating these instances as ephemeral and short-lived, and you are only accessing them through your own code on the back end, perhaps the SQLite Extensions or Export CLIs might be a better, more lightweight fit?

[aitoehigie]

When you say displaying output in a dashboard do you mean a Steampipe/Powerpipe dashboard or your own application's dashboard?

My application's dashboard

[aitoehigie]

If you are treating these instances as ephemeral and short-lived, and you are only accessing them through your own code on the back end, perhaps the SQLite Extensions or Export CLIs might be a better, more lightweight fit?

I think I will try the export CLI, I see it allows me to pass a --config flag. Which is fits what I am trying to accomplish.

[kaidaguerre]

closing this issue - please raise another or get in touch via the steampipe community of you have further issues/questions