changing AWS_PROFILE value does not appear to change the authentication used to get information

[jpcope]

Describe the bug
When a command like AWS_PROFILE=<some-profile> steampipe query "<some-query>" is run and the operator changes the profile between invocations the results remain the same and the new profile is never used or the results are never updated with values from the new account. When running .cache clear and then trying again, they still remain the same. It's like the credential used is cached and is not removed. In this specific setup the profiles are assuming a role to target different aws accounts from a technically third common federated access account.

Steampipe version (steampipe -v)
Steampipe v0.19.4

Plugin version (steampipe plugin list)
+--------------------------------------------+---------+-----------------+
| Installed Plugin | Version | Connections |
+--------------------------------------------+---------+-----------------+
| hub.steampipe.io/plugins/turbot/aws@latest | 0.103.0 | aws_sometestaccount,aws |
+--------------------------------------------+---------+-----------------+

To reproduce

1. Configure two profiles that assume roles in different AWS accounts
2. Run AWS_PROFILE=<profile-1> steampipe query "<some-query>"
3. Run AWS_PROFILE=<profile-2> steampipe query "<some-query>"

Expected behavior
Clearing the cache and/or changing the profile should result in data being read from the new account and not from the previous account

[cbruno10]

Hi @jpcope , can you please share your ~/.steampipe/config/aws.spc file?

Also, do you have any other AWS env vars currently set (env | grep AWS)?

[jpcope]

00:~ $ cat ~/.steampipe/config/aws.spc
connection "aws" {
  plugin = "aws"
}

connection "aws_sometestaccount" {
  plugin  = "aws"
  profile = "sometestaccount"
  regions = ["us-east-1"]
}
00:~ $ env | grep AWS
AWS_PAGER=
AWS_SDK_LOAD_CONFIG=true
AWS_PROFILE=sometestaccount
00:~ $

[graza-io]

Hey @jpcope

Sorry to see you're not getting what you need... I did a quick test and as you can see, I get what I expected to see.

Note: sq is just a convenience alias in my shell for steampipe query (alias sq="steampipe query")

❯ aws s3 ls --profile=dd | wc -l
     121
❯ aws s3 ls --profile=grz | wc -l
       1
❯ AWS_PROFILE=dd sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 121   |
+-------+
❯ AWS_PROFILE=grz sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 1     |
+-------+
❯ cat ~/.steampipe/config/aws.spc
connection "aws" {
  plugin = "aws"
}
❯ env | grep AWS
AWS_SDK_LOAD_CONFIG=true

I then noticed you had a second connection in your Steampipe config, so added a similar one to mine and retested:

❯ cat ~/.aws/config
[default]
region = eu-west-2
[profile grz]
region = eu-west-2
output = json
[profile dd]
region = eu-north-1
output = json
❯ cat ~/.steampipe/config/aws.spc
connection "aws" {
  plugin = "aws"
}

connection "aws_dd" {
  plugin  = "aws"
  profile = "dd"
  regions = ["eu-north-1", "us-east-1"]
}
❯ env | grep AWS
AWS_SDK_LOAD_CONFIG=true
❯ aws s3 ls --profile=grz | wc -l
       1
❯ aws s3 ls --profile=dd | wc -l
     121
❯ AWS_PROFILE=grz sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 1     |
+-------+
❯ AWS_PROFILE=dd sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 121   |
+-------+

Full disclosure I'm using the newest version of Steampipe:

❯ steampipe --version
Steampipe v0.20.3

Addition Notes

If you define multiple connections (one per profile), you don't need to utilise the AWS_PROFILE environment variable to utilise them, you can prefix the table with the connection name in your query, in example below using env var for grz profile but using the connection I set up for dd to utilise the prefix approach. Here is the documentation on querying multiple accounts.

❯ AWS_PROFILE=grz sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 1     |
+-------+
❯ sq "select count(*) from aws_dd.aws_s3_bucket"
+-------+
| count |
+-------+
| 121   |
+-------+

You may sometimes wish to gather information from multiple profiles at the same time, for this you can utilise aggregators. For example if I extend my aws.spc config to be the following:

connection "aws_dd" {
  plugin  = "aws"
  profile = "dd"
  regions = ["eu-north-1", "us-east-1"]
}

connection "aws_grz" {
  plugin  = "aws"
  profile = "grz"
  regions = ["eu-west-2"]
}

connection "_all_aws" {
  type        = "aggregator"
  plugin      = "aws"
  connections = ["aws_dd", "aws_grz"]
}

Then when I execute the query against the aggregator I can obtain results from both my other connections...

❯ sq "select count(*) from aws_s3_bucket"
+-------+
| count |
+-------+
| 122   |
+-------+

❯ sq "select count(*) from _all_aws.aws_s3_bucket"
+-------+
| count |
+-------+
| 122   |
+-------+

Note: unqualified (no connection prefix on table) will default to first loaded in the search path - iirc this is alphabetical, hence the _ prefix on the aggregator making it first loaded, but as you can see in example above could always specify it as the prefix to table to ensure its usage.

I apologise this is a long response but I thought it best to provide information I could on attempting to replicate the issue and other approaches to obtain the information you require.

HTH!

[graza-io]

Hey @jpcope - hopefully the above helped you resolve your issue, could you confirm if that is the case or if you're still having issues?

[jpcope]

In the two profiles used to test, were they using a third and same source_profile with two different role_arns? I am pretty sure this plays a factor in this issue.

If the profiles tested with don't specify these attributes but use different key id and secret keys to grant access I do not think the issue will manifest.

https://docs.aws.amazon.com/cli/latest/topic/config-vars.html

[jpcope]

Thanks for taking a look at this! Frustratingly I cannot recreate the issue at this time either though I've shown it to my peers at the time.

In my workflow I'm often using the interactive cli to build a query on one account, then drop out to run it on specific accounts where it's relevant rather than the larger set of all of them (for speed reasons).

I'll look out for it again and post some cleaner screenshots in a new issue if it ever resurfaces. I've gone ahead and updated my versions of everything to hopefully keep the issue away! Hopefully it was just something unique and bad on my work machine that won't happen again.

😄

[jlm0x017]

I thought I was experiencing this, but ultimately determined my real issue was here: turbot/steampipe#4155. TL;DR my local AWS_PROFILE was not being honored because I had left a an open steampipe query session going in another window.

Dropping this note for future readers as one more thing to check.