ozone sh key list fails entire FSO bucket listing when one child directory/key lacks READ ACL

[Pigueiras]

Hello,

I would like to check whether the following behavior is expected, or whether it should be considered a bug. My environment is:

Ozone version: 2.0.0
Authorization: Native ACLs
Bucket layout: FILE_SYSTEM_OPTIMIZED

We have a bucket where the user has READ, WRITE, and LIST permissions on the bucket:

ozone sh bucket getacl /<volume>/<bucket>/

{
  "type": "USER",
  "name": "<user>",
  "aclScope": "ACCESS",
  "aclList": [ "READ", "WRITE", "LIST" ]
}

Listing with a prefix works correctly:

ozone sh key list /<volume>/<bucket> -p <prefix>

However, listing the bucket without a prefix fails completely:

ozone sh key list /<volume>/<bucket>

Error:

PERMISSION_DENIED User <user> doesn't have READ permission to access key Volume:<volume> Bucket:<bucket> Key:.Trash

The reason seems to be that even if there are other keys inside that bucket where the user has permissions, if any of the keys the permissions are not correct, the command fails completely to list the keys where the user has access.

ozone sh key getacl /<volume>/<bucket>/.Trash

[
  {
    "type": "USER",
    "name": "<a different user>",
    "aclScope": "ACCESS",
    "aclList": [ "ALL" ]
  }
]

My expectation was that the command would list the entries the user is allowed to list, skipping entries that are not readable. Instead, one unreadable child entry appears to make the entire bucket listing fail 😢

Is this the intended behavior for ozone sh key list on an FSO bucket with native ACLs?

I am also wondering whether this might be a known issue in Ozone 2.0.0 that has already been addressed in a newer release or on master.

Thanks a lot 😄

[ivandika3]

From my quick search, the issue is that "ozone key list" calls OzoneBucket#listKeys with shallow=false so it will list all the subdirectories in the bucket. If one of the subdirectory list permission fails, it fails the whole operation.

Currently, there is no option in ozone sh key list (ListKeyHandler) to make the shallow configurable. You can raise a PR to support this.

Alternatively, if you use S3G list-keys, the shallow flag is set (ozone.s3g.list-keys.shallow.enabled) by default, so you should not encounter permission issues.

Feel free to test this behavior.

[Pigueiras]

Thanks @ivandika3, that makes sense. My question was mainly to confirm whether this was expected behavior or not. If this is how it works by default for ozone sh key list, then that should be fine. Indeed, via s3g the behavior is different.

[ivandika3]

Yes, it seems the shallow flag was introduced after ListKeyHandler.