Permissions spec for Beta

[pavish]

This adds the revamped permissions spec for beta to utilize the PostgreSQL permission system.

Rendered document.

[pavish]

@mathemancer I'd particularly like you to go through the Dealing with Ownership and Collaboration issues section and the Default role section, since they add more importance to the default role and require architectural sign-off.

This approach isn't something I'm strongly tied to, my concern is to solve the collaboration issues that would come up when working with multiple roles. If you have a better recommendation, I'm open to it.

[mathemancer]

I left many specific comments.

My big overarching comment is that ALTER and DROP are controlled by ownership, and can't be granted except by granting ownership, or granting access to act as the role that owns the object in question. I think we should consider having a call to go through the details of that. I also think we should avoid trying to squeeze schema-level access in before beta.

[mathemancer]

I don't quite understand. Should they just be able to create other roles on the internal DB server in general?

[pavish]

Should they just be able to create other roles on the internal DB server in general?

Yes, that's correct.

[pavish]

This is mentioned more clearly in 2ad0d72

[mathemancer]

With our newly-merged models, the "associate them with the database" part is a bit wonky. What, precisely, do you mean by that? Also, this moves into the territory of creating abstractions that aren't real on the underlying database server, and they're not simplifying abstractions. In particular, this means (if I'm understanding you correctly) that you could need multiple identical roles on the server to associate with various databases, wouldn't it?

[seancolsen]

With our newly-merged models, the "associate them with the database" part is a bit wonky.

This is part of the point I was trying to make with my critique of the models.

(Not trying to beat a dead horse — just trying to connect some dots.)

[pavish]

What, precisely, do you mean by that?

I meant that the newly created roles should have CONNECT, and CREATE to the database. They should also have the ability to SELECT the existing schemas and tables within the database.

I will make this clearer in the spec.

[pavish]

I hope this is made clear in 2ad0d72

[mathemancer]

What about simply granting any role created or managed via Mathesar to the 'default role'? If you transfer ownership to the 'default role', you won't be able to ALTER or DROP a table you yourself created, regardless of your other privileges.

[seancolsen]

granting any role created or managed via Mathesar to the 'default role'?

If we grant the default role to any role created via the Mathesar, then I'm confused about how a person would be able to create more restricted roles via the Mathesar UI. Say I want one role with full access and one read-only role. If both roles are granted "default", then I'd need to ensure that "default" has minimal access, right? This seems counterintuitive to me. I've been assuming that the default role should actually have maximum access since its the role that owns things and the role that gets used by Mathesar admins. Perhaps I've assumed incorrectly though.

[mathemancer]

I'm thinking the other way around in this case. If we configure the roles brent and sean to be used from Mathesar, then we'd GRANT brent TO default_role and GRANT sean TO default_role. This means default_role would be able to do anything brent or sean (inclusive) could do, e.g., drop one of brent's tables. But, brent wouldn't be able to drop one of sean's tables.

With that said, I had a chat with Pavish today, and my (our) thinking has evolved a bit.

[seancolsen]

I'm thinking the other way around in this case

Ah right. Sorry — I misread your comment.

[pavish]

This is no longer relevant.

We've introduced displaying the ownership for objects and there are changes revolving granting of privileges in 2ad0d72.

[mathemancer]

They still wouldn't be able to ALTER or DROP.

[seancolsen]

wouldn't be able to ALTER or DROP

I think this might be okay.

There is potential for users to end up in a situation where they can create things but not delete them. That would be confusing. We want to avoid that situation.

As I see it, we basically want to steer people towards using the default role if they're trusted with the privilege of creating or deleting things.

I would see us doing this steering by providing clear in-app documentation that makes this behavior apparent to users as early in their flows as possible. Like, we'd need to make Mathesar admins aware of this behavior when they are configuring the roles for a DB.

[pavish]

We've introduced displaying the ownership for objects and there are changes revolving granting of privileges in 2ad0d72.

[mathemancer]

Perhaps I don't understand what this means.

Would every role visible in Mathesar should be granted privileges automatically? If I'm correct that this is the intent, I don't think we should do that. It would have massive effects on the underlying security of the DB role setup, and would make it pretty much pointless to add any roles into Mathesar since they wouldn't be useful for controlling access to objects created in Mathesar.

[seancolsen]

All other priviliges are granted automatically to each role configured within Mathesar.

I would also like clarity on this. Can you give some examples?

[pavish]

This was under the assumption that we'd allow admins to have different roles. From the last call I had with @mathemancer, our thoughts are to always use only the 'default role' for the admins.

I will update the spec after we discuss this more.

[pavish]

No longer relevant.

We've introduced displaying the ownership for objects and there are changes revolving granting of privileges in 2ad0d72.

[seancolsen]

This document is immensely helpful. Thank you @pavish.

Generally, I think this all looks great.

My one small hesitation at the product level is this:

• This spec has a limitation which I don't love: it's hard to create a new database on a remote server because I need to enter a password.

  For the "admin interface" kind of product, this limitation is perfectly acceptable.

  However, during our strategy meeting, several people expressed interest in continuing to design for (and market towards) the "from scratch" use case where people use Mathesar as a relational spreadsheet that stands on its own, like Airtable. For that use case I think Mathesar needs a product-level abstraction akin to an Airtable "Base", i.e. a self-contained project that can be easily created, managed, and destroyed. Without schema-level permissions (unlikely by beta, it seems), then the "project" abstraction might tend to become a "database" in people's minds. If they use the internal DB server, then they can easily create new databases, but I think that those databases will basically end up becoming "global" because there's not a password-free way to create them using more granular ownership and permission. For example, it would be quite cumbersome to create a new DB that only I can see. I might want to do this just to experiment without cluttering up other people's experience in Mathesar. And if for some reason they're not using the internal DB (e.g. SQLite internal DB, or they want managed backups for their user data), then they can't even create new databases in a password-free way.

  These limitations make Mathesar less compelling for the stand-alone "relational spreadsheet" use case. That's okay with me because my inclination is to focus on the "admin interface" type of product for now anyway. But I just want to flag this as a concern since other people didn't seem as keen to focus exclusively on the admin interface approach in the short term.

Also I have a side note on the sequence of planning:

• Throughout our permissions conversations, Pavish and I have had a hard time reconciling what has appeared to be different approaches to product design: I've been pushing for understanding the models sooner, and Pavish has been pushing for understanding the user experience sooner. Now that we have this product spec, I'll say that this is exactly the kind of design that I need to understand before understanding the models. My personal approach is (1) product spec, (2) data models, (3) UX. We can debate semantics on what sort of design actually counts as "UX", but point is that (for me) the document in this PR takes precedent over Brent's models doc.

  What I'm suggesting here is that we work towards agreeing first on this product spec and then (subsequently) ensure our models are appropriately designed to implement it. I make this suggestion because my opinion is that we still have significant architectural incompatibilities between this product design and our planned models, but I don't want to raise that critique here. I'd rather agree on this product design on its own — with zero dependencies — and then base my critique of the models on this product design.

[ghislaineguerin]

Thanks @pavish . I don't have any question or comments for now. This seems clear enough for me to get started.

[pavish]

@ghislaineguerin There were further discussions that took place regarding ownerships which aren't updated in the spec yet. I'll update it and request review again.

[pavish]

@mathemancer @kgodey @seancolsen I've updated the spec based on the recent DB ownership discussion. Here's the commit: 2ad0d72.

Please take a look.

@ghislaineguerin This introduces some new concepts around ownership. I'll be glad to discuss this over our 1-1 call tomorrow or have a separate call for it, if you need one.

[seancolsen]

Looks good @pavish

[seancolsen]

Sorry... one more thought/question/suggestion that's been nagging at me...

A couple weeks ago we (Sean/Brent/Pavish) had an impromptu call to discuss my concern about the data models. Near the end of the call I asked Pavish a couple questions which helped clear up a misunderstanding I had about Pavish's intended product-level functionality.

Our dialog went something like this:

• Sean: So lets say I use Mathesar to connect to 10 different Postgres DBs on the same server with the same Postgres role. How many entries should Mathesar have for that Postgres role?

• Pavish: Ten.

• Sean: Ok. Now let's say I've changed the password for that role in Postgres. How many times do I need to update the password in Mathesar?

• Pavish: Ten.

• Sean: Ok.

To be clear: this product design is perfectly fine with me. But it was definitely not how I thought the permissions system would work until we had that call.

I'd like to make sure everyone is on the same page about this behavior, and I think it would help to put something in this PR. I'm proposing that we add the following section to the end of the permissions-revamp.md document:

Encapsulation

Within Mathesar, configuration of a Postgres server and its roles should be performed within the context of a database. All such configuration should be siloed to that database. For example:

• Within a database, Mathesar should only present Postgres roles which have explicitly configured for that specific database. Mathesar should hide roles which have been configured for other databases, even if those other databases exist on the same server.

• Changing the password stored in Mathesar for a Postgres role should only affect the Postgres role configured within one database. If the same role is used for multiple databases, then the password will need to be changed multiple times.

• Changing the port number configured for one Mathesar user database should not affect the port number configured for other Mathesar user databases. Like passwords, updates to port numbers will need to be performed multiple times.

[mathemancer]

The long and short of it is: No matter how I try to see the namespacing of a role under a database as a simplifying abstraction, it just seems to make things more complicated, difficult, and confusing.

[mathemancer]

I don't think we should do this by default. It makes configuring any owning role in Mathesar a dangerous proposition. It also ruins any chance of auditing user actions on the underlying database.

[mathemancer]

To double-check: This doesn't imply creating a new default role for each database, correct? I.e., we could have a DB server with only one role, used as default for each DB?

[mathemancer]

Or do you mean the credential model instance here? In that case, we really need to consider how to present this in a non-confusing way to users. Are they supposed to understand that multiple roles in the Mathesar UI all map to the same actual role on the underlying database (server)?

[pavish]

This doesn't imply creating a new default role for each database, correct? I.e., we could have a DB server with only one role, used as default for each DB?

Yes, this is correct.

[mathemancer]

This implies you mean the credential instance. If correct, we should make that obvious in the document.

[mathemancer]

I know we've already discussed this, but this walkthrough makes the namespacing of a role under a database more obviously overcomplicated. Why can't the admin who has a connection to the external server already established (with sufficient privileges) just create a new DB by entering a database name?

It's also confusing for the user. They'll have to enter a previous database, as well as credentials that have privileges to connect to that database (along with appropriate CREATEDATABASE privileges. If I were a user, I'd be trying to figure out:

• Why I had to enter an arbitrary preexisting database (You could enter any that the role in question has CONNECT privileges on)
• What, exactly, am I creating here?
  • the database (this one is obvious)
  • A new role?
  • A copy of the role?
    Even more questions arise for a user who does understand how roles, etc. work in PostgreSQL, since they'll be naturally confused as to why we're simulating a bunch of roles when there's actually only one.

After reading through this in its current form, I'm even more convinced that trying to namespace roles under databases in the users' minds is a complicating rather than simplifying abstraction.

[pavish]

I updated an older version of the spec I had and forgot to remove this point. Thanks for pointing this out, I'll remove this.

Mathesar admins should not be able to create a new database on external servers.

We decided to remove the feature of being able to add a new database on external servers and sharing credentials. When/if we want to introduce it in the future, we can figure out a way of introducing the concept of database servers.

[pavish]

Here's the commit 189b772 that should make this clear.

[mathemancer]

What does CREATEDATABASE even mean as a privilege if roles are subordinate to databases (in the UI, and in users' minds)? I.e., how is an admin user supposed to think about this if we're avoiding the concept of a server that has both roles and databases?

[pavish]

This is no longer relevant after 189b772

[mathemancer]

I would say they're acting as the owner. They're not really obtaining anything, and we need to make sure they understand that in the UI. They'll get the wrong understanding if they think acting on the table somehow takes ownership from the user who created the table.

[mathemancer]

Encapsulation

Within Mathesar, configuration of a Postgres server and its roles should be performed within the context of a database. All such configuration should be siloed to that database. For example:

• Within a database, Mathesar should only present Postgres roles which have explicitly configured for that specific database. Mathesar should hide roles which have been configured for other databases, even if those other databases exist on the same server.
• Changing the password stored in Mathesar for a Postgres role should only affect the Postgres role configured within one database. If the same role is used for multiple databases, then the password will need to be changed multiple times.
• Changing the port number configured for one Mathesar user database should not affect the port number configured for other Mathesar user databases. Like passwords, updates to port numbers will need to be performed multiple times.

Why attach it to the database in this way? It seems to add a dependency without any value. I can see the value of listing the roles that were configured in a given context, but the rest of the abstraction seems to add complexity.

[pavish]

@seancolsen and @mathemancer

1. Regarging Encapsulation, I consider it a UX decision and not a product level spec.
   • In our discussions regarding the concern raised by Sean, my stance was that we keep it flexible so that we'd have more freedom to experiement with the UX.
2. I do not want the ability to share credentials for the beta and I would like the role configuration panels to be present within the context of a 'Database page', as stated in Sean's comment.
3. I would however like to see if we can make it clear to the user in the UI that the roles are created in the underlying database server, even if we do not have a dedicated UX for managing database servers.

I'd say let's move this particular discussion to the UX design sessions.

[seancolsen]

@pavish When we release Mathesar beta, perhaps it will function as I've described within the "encapsulation" section above. Perhaps not. There could be subtle differences from what I've laid out, or drastic differences. I'll call this the "encapsulation question". "What is our approach to encapsulating server and role config within a database?" — that's the encapsulation question. I want to answer that question immediately. You seem to want to answer that question later in our design and development process. I don't care whether we classify the encapsulation question as "Product design" or "UX design"; as far as I'm concerned, that classification is a matter of semantics. But I do feel very strongly that kicking this can down the road will introduce significant risk of implementation delays before beta. The sooner we figure this out, the faster we can build. Why can't we answer the encapsulation question out now? What's blocking us from making a decision on that?

[pavish]

@seancolsen

The sooner we figure this out, the faster we can build.

I agree.

Why can't we answer the encapsulation question out now? What's blocking us from making a decision on that?

I thought that the question was answered already. Things would work like you've mentioned in your comment. My recent comment was related to documenting it in the product spec instead of thinking of it as a UX concern.

I'd like @ghislaineguerin to work on the UX, ask questions we might have not thought about yet, and present a preliminary barebones UX flow so that we can eliminate any remaining confusion.

Mainly, I'd like to prevent any questions later down the road, like Brent's recent comment, and I think seeing the UX flow would help a lot more than discussing it further on the doc. I'm good with updating the spec if you feel strongly about it.

We're not going to be kicking this down the road till implementation, the UX design phase is the immediate next step which Ghislaine has already started work on. I've added Ghislaine to the "Technical beta check in" call on Tuesday so that we can go through the UX flow.

[seancolsen]

@pavish

I thought that the question was answered already.

If this question is answered already, then I think it would be very useful to have it in the spec. But it's not clear to me that everyone agrees on one unified approach to encapsulation. If we have disagreement, then I don't see the question as being answered. Just to reiterate: I'm personally staking out a neutral position on the encapsulation question. I'm perfectly fine with many different approaches to encapsulation. What I am pushing for is greater clarity on this question, because it will allow me to better tailor my critique towards the other aspects of our permissions plans and designs.

It sounds like you think we need to see a "preliminary barebones UX flow" before we can get the level of clarity that I want. Personally I don't need to see that; I'd be inclined to make decisions about encapsulation before doing any wire-framing or mockups, just for the sake of speed. But that's only an inclination, not a strong opinion. If other people need to see wire-frames or mockups in order to form opinions and come to agreements, then I'm okay with that — so long as we do it very soon. It sounds like the next steps you have in-mind are for us to look at wire-frames or mockups on Tuesday and then make sure we're in agreement on our approach to encapsulation. That's fine with me, but I'd like to request that the design material be sent out in advance of that meeting so that Brent and I have a chance to review it thoroughly before the call. And I would request that we prioritize seeking agreement on this encapsulation question at the start of the meeting, before diving into detail-level conversation about the myriad other UX questions and concerns that I predict we'll all inevitably have. I'd expect that call to take at least 1.5 hr.

[pavish]

@seancolsen I've added encapsulation to the spec in this commit: e135607.

My initial hesitation towards adding your comment to the spec was also due to how it was phrased. I've modified it to clearly mention that this is how we'd like the user experience to look like, and not necessarily as a rule of thumb on how the models and code should be structured. I hope this clears things up regarding encapsulation.

[kgodey]

I left some comments (feel free to ignore the formatting related ones, they're just suggestions). I am confused about the state of the spec, though, given the ongoing discussion about encapsulation.

[kgodey]

@pavish please note I updated this list's formatting since the combination of the bullets + lettered list just seemed wrong to me somehow.

[kgodey]

aren't we also removing the ability for users to have DDL permissions to a schema or DB through the UI?

[kgodey]

Can we put the terms in quotation marks for readability?

[kgodey]

Another minor formatting thing, I would put the privilege names in backticks for readability.

[kgodey]

How are we planning to communicate this requirement?

[kgodey]

How will we figure this out?

[kgodey]

I think the spec would be easier to read if we put all future considerations into a separate section at the end rather than within other sections. It will also allow us to find our future plans easier.

[github-actions]

This pull request has not been updated in 45 days and is being marked as stale. It will automatically be closed in 30 days if not updated by then.

[pavish]

I'm going to go ahead and merge this PR in. The wiki is updated with the latest agreed upon product spec (the periwinkle proposal). I do not think this requires any further review since everyone is in agreement.