Understanding Pow + Many Questions around persistent sessions and SSR

[Francesco-Lanciana]

Hello! I'm in the process of seeing if I can switch my auth system to use POW but it's taking a while to wrap my head around Pow and the extensions. I would appreciate if I could have some people look over how I think it works and tell me what I'm not understanding:

By default without any extensions
We have two stores:
Client store = session cookie
Cache = ETS by default

Pow keeps a signed session ID in the client store. When a new request comes in we attempt to fetch the user given the session ID:

• Check client store for the session ID. If none then exit.
• Validate that the session ID hasn’t been tampered with. If yes then exit.
• Use the session ID to fetch the user and metadata from the cache (ETS by default). The metadata contains when the session was created and a fingerprint. By default if you have gone more than 30 minutes than this data will have been deleted and we will exit.
• We check if the session is close to expiring, if it is we renew it.
• This means deleting the session from the cache and the session ID from the client store. We will then generate a new session by creating new metadata (carry the fingerprint, new date of creation), and storing this along with the user in the cache.

Persistent sessions extension
The persistent_session adds another another cookie (called persistent_session by default) and another ETS table by default. This cookie is a refresh token that allows us to renew sessions that have expired. This plug is placed after Pow.Plug.Session and kicks into gear if it detects that no user has been assigned to the connection. In this case it will:

• Check the connection cookies for the refresh token. If none then exit.
• Validate that the token hasn't been tampered with. If yes then exit.
• Use the token to fetch the session metadata and clauses to get the user from the DB. These clauses confuse me but I think the point is that you don't want to store a cached user object in the cache for a refresh token because it is much longer lived, so these clauses provide a way to get a fresh user.
• If we succeed in getting the user and metadata then we go about recreating the session

Questions

1. Why is there no off the shelf Postgres persistent store? If Pow is already generating DB migrations to store the hashed password why wouldn't it just use the DB to store the refresh token as well?
2. Pow doesn't seem to delete the session ID if it was tampered with (Code Link), why?
3. Pow doesn't seem to rollover refresh tokens when they are used to recreate a session, is this true? If so why?
4. Why are the user details stored in the session, why not the user ID? Is it just to save the extra lookup? I don't understand why the user_id isn't stored and the lookup done on each request by default.
5. What happens if request A comes in and causes the session to be refreshed, and before the response is received by the browser another request is made that still has the old session ID? Wouldn't that cause a 401? Does POW help us get around that issue?
6. I struggled to find how the fingerprint was being utilised. What is it actually used for?
7. Why is the refresh token kept in a seperate cookie and not in the session instead?
8. Why is the session ID signed if it is stored in a cookie that are also signed? Isn't this unnecessary?
9. Has anyone used Pow for server side rendering a React app? I currently SSR my entire site (even authorised content if you have permission) because I want to enable content to be made public and it seems like SSR helps Google and others index your content better. To do this I grab the users cookies when making the API calls - effectively impersonating them. The problem is that when server side rendering I'm not in a browser environment so I want to avoid having the session or refresh token cookies renewed. If they were renewed then I can get into a situation where the user loads the site and everything appears to work fine, but any subsequent request to the API will fail because they have an old session ID/refresh token. Is this something POW could support?

Thanks in advance for the help!

[danschultzer]

Work has been brutal so I haven't had a chance to check up on this before now. Thanks for your patience @Francesco-Lanciana.

1. Why is there no off the shelf Postgres persistent store? If Pow is already generating DB migrations to store the hashed password why wouldn't it just use the DB to store the refresh token as well?

Pow has a clear separation between Ecto, Plug and Phoenix. You can easily use Pow in a setup using Phoenix without Ecto. Making the session store depend on Ecto defeats this purpose.

This was the primary reason, though there are a few other reasons for not using Ecto/Postgres:

• The backend cache is built around key-value store, and uses Erlang match specification for lookups. This works really well for ETS and Mnesia, but it's awkward with external storage like postgres. There's current discussion about this in Change Pow.Store.Backend.Base interface for performance #562
• There's also the idea of sessions (including persistent session) being ephemeral. They expire, and they can't be updated after they have been inserted. So keeping it persistent in the DB didn't seem right.

I do realize now that in practice most people would want to keep everything the same place, especially with container based deploys. Looking into making that easier with #562.

2. Pow doesn't seem to delete the session ID if it was tampered with (Code Link), why?

You mean delete the cookie from the client? If yes it's to prevent race condition. We just want to reject invalid tokens. If we delete the cookie from the client, we can end up with a race condition where the session was rolled in one request, but another slow request is using the old session id. It would mean that the cookie is deleted even though it was just updated.

3. Pow doesn't seem to rollover refresh tokens when they are used to recreate a session, is this true? If so why?

The persistent session (refresh token) is single-use. It doesn't need to be rolled every time the refresh token is rolled (though I think that would be more secure). It'll only be rolled when it's used. I think this is safe enough as when a new session is created any previous sessions with the same fingerprint will be deleted. The fingerprint is carried over from the persistent session.

4. Why are the user details stored in the session, why not the user ID? Is it just to save the extra lookup? I don't understand why the user_id isn't stored and the lookup done on each request by default.

#563 addresses this. The current_user could be anything (it could just be the id), since Pow keeps clear separation by not letting the session logic be dependent on Ecto.

5. What happens if request A comes in and causes the session to be refreshed, and before the response is received by the browser another request is made that still has the old session ID? Wouldn't that cause a 401? Does POW help us get around that issue?

Yeah, it would cause 401. Pow doesn't handle this situation, but you could let old sessions live for a short while after they have been rolled. I'm unsure how prevalent of an issue this would be, as it seems very application specific. It would only really happen with apps where the end-user opens multiple tabs in quick succession.

6. I struggled to find how the fingerprint was being utilised. What is it actually used for?

It's described here:

https://hexdocs.pm/pow/1.0.20/Pow.Plug.Session.html#content
https://hexdocs.pm/pow/1.0.20/PowPersistentSession.Plug.Cookie.html#content

It's to track session lifecycle, and ensures that only one session exists throughout the cycle.

7. Why is the refresh token kept in a seperate cookie and not in the session instead?

The plug session expires when the browser session ends.

8. Why is the session ID signed if it is stored in a cookie that are also signed? Isn't this unnecessary?

It is. The session ID is signed to conform with how all other tokens are handled in Pow. It was a mistake to depend so heavily on Plug.Session when I designed Pow. Ideally the Pow.Plug.Session can be stored in any format, and not be dependent on Plug.Session. In that case the Pow session would exist in a separate cookie.

9. Has anyone used Pow for server side rendering a React app? I currently SSR my entire site (even authorised content if you have permission) because I want to enable content to be made public and it seems like SSR helps Google and others index your content better. To do this I grab the users cookies when making the API calls - effectively impersonating them. The problem is that when server side rendering I'm not in a browser environment so I want to avoid having the session or refresh token cookies renewed. If they were renewed then I can get into a situation where the user loads the site and everything appears to work fine, but any subsequent request to the API will fail because they have an old session ID/refresh token. Is this something POW could support?

That's possible, but very dependent on the use case. You can set up a custom auth plug to handle the sessions conditionally. The API guide can also give you an alternative idea for how you can tackle this: https://hexdocs.pm/pow/1.0.20/api.html#content

There has been some who have used Pow with react app, I think most of them just went the API route.

[Francesco-Lanciana]

Hey thanks for taking the time to write this up!

Had a look at #562 and I don't really understand what you mean by "interface requires execution/checking of a match spec" and "converting a match spec into a performant query is difficult or impossible". What's a match spec and why is that the case?
Why is postgres too slow for storing and looking up tokens?

You mean delete the cookie from the client? If yes it's to prevent race condition. We just want to reject invalid tokens. If we delete the cookie from the client, we can end up with a race condition where the session was rolled in one request, but another slow request is using the old session id. It would mean that the cookie is deleted even though it was just updated.

I see, that makes sense. So by that logic would you only ever delete the cookie if the user logged out? Since it would be hard to detect if an old session ID belongs to a slow request.

Yeah, it would cause 401. Pow doesn't handle this situation, but you could let old sessions live for a short while after they have been rolled. I'm unsure how prevalent of an issue this would be, as it seems very application specific. It would only really happen with apps where the end-user opens multiple tabs in quick succession.

That's essentially the approach I took. Was unsure about it though because it felt slightly insecure and hacky (the window of time to allow through is arbitrary). I'm not sure how this isn't hit by more people but for me this seems to happen a lot because I batch requests on the client, sending them all at once with Promise.all.

--

I don't know if I'm alone in thinking this way but it seems like POW is quite powerful and extensible but out of the box would work for very few people. Using ETS/Mnesia doesn't work nicely in production in a lot of use cases. Caching the user object by default is confusing because it can easily become stale - it feels like premature optimisation. From what I can gather it seems like these choices were made so that it doesn't rely on Ecto by default, but how many people actually use Phoenix without Ecto?

[danschultzer]

What's a match spec and why is that the case?

It's erlang match spec: http://erlang.org/doc/apps/erts/match_spec.html

I use that because it's standard erlang, and very performant. The alternative is to use binary namespace, something I did before, but I don't think it's good practice.

Why is postgres too slow for storing and looking up tokens?

I don't think it's too slow for most use cases. It's also used in prod with https://github.com/ZennerIoT/pow_postgres_store

So by that logic would you only ever delete the cookie if the user logged out?

Yeah, it's only deleted in that case.

Was unsure about it though because it felt slightly insecure and hacky (the window of time to allow through is arbitrary)

It is definitely more insecure, but as with all security it's all about your particular setup and threat vectors. You should think in terms of REST for the session handling. The weakest part here is the browser/client.

I'm not sure how this isn't hit by more people but for me this seems to happen a lot because I batch requests on the client, sending them all at once with Promise.all.

I've seen it happen before, but it works for most since they have a standard REST app. It becomes an issue with queuing requests as the cookie has already been set in the request header at that point. There are many ways to get around this, and one way is to permit reuse of old sessions (as you have done it). Another one could be to let the authorization be per batch rather than per request in the batch.

I don't know if I'm alone in thinking this way but it seems like POW is quite powerful and extensible but out of the box would work for very few people. Using ETS/Mnesia doesn't work nicely in production in a lot of use cases.

That's true. Pow was built with full infrastructure control in mind. For container based approach that doesn't allow persistent disk storage it simply doesn't work. It's unfortunate, and something I'm working on to resolve.

Caching the user object by default is confusing because it can easily become stale - it feels like premature optimisation.

Yeah, I agree. It's something I would like to solve with #563. My original thought process was that it's super easy to just add in the plug (I rather do less than more by default), but I've come to understand that for most it makes sense to always pull the data from DB.

From what I can gather it seems like these choices were made so that it doesn't rely on Ecto by default, but how many people actually use Phoenix without Ecto?

There's a very clear separation between Ecto, Phoenix and Plug modules in Pow. If they depend on each other then it'll become a lot more difficult to extend and customize. The drawback is what you see here. It's a balance, and something I constantly try to improve. Coherence is an example of auth framework that mixes them together.

[danschultzer]

#563 has been fixed, though it's a config option you have to turn on, 1.1.0 will have it turned on by default 😄