Validate and keep track of all users JWT token

[pdonorio]

Mainly we have different subtasks:

• Create a table/json inside a database (graphdb at least) to store current release tokens
• Define the standard content for JWT
• Implement token TTL
• Implement a valid mechanism to invalidate a token on user request
• Create a profile page on frontend side to list and invalidate current user's token

[pdonorio]

discussion starts from commit 931134c

[pdonorio]

We must make some decisions on how to use JWT and how to validate them. Here I will write some notes to take these decisions all together.

First of all, citing http://security.stackexchange.com/a/99741, we must remember how JWT are usually used:

JWT best practice is to not use the database or cache at all, the whole idea of JWT is stateless validation check, you can store the user ID within token payload and use it when necessary by several machines without the need to sync a session ID or alike.

Reading around i see that validation based on action is the first reason people give up on the stateless JWT system. I didn't know, and personally I don't like it too much. So here is my summary, after documenting my mind, of the different characteristics that could be implemented:

• Automatic expiration (TTL)
  Compute the expiration (e.g. one day) by summing to now(), save it inside the payload, check every time the token is opened. This doesn't brake anything.
• Expiration at user logout
  When the user logs out add the token to a blacklist inside some DB table. Check at every request the token against the db table...
• Keep track of opened sessions across hosts
  When the token is emitted, it is also saved inside some Whitelist DB table. When users logout or require to remove access to a token, it is simply removed from the table. Check at every request if the token is available inside the whitelist db table......
• Refresh token
  If you use a very short TTL you can easily check informations you want to save inside the Token. But this requires the grant_type/access_token/refresh_token standards from Oauth lib. I think it could take MUCH more time than we have.

[pdonorio]

Also, to remember for the future security, still citing from the source:

Make sure to use long and random user IDs, so if an attacker manages to forge a token, he will only risk one user and will not be able to access other users, unlike sequential IDs.

And, in fact, we have sequential at the moment.

[pdonorio]

@mdantonio we should do something here and close it in the very beginning of the next week

[mdantonio]

And, in fact, we have sequential at the moment.

So we should replace IDs with UUIDs

We can create UUIDs from graph-side by installing with library:

https://github.com/graphaware/neo4j-uuid

Otherwise we can create such IDs from python-side:

>>> import uuid
>>> str(uuid.uuid4())
'830707ea-438f-4d99-b02b-6b40b17a523f'

Note that to force the invalidation of tokens an option is to change the user ID.
In this case the token will be invalid because no user will be re-created back starting from such ID

I don't like very much to change the user ID but we could merge the two things:

• use the numeric progressive ID as immutable identifier (as now, but not used for tokens)
• add a mutable UUID used for tokens

[pdonorio]

we could merge the two things

I like that. So we are going to have id maintained by the database also as autoincrement and token_uuid field which will not match a right user if we invalidate it.

Sounds great!

[pdonorio]

You may start from this model

[mdantonio]

Create a table/json inside a database (graphdb at least) to store current release tokens

It is done. We have to decide which information add to the token (creation date, TTL, IP address?)

Implement a valid mechanism to invalidate a token on user request

We need to modify the verify_token method to check if the uuid contained in the token (to be added) match a node in graph (uuid to be added in the model)

Implement a valid mechanism to invalidate a token on user request

After implemented the UUID the invalidation will only require a refresh of the UUID in the user node

Expiration at user logout
When the user logs out add the token to a blacklist inside some DB table.
Check at every request the token against the db table...

To implement this we could:

• remove the token node
• remove the connection with the user (keeping the token node)
• invalidate the token node by adding/changing some label/property

[pdonorio]

We have to decide which information add

IP, date of expiration, last access, host(?)

For standard information, see this instead:
"Structure of a JWT" in http://blog.apcelent.com/json-web-token-tutorial-example-python.html

[pdonorio]

UUID we must check how we could do it with python to avoid the graph extension

[pdonorio]

remove the connection with the user (keeping the token node)

I like this option

[mdantonio]

• Added properties to token model (creation, expiration, last_access, IP, hostname)
• Added information to token node (creation, last_access, IP, hostname) [expiration still missing, last_access not updated)
• Added abstract methods invalidate_token and invalidate_all_tokens to BaseAuthentication
• Implemented methods invalidate_token and invalidate_all_tokens in GraphDBAuthentication (not used yet)
• Added uuid property to User model
• Added a getUUID method (not yet implemented, fixed return) in GraphDBAuthentication (method used in init_users_and_roles and invalidate_all_tokens
• Modified fill_payload to add the User.UUID (replacing the _id)
• Modified verify_token to retrieve Users based on UUID

What is missing:

• add information to JWT payload
• TTL in verify_token and save_token
• update last_access in token node
• use invalidate_token in Logout
• use invalidate_all_tokens somewhere
• implement getUUID (should be trivial, but sublimetext crashes when i write import uuid and implementation is commented for now)
• implement an endpoint to list active tokens for a user
• implement profile page with list of active tokens and option to revoke tokens
• implement an endpoint for admin to clean the token left hanging after launching automatic tests
• code refactoring (for example getUUID method could be generalized and moved out from GraphDBAuthentication)

I found a problem:

since verify_token is implemented in BaseAuthentication it is not able to make graphdb-specific operations (for example verify che connection with the User)... we have to discuss about this

[pdonorio]

You broke the tests :P

https://travis-ci.org/EUDAT-B2STAGE/http-api/builds/135987324

[pdonorio]

OOps i broke them!

[pdonorio]

should be trivial, but sublimetext crashes when i write import uuid and implementation is commented for now

LOL

[pdonorio]

We may create and endpoint for admin to clean the token left hanging after launching automatic tests.

[mdantonio]

Implemented endpoints:

GET /auth/tokens
GET /auth/tokens/int:id
DELETE /auth/tokens
DELETE /auth/tokens/int:id

[mdantonio]

SESSION_REFRESH_EACH_REQUEST

how can we use it for tokens?

We have to found a compromise between security (short life tokens) and ease of use (long life tokens)

The auto-refresh could be a solution.
We could set a few days expiration (no more than one week) with automatic extension when token is used.

So an unused token will expire in 1 week (good for security)
A frequently used token will be valid for a long time (good for ease of use)

Expiration extensions cannot be implemented if TTL is encoded in payload (change the information will change the token) but we can implement a double TTL system:

In the paypload is encoded a long term TTL (1 year?)
In the database is handled a short term TTL (1 week?)

Only the database TTL can be extended and the payload long term TTL is used as hard limit to avoid immortal tokens

To implement this we can use the "last_access" and "expiration" fields already added to the Token model (and still unused) + the TTL information in the payload (still to be added)

[pdonorio]

Only the database TTL can be extended and the payload long term TTL is used as hard limit to avoid immortal tokens

Should we take in consideration now the oauth2 refreshable tokens?

[mdantonio]

The Oauth2 refresh tokens mechanism has a similar purpose (extend the life of short-term access tokens) but it works with to the exchange of client_id and client_secret, required to obtain a new access token from a refresh token.
Since we are not implementing an oauth2 server, we have not such information.

We could implement refresh code exchangeable with new access tokens with the requirement of no other information... So a explicit token refresh instead on a automatic/implicit refresh as I proposed

This could be a little more secure because if my access token is compromised the problem is time-limited due the short token duration.
But if my access token is compromised probabily also the refresh token will (since they are obtained in the same moment, on the same client and from the same server) so the gain in security is very limited.
As cons the client must be implemented to take in consideration this feature.

My question is why to complicate both the backend and the client implementation to gain very few in security?

[pdonorio]

I agree, too much work for now. Let's go with the first proposal then, 2 TTL inside payload + database.

[mdantonio]

Code refactoring to implement the token payload with TTL
Common payload moved from specific authentication classes to basic authentication class
Added a custom_payload method to add service-specific information in payload

EUDAT-B2STAGE/http-api-base@3dff75f

[mdantonio]

Implementation of TTL fields in payload

Note that jwt.decode checks by it self the token validity by verifying both nbf (not before) and exp (expiration) fields

[mdantonio]

The double TTL workflow is now working (for both graphdb and relational).

You can make some test, then the issue can be closed.

I not really sure about the choice to disconnect invalid tokens instead of delete it... We create a lot of garbage in the graph... maybe in the future you may re-consider this choice.

A point is missing:

implement profile page with list of active tokens and option to revoke tokens

I open a dedicated issue for this sub-sub-task

[pdonorio]

I not really sure about the choice to disconnect invalid tokens instead of delete it

Could we already prepare the method that cleans those disconnect tokens with a Cypher query?

[mdantonio]

Implemented the endpoints

GET /auth/tokensadminonly/UUID

DELETE /auth/tokensadminonly/UUID

Since the role decorator is still missing i used an endpoint name with an explicit intention (admin only), to be modified after the implementation of role restrictions.

To not forget it i added a logger.critical("This endpoint should be restricted to admin only!") when using this endpoint

relational endpoint is not tested (if you test it you can close the issue)

[pdonorio]

Verified!

[mdantonio]

I not really sure about the choice to disconnect invalid tokens instead of delete it... We create a lot of garbage in the graph... maybe in the future you may re-consider this choice.

Finally we decided to modify the invalidate token to delete the node instead of only disconnect it from the user

I'm implementing this modification in graphdb auth, the same approach should be replicated on sql and mongo

[pdonorio]

Todo list here:

• neo4j
• sqlalchemy
• mongo

[mdantonio]

I'm implementing this modification in graphdb auth

Done here:

EUDAT-B2STAGE/http-api-base@347961b