New authentification method

[julien94800]

Hello,
Starting October, current way to connect to Netatmo server will be removed and a new one introduced. We will need to find an other way to connect and keep this plugin working

[julien94800]

To improve the security of our products, we inform you that the Client Credentials grant type method will be completely removed. It will no longer be possible to authenticate with the username and password of the user.

The effective date of this update is October 2022.

How can you authenticate with Netatmo API ?
From this date, the OAuth2 authorization code flow must be followed for authentication.

Extract from a mail I received from Netatmo

[jostrasser]

Adding documentation from mail: https://dev.netatmo.com/apidocumentation/oauth#authorization-code

[skrollme]

Thanks for opening the issue. I also got the infomail from Netatmo. I will take a look at it soon...

[skrollme]

Since I still had no time to take a look at this refactoring and also no good starting point to integrate a web-redirect-based authentication in the plugin, it looks like it won't be done by Netatmo's deadline this October.

So be aware, that this plugin could be stop working and/or breaking your Homebridge setup any time soon.

If anyone is interested in participating here feel free to start coding and opening a PR in this repository.

[jostrasser]

Hi @skrollme
that is really bad... right now there is no alternative to integrate Netatmo Weather into Homebridge. And you have done a really good job :)

I contacted Netatmo Dev Support to get more details about the new auth method, as I saw that users are now able to create an "access token" via the web portal in addition to the "client_id" and "client_token".
This means a web-based redirect is maybe "not a must" for now. You only have to refresh the token periodically:

Our development team are going to allow the creation of the refresh and access token directly from the dev.netatmo.com account owning the application. This means that you will only need to manually generate it once, then you will simply be able to refresh it and save the new ones after refresh.

At the moment the token validity is 3 hours.

Will this makes it easier to keep the plugin "alive"?
I hope this can help to solve the issue.

Thank you!

[jostrasser]

Hi @skrollme
I want to add some more informations from Netatmo:

Note that all script that are actually using the refresh token method instead of fully remaking the client credential (that should be used once to create the token and its access level) will continue to work.

Eitherway for the node if they can save emails and password, saving the key is identical. We will however also need to change both the refresh and access key with each refresh in the near future. They will only need to copy paste the new access and refresh with every refresh. Here is how to refresh the token: https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

[skrollme]

Thanks for your investigation, @jostrasser I will take a look at the possibilities we have with your findings this weekend.

[jostrasser]

Thanks for your investigation, @jostrasser I will take a look at the possibilities we have with your findings this weekend.

Many thanks @skrollme !

[skrollme]

Just checked https://dev.netatmo.com/apps/ and to be honest I'm only able to generate access-token in my app's edit-form. No RefreshToken! And in this case it does not really help unless you want to manually generate a new access-token every 3 hours and put it into your homebridge setup :D

Or did you find out where you can generate access- AND refresh-token, @jostrasser ?

[juani13973]

Hi skzollme, thanks for your effort.
I just created a token and it works again. I am not a technical person so I am not able to have a proper conversation with Netatmo but would it be worth asking if they can make the token last 3 months at least ?

[jostrasser]

@skrollme

The access token can be generated here: https://dev.netatmo.com/apps/
And I found the documentation about the token refresh process here:
https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

I think this "should" be triggered within 3 hours periodically to stay authorized.

[jostrasser]

Hi skzollme, thanks for your effort.
I just created a token and it works again. I am not a technical person so I am not able to have a proper conversation with Netatmo but would it be worth asking if they can make the token last 3 months at least ?

Netatmo confirmed that the Access token will be permanent but the refresh token has to be refreshed within every 3 hours.

Some statements:

Our developement team are going to allow the creation of the refresh and access token directly from the dev.netatmo.com account owning the application. This means that you will only need to manually generate it once, then you will simply be able to refresh it and save the new ones after refresh.

At the moment the token validity is 3 hours.

At the moment you do not need to recreate a new token, you can refresh it ad ifinitum once you have a valid token. This is however also prone to change in the future, also to improve the security of the devices. At the moment I can't confirm if the refresh token, the access token or both will have to be changed every three hours as we are still considering the best option.

What you can do is setting up a server on your device hosting the code and access it via your smartphone or any display you would have. Your mobile phone has to be on the same network than the device hosting your code.

The device should then redirect the call directly to https://api.netatmo/com/oauth2/authorize?client_id=''&redirect_uri=''&scope='' and setting the redirect uri as the locale ip of your device 192.168.xx. It does not need to be accessible from the outside but only on your local network.

On the mobile phone, you'd then be redirected on Netatmo front and it will ask you whether you want to gave access to the API app. When you'll click on 'Yes, I accept', Netatmo servers will send the code to Netatmo frontend which will redirect it to the locale ip from the redirect uri. In the end https://192.168.xx/?code='' will be received by the device hosting your development and will then be able to get the pair of tokens (access tokens& refresh token).

Note that if you already have a valid refresh and access token, you do not need to redo the access.

Feel free to let me know should you have any further request.

I hope this can help you :)

[jostrasser]

I just created a token and it works again.

Hi @juani13973

where did you refreshed the token and what exactly "works again"?
At the moment Netatmo is not restricting the old auth method and will do this starting October.

This means all 3rd party apps / plugins access are not impacted right now but has to be adopted (Oauth2 authentication) to keep them working.

And homebridge-eveatmo is one of it.

[julien94800]

Hello @jostrasser
From my side my log was full of the following message

I just logged into Netatmo dev portal and generated a token. Now it works again in Homebridge. Not sure if it's really this which fixed the situation.... but sounds strange

[jostrasser]

Hello @jostrasser
From my side my log was full of the following message

I just logged into Netatmo dev portal and generated a token. Now it works again in Homebridge. Not sure if it's really this which fixed the situation.... but sounds strange

Ah, I am aware of this issue (run into it a few months ago)
It's a platform issue on the Netatmo servers. Creating a fresh "secret" should solve the issue.

FYI: This isn't related to the auth method change which Netatmo is starting in October.

[juani13973]

Thx julien94800.
Same with me. Same message in the console and home bridge had lost access to fetch the data. The temp, humidity etc did not work. I went into the Netatmo dev website where I had created the connection to the plug in and there is a link to create a token. I just clicked, the token was created and "magic" the plugin started working again.

I guess based on the conversation here that I need to do this token refresh every 3 hours with is impossible. once every 3 month would be OK

[jostrasser]

Hmm, the "access token" (Oauth2) should not be related to your issues because when you generate a token you have to enter it in your json.conf of Homebridge, otherwise it will not be used... and this is not possible right now ;)

But creating a token can force a sync of the authentication change @ Netatmo´s Platform. And this can solve such issues.

THIS token will be needed starting October
To use it we have to adopt some changes in homebridge-eveatmo plugin:

[julien94800]

Hello @jostrasser
Yes that's exactly on this button I clicked. Then I get tokens (I assume the access one and the refresh one separated by |)
Indeed I did not used them anywhere.... but by any reason it unlocked my trouble. Surely something strange on Netatmo server.

Thanks anyway, and let's hope we could find a way to get this plugin still working on future!

[jostrasser]

Hello @jostrasser Yes that's exactly on this button I clicked. Then I get tokens (I assume the access one and the refresh one separated by |) Indeed I did not used them anywhere.... but by any reason it unlocked my trouble. Surely something strange on Netatmo server.

Thanks anyway, and let's hope we could find a way to get this plugin still working on future!

Hi @julien94800
Thanks for your confirmation :)

Yes, I think Netatmo is having troubles right now on their side and the described steps are solving it.
However, I also hope we can find a way to integrate the new authentication method.

[skrollme]

Normally you get access- and refresh-token after auth. You can use the access-token as long as it is valid. If it becomes invalid you need to use the refresh-token to get a new pair of token and everything starts over again.

But when I generate a token on https://dev.netatmo.com/apps/ I only get an access-token, no refresh-token. Not sure if Netatmo does something special here or am I missing anything, but to only have the access-token seems useless for long-term usage.

[jostrasser]

Normally you get access- and refresh-token after auth. You can use the access-token as long as it is valid. If it becomes invalid you need to use the refresh-token to get a new pair of token and everything starts over again.

But when I generate a token on https://dev.netatmo.com/apps/ I only get an access-token, no refresh-token. Not sure if Netatmo does something special here or am I missing anything, but to only have the access-token seems useless for long-term usage.

Hi @skrollme

I have an open ticket at Netatmo's Dev Support. I can request some more informations about it.

[skrollme]

But from your statements it looks like we have the current situation now:

1. refresh-token generation does not work at the moment, but it is coming?
2. access-token (which normally expire) do not expire

If 2) is the case, it could be an easy (temporary) fix to keep the plugin running. I will generate a token now and check if it still works tomorrow.

[skrollme]

@juani13973 and @julien94800
The problems you are describing does not seem to be related to this topic/issue here. Netatmo seems to have some general and unrelated problems with their infrastructure which happens every now and then.

[jostrasser]

But from your statements it looks like we have the current situation now:

1. refresh-token generation does not work at the moment, but it is coming?
2. access-token (which normally expire) do not expire

If 2) is the case, it could be an easy (temporary) fix to keep the plugin running. I will generate a token now and check if it still works tomorrow.

Hi @skrollme
I have requested a confirmation from Netatmo for your questions.

1. waiting for confirmation from Netatmo
2. I think if a access token is created (via dev.netatmo.com) only the refresh token has to be "refreshed" periodically by homebridge-eveatmo in background. Netatmo will change this in the future (not official confirmed now) but for now this procedure should work. If they are changing also this procedure they will inform the customers again (hope so) ;)

[skrollme]

The access-token did not survive the night, so the "easy solution" won't work. I will start evaluating other ways...

[jostrasser]

Feedback from Netatmo Dev Support:

1. refresh-token generation does not work at the moment, but it is coming?
2. access-token (which normally expire) do not expire

---

1. The token generation works, it's the refresh token generation directly from dev.netatmo.com application's webpage that is coming. This is to allow people to generate the token if the machine running the script is a pi for exemple. They will only need to copy/paste the key then refresh the token & save the new token to replace the old one.

2. The access token do expire after 3 hours, this is why the refresh is recommended instead of doing the aouth/cred process all over again.

[skrollme]

I just releases a new beta-version (0.7.0-beta2) of this plugin which uses a refresh_token for authentication.

• The new version only needs a client_id, a client_secret (like before) and a valid refresh_token for this client.
• Upon startup the plugin uses the refresh_token to get an access_token which is then used in all API-calls like getStationData
• The plugin also tries to fetch a new access_token when it is about to expire (every 3h at the moment)

A first and short test went fine so I'm about to install it on my raspberryPi to use it at home because it definitely needs some long-time testing. Everyone who is able to generate a refresh_token (also not officially supported in the https://dev.netatmo.com/apps/ console) is also invited to do this and test the beta-version.

[julien94800]

Thanks very much @skrollme and all other people who helped for this! Great work, im happy being able using it again in future!
Thanks

[childledj]

Today I’ve received email from Netatmo:

Authentication update

Dear Netatmo developer,
As of today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers respond with a couple of tokens : an Access Token and a Refresh Token.If the previous Access Token is still valid, the newly returned access token is identical but its expiration time is extended for 3 hours.In any case, the refresh token is not renewed.
Starting from the 17/04/2023, this behavior will change to to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users.When refreshing tokens, Access Token and Refresh Token will be automatically renewed and former tokens invalidated.
What does it means for me ?If you were already updating the tokens provided when refreshing your tokens, this change will not impact you.If you do not update the refresh token when refreshing your Access Token, your users will be disconnected after 3 hours as the former tokens will become invalidated.To fix it, you need to update the tokens as soon as you get the newly generated ones

Authentication update

Dear Netatmo developer,
As of today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers respond with a couple of tokens : an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned access token is identical but its expiration time is extended for 3 hours.

In any case, the refresh token is not renewed.

Starting from the 17/04/2023, this behavior will change to to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users.

When refreshing tokens, Access Token and Refresh Token will be automatically renewed and former tokens invalidated.

What does it means for me ?

If you were already updating the tokens provided when refreshing your tokens, this change will not impact you.

If you do not update the refresh token when refreshing your Access Token, your users will be disconnected after 3 hours as the former tokens will become invalidated.

To fix it, you need to update the tokens as soon as you get the newly generated ones

[jostrasser]

I think with 1.1.0 we are well prepared for this change... 😉

[childledj]

I think with 1.1.0 we are well prepared for this change... 😉

Hope so!

[skrollme]

I have totally forgotten, that I'm on the beta2 myself 😄 Since I did not have any problems I will prepare the 1.1 release soon and publish it

[jostrasser]

I have totally forgotten, that I'm on the beta2 myself 😄 Since I did not have any problems I will prepare the 1.1 release soon and publish it

Yes, now is the right moment for this I think ;) thank you! :)

[skrollme]

Version 1.1.0 was published a few minutes ago ✅

[Sergbmw]

[3/23/2023, 9:44:51 AM] [eveatmo platform] Loaded homebridge-eveatmo v1.1.0 child bridge successfully
[3/23/2023, 9:44:51 AM] Loaded 0 cached accessories from cachedAccessories.0ECE8DF54E27.
[3/23/2023, 9:44:51 AM] [eveatmo platform] Authenticating using 'refresh_token' grant
[3/23/2023, 9:44:56 AM] [eveatmo platform] Loading new data from API for: weatherstation
[3/23/2023, 9:44:56 AM] [eveatmo platform] Loading new data from API for: airquality
[3/23/2023, 9:44:57 AM] [eveatmo platform] WARN - Netatmo: Error: getStationsDataError error: Invalid access token
[3/23/2023, 9:44:57 AM] [eveatmo platform] WARN - Netatmo: Error: Authenticate refresh error: invalid_grant
[3/23/2023, 9:44:57 AM] [eveatmo platform] WARN - Netatmo: Error: getHealthyHomeCoachData error: Invalid access token

[The-Paw]

Today nothing happen

12.7.2023, 19:42:50HOOBSeveatmo platformWARNUNGWARN - Netatmo: Error: Authenticate refresh error: invalid_grant
12.7.2023, 19:42:50HOOBSeveatmo platformWARNUNGWARN - Netatmo: Error: getStationsDataError error: Invalid access token

What do I select for the "Choose Scopes" on the Netatmo Page ??

[jostrasser]

What do I select for the "Choose Scopes" on the Netatmo Page ??

Hi @The-Paw

Create a fresh token pair. I documented the scopes here:

#62 (comment)

[The-Paw]

What do I select for the "Choose Scopes" on the Netatmo Page ??

Hi @The-Paw

Create a fresh token pair. I documented the scopes here:

#62 (comment)

@jostrasser
Installed Postman ..

https://api.netatmo.com/api/getstationdata .. click "get" -> than:

{
"error": {
"code": 404,
"message": "Not Found"
}
}

[jostrasser]

@The-Paw

And if you try the token generator at https://dev.netatmo.com ? Does this work? I checked my installation right now and all looks fine for me.

[skrollme]

I'm also having these problems by myself lately. But I read somewhere that Netatmo has (even more) API problems in the last weeks so this could be their reason now. At least it did not change anything with my setup and problems started to occur out of nowhere.

[jostrasser]

I'm also having these problems by myself lately. But I read somewhere that Netatmo has (even more) API problems in the last weeks so this could be their reason now. At least it did not change anything with my setup and problems started to occur out of nowhere.

If this issue occurs: is there something to do or will it solve automatically after the API is responding again?

[DANSI]

same here.
All my Netatmo scripts throw errors.
An error happened while trying to retrieve your tokens: bad request
also error unauthorized_client https://api.netatmo.com/oauth2/token
if this is now happen, so many scripts on the Internet need some changes.
Netatmo is down for me since yesterday 07/12/2023 11:00 am

[jostrasser]

same here.

All my Netatmo scripts throw errors.

An error happened while trying to retrieve your tokens: bad request

also error unauthorized_client https://api.netatmo.com/oauth2/token

if this is now happen, so many scripts on the Internet need some changes.

Netatmo is down for me since yesterday 07/12/2023 11:00 am

This is correct but the client credential grant type deprecation is well known in the meantime.

I think the issue we are facing is not related to this right now because homebridge-eveatmo is already using the new auth method with tokens.

A few months ago (18.04.23), Netatmo tries to disable the deprecated grant type and rolled it back after approx. 12 hours because the impact was massive.

I contacted the DEV Support and got the confirmation:

(Netatmo Support, original text)
Teams performed a rollback. Indeed some "legit" third-party apps were unable to perform /token calls. They are investigating to find the root cause, fix it, and re-deploy the change.

I didn't get more informations when they will try it again:

(Netatmo Support)
They prefer to take the necessary time to resolve it, as the change is a minor security feature but can have important undesired side impacts. I don't know for now when the change will be effective again.

So there is maybe a service interruption on the Netatmo Cloud infrastructure. The issue looks selective right now and there is no known maintenance window or global outage reported.

FYI:
I am pulling the data from the Netatmo Cloud with different systems / solutions (Homebridge, Node-Red, python scripts) and right now I cannot see any issues on my side. I am using only the new authorization grant type on all of them without any issues (for now) ;)

[jostrasser]

Ah, one more thing:
Has anyone tried to recreate the tokens to see if this is solving the issue?

[DANSI]

i have created "registered" a new app on dev.netatmo.com. So i got new client_id und client_secrets.
I have also created new tokens, and the errors does still appear.
So i think it could be a problem on cloud-side, as @jostrasser already telled us.
Lets wait a few days, before do more investigation at this problem.
The availability of their servers was never be the best at all time.
have a nice day, folks

[jostrasser]

@DANSI thanks for testing! 👍

[skrollme]

I'm also having massive problems in the last week with both authentication methods (password and refresh-token). Today I created a new app at dev.netatmo.com (with new client id / secret and so on). It is still working for now 🤞

[Smith166]

Hi,
I would much appreciate if anyone can advice me how to fix the authentication issue. :(

[7/16/2023, 9:49:49 PM] [eveatmo platform] ERROR - Netatmo: Error: Authenticate error: unauthorized_client
[7/16/2023, 9:50:46 PM] [homebridge-eveatmo] This plugin is taking long time to load and preventing Homebridge from starting. See https://homebridge.io/w/JtMGR for more info.

My config:
{ "name": "eveatmo platform", "ttl": 540, "extra_co2_sensor": false, "weatherstation": true, "airquality": false, "co2_alert_threshold": 1000, "log_info_msg": true, "auth": { "client_id": "64b444715ca6dba90c098cdb", "client_secret": "6wgJzqXY5zUSxEX5CXocio2YJ3KyolaBuwUCLDDE6", "grant_type": "password", "refresh_token": "55b1182d1c7759ece68b45c4|afe646f0585be615d5ae4b23af5e8222", "username": "xxxxxxxxxxxxxxxxx", "password": "xxxxxxxxxxxxxx" }, "platform": "eveatmo" }

Thanks.

[mrmaximas]

What are we gonna do? According to last Netatmo message May 29, 2024 is deadline.

[Gregor-Weissenborn]

Since about two weeks, it doesn't work for me anymore wit node-red-contrib-netatmo-dashboard.
Is the version 1.1.0 that support the new authentication method comming?

[mrmaximas]

Since about two weeks, it doesn't work for me anymore wit node-red-contrib-netatmo-dashboard.

node-red-contrib-netatmo-dashboard works if you store tokens in the local file system, not in memory.

Since about two weeks, it doesn't work for me anymore wit node-red-contrib-netatmo-dashboard. Is the version 1.1.0 that support the new authentication method comming?

latest version of homebridge-eveatmo works good

[Gregor-Weissenborn]

I use node-red on my raspberry pi for home automation reasons.

I don't know how to configure "store tokens in the local file system" what you recomended.

[mrmaximas]

I don't know how to configure "store tokens in the local file system" what you recomended.

please read from here https://gist.github.com/matz33/843165d0046cac534864b65e633801a5?permalink_comment_id=4785938#gistcomment-4785938