Country codes and other differences between Matomo API and Matomo for WordPress API

[demostanis]

Hello,

I am developping an application which needs our clients' page analytics, such as page visitor count and country.

I was able to develop the part which communicates with the Matomo API as I wanted.
But when I wanted to use the WordPress API, I figured out some things don't work the same as in the Matomo API.

Firstly, when fetching Actions.getPageUrls() (https://mywordpress/index.php?rest_route=%2Fmatomo%2Fv1%2Fapi%2Fprocessed_report&apiModule=Actions&apiAction=getPageUrls&format=json&period=year&date=today&flat=1), the url field is absent, instead replaced by label, which I'm not sure is a strict equivalent.

Secondly, when fetching UserCountry.getCountry() (https://mywordpress/index.php?rest_route=%2Fmatomo%2Fv1%2Fapi%2Fprocessed_report&apiModule=UserCountry&apiAction=getCountry&format=json&period=year&date=today), the code field is absent, without an equivalent this time.

I tried to debug the issue myself, but unfortunately I can't get the docker-compose.yml to work, with logs ranging from "address already in use" to "cannot set siteurl".

I also noticed passing in the URL parameters format=JSON instead of format=json will miss this condition, making the response encoded twice.

Why is the API so different? What are the reasons token_auth authentication was disabled (depsite being used by the Matomo interface)?

Thanks,
demostanis

[diosmosis]

Why is the API so different?

The fields that are missing are treated as row metadata in Matomo. They should be in the output, but if they're not that would be a bug. I'll look into this.

What are the reasons token_auth authentication was disabled (depsite being used by the Matomo interface)?

Matomo for WordPress uses WordPress' authentication methods, so it should work to use an app password.

I also noticed passing in the URL parameters format=JSON instead of format=json will miss this condition, making the response encoded twice.

Thanks for noticing this, we'll fix this in the next release.

[demostanis]

Matomo for WordPress uses WordPress' authentication methods, so it should work to use an app password.

Wouldn't it be easier if it also allowed token_auth for authentication? This way users of my app wouldn't see two separate authentication methods for the same service, could refer to the same doc, those with experience with Matomo wouldn't struggle with Matomo for WordPress...

[demostanis]

Plus some WP plugins (which might be bad, but are in use among WP websites...) restrict the use of application passwords (e.g. Wordfence)

[diosmosis]

For the moment we will be sticking with WordPress' authentication methods.

[demostanis]

The fields that are missing are treated as row metadata in Matomo. They should be in the output, but if they're not that would be a bug. I'll look into this.

Hello, any update?

[diosmosis]

It's still scheduled. Aiming for it to be in a release on Monday NZT.

[diosmosis]

@demostanis can you provide the broken API output you are seeing? Calling API.getProcessedReport with Actions.getPageUrls (with and without flat=1) displays the metadata you're looking for in the <reportMetadata> element for me..

[demostanis]

hello,
i'm not even able to query the API my anymore, i keep getting Date::factoryInTimezone() should not be used with year.

[demostanis]

using the example request in the Matomo for Wordpress docs

[demostanis]

I managed to do the API request, and can see it is in reportMetadata.
Why is it not in the same object as reportData, like in the normal Matomo API?

[diosmosis]

@demostanis The API.getProcessedReport has always looked like that, see for example the API output on demo.matomo.cloud: https://demo.matomo.cloud/index.php?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&token_auth=anonymous&force_api_session=1&filter_limit=10

Row data and metadata are grouped together when accessing the reports directly, for example, accessing Actions.getPageUrls instead of API.getProcessedReport. For the WP REST API you'd need a REST route of /matomo/v1/actions/page_urls.

That said, you should be able to access the Matomo API directly by sending requests to https://yoursite.com/wp-content/plugins/matomo/app/index.php assuming that endpoint is not blocked by your server configuration. You would send the WP app password via a header as you would with any normal WordPress REST request. I'm actually not sure if this will work with the current release, but it definitely will with the next one that will go out today.

If you don't want to send the app password as a HTTP header, I made a recent change that will allow you to pass it in as the token_auth (you would need to set [WordPress] allow_app_password_as_token_auth in your config.ini.php). Also note Matomo will force you to send it as a POST parameter to keep it from being cached.

[demostanis]

@diosmosis thanks for the response.

Why is there no /matomo/v1/actions/page_urls?

I wasn't able to use https://yoursite.com/wp-content/plugins/matomo/app/index.php. I keep getting {"result":"error","message":"Wordpress_TokenAuthMissing"} although I've specified --user in my curl command..

[diosmosis]

Why is there no /matomo/v1/actions/page_urls?

Hmm, it seems they simply weren't added. This was done before I started working on the project, so I can't say why. We'll probably add them in the future, but with direct API access available, it won't be as high a priority.

I wasn't able to use https://yoursite.com/wp-content/plugins/matomo/app/index.php. I keep getting {"result":"error","message":"Wordpress_TokenAuthMissing"} although I've specified --user in my curl command..

Yes, you're right. Apologies, it seems there's one change missing: #1263
If you apply that change you should be able to continue with your work. It will be in another release soon.

[diosmosis]

Ah yes, I almost forgot, after applying #1263, you would need to add [WordPress] allow_wp_app_password_auth to your config.ini.php.

[demostanis]

I'd love to apply #1263, but as I've mentionned in the beginning of this issue, I'm not able to run the project locally. Doing so leads to Error response from daemon: cannot listen on the TCP port: listen tcp4 :80: bind: address already in use since port 80 seems to be exposed by both wordpress and nginx containers.

[diosmosis]

You don't need to use the packaged local environment, you can use any WordPress with the plugin installed and patched. If you're set on using the packaged local environment, you should run npm run compose up wordpress; npm run compose stop, not docker compose directly. After setting relevant values in a new .env file (see .env.default/readme.md for available values, though the list may not be complete).

[demostanis]

When running npm run compose up wordpress; (i was running npm run compose uppreviously), it just hangswaiting for database...`

[diosmosis]

Try running npm run compose up wordpress mariadb if you've configured .env to use mariadb. Also if you're not using linux, that might be an issue.

[demostanis]

When running npm run compose up wordpress mariadb, wordpress container is in an infinite loop of being able to install matomo but failing to install matomo marketplace. I updated submodules and it works. Maybe this should be mentionned in the README?

[demostanis]

I am using Linux

[diosmosis]

Maybe this should be mentionned in the README?

Yes, initializing submodules was not documented, it could be.

Are you still having issues then?

[demostanis]

I had issue with setting PORT=8080 in .env, so I kept the default of 80...
I checkout your branch, but it doesn't seem to work?

% curl --user 'root:aISQ MRGY 7v7P gBYN JvBc DLbC' http://localhost/wp-content/plugins/matomo/app/index.php'?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&token_auth=anonymous&force_api_session=1&filter_limit=10'
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.62 (Debian) Server at localhost Port 80</address>
</body></html>

am I missing something?

[demostanis]

My bad, I needed to curl http://localhost/6.7.2

[demostanis]

% curl --user 'root:aISQ MRGY 7v7P gBYN JvBc DLbC' http://localhost/6.7.2/wp-content/plugins/matomo/app/index.php'?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&token_auth=anonymous&force_api_session=1&filter_limit=10' 
<?xml version="1.0" encoding="utf-8" ?>
<result>
	<error message="You can't access this resource as it requires 'view' access for the website id = 1." />
</result>

[demostanis]

and idSite is not automatically guessed

[diosmosis]

This URL works for me: http://localhost/6.7.2/wp-content/plugins/matomo/app/index.php?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&filter_limit=10

I also have this in the config.ini.php:

[WordPress]
allow_wp_app_password_auth = "1"

[demostanis]

with your URL, i keep getting Wordpress_TokenAuthMissing errors
(although i use --user root:...)

[demostanis]

sorry, i was testing with the wrong branch the second time.

i'm still getting the same issues as the first time, even with your URL:

% curl --user 'root:Op3v FL8u wZXy b1Hy sOJ2 udzX' 'http://localhost/6.7.2/wp-content/plugins/matomo/app/index.php?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&filter_limit=10'                 
<?xml version="1.0" encoding="utf-8" ?>
<result>
	<error message="You can't access this resource as it requires 'view' access for the website id = 1." />
</result>
% curl --user 'root:Op3v FL8u wZXy b1Hy sOJ2 udzX' 'http://localhost/6.7.2/wp-content/plugins/matomo/app/index.php?module=API&format=XML&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&filter_limit=10'                                
<?xml version="1.0" encoding="utf-8" ?>
<result>
	<error message="Please specify a value for 'idSite'." />
</result>

[diosmosis]

It's working for me, using curl with --user. Maybe there is something off with the app password your using.

[demostanis]

I tried twice, with different wordpress instances and app passwords. Why isn't the error message more specific if it is an app password issue?

[diosmosis]

That was a decision made in the core product.

[demostanis]

Lemme list my reproduction steps, so you can tell me if I messed up at any point:

• cd /tmp
• git clone https://github.com/matomo-org/matomo-for-wordpress/
• cd matomo-for-wordpress
• git checkout direct-api-access
• git submodule update --init
• I modify config/common.config.ini.php and add [WordPress] allow_wp_app_password_auth = "1" although that seems to be done in the docker init scripts.
• touch .env
• sudo npm run compose up wordpress mariadb
• wait a bit
• go to localhost/6.7.2/wp-admin and log in with root/pass
• edit the root user
• add a new application password (there is already one named wp_rest, but I can't reveal its value)
• curl --user 'root:xxxxxxxxxxxxxxxxxxxxxxxxxxxxx' 'http://localhost/6.7.2/wp-content/plugins/matomo/app/index.php?module=API&format=XML&idSite=1&period=day&date=yesterday&method=API.getProcessedReport&apiModule=DevicesDetection&apiAction=getType&expanded=1&filter_limit=10' (the URL that works for you)

[diosmosis]

Instead of this file: config/common.config.ini.php, try adding it to /path/to/checkout/docker/wordpress/6.7.2/wp-content/uploads/matomo/config/config.ini.php.

[demostanis]

that seems to be it, thanks.
is there any specific reason allow_wp_app_password_auth = "1" wouldn't be the default?
it seems complicated to ask users to modify a file just to use our service.

[diosmosis]

We take security seriously, all changes to authentication are done with care especially before pushing to all our users.